These are chat archives for ManageIQ/manageiq/performance

29th
Sep 2015
Alex Krzos
@akrzos
Sep 29 2015 15:26
@dmetzger57 I built a testbed to try and replicate the purge spiral for you with 5.4 code
@dmetzger57 I'll let you know if I see that impact
Your working on that one right?
Also @kbrock should be able to see perf_capture_timer scheduling issues if the scale is large enough
And ftw ttrying to have grafana show us all out system metrics for it
Oleg Barenboim
@chessbyte
Sep 29 2015 15:27
@akrzos are performance issues getting reviewed/addressed since @dmetzger57 started focusing on them?
Alex Krzos
@akrzos
Sep 29 2015 15:29
@chessbyte Yes dennis already provided code that provides immense savings on large scale vmware providers refreshes, and he provided the same change for larger rhevm providers as well
Oleg Barenboim
@chessbyte
Sep 29 2015 15:30
@akrzos glad to hear - I know we had nobody focused on performance/scalability issues before - making your job of uncovering issues not too interesting -- hoping the situation is changing for the better now
Alex Krzos
@akrzos
Sep 29 2015 15:31
@chessbyte definitely, and with @kbrock and @Fryguy hopefully we can find a suitable TSDB for C&U to make that more scalable
Oleg Barenboim
@chessbyte
Sep 29 2015 15:32
@akrzos I will probably be in RDU Nov 10-12 -- hoping we can catch up f2f then
Alex Krzos
@akrzos
Sep 29 2015 15:32
I have influxdb being fed by collectd and displaying graphs from grafana, looks way better than other realtime performance metrics graphing tools we have used
Jason Frey
@Fryguy
Sep 29 2015 15:34
akrzos, have you tried my changes to get the C&U data into influxdb?
Dennis Metzger
@dmetzger57
Sep 29 2015 15:34
@akrzos I am indeed working on the purge spiral, was slightly sidetracked by region rollups being broken
Alex Krzos
@akrzos
Sep 29 2015 15:35
@Fryguy I have not, but we can standup a small environment with it to get a feel for it
@Fryguy Better to use 5.4? or a 5.5 alpha build for that?
@dmetzger57 another easy one (At least I think it might be easy) would be solving this one - https://bugzilla.redhat.com/show_bug.cgi?id=1085988
Dennis Metzger
@dmetzger57
Sep 29 2015 15:40
@akrzos will followup with you on the easy one :smile:
Alex Krzos
@akrzos
Sep 29 2015 15:41
Cool Thanks :smile:
Jason Frey
@Fryguy
Sep 29 2015 15:44
anything will work
it doesn't really change the collection at all
it just injects itself right before we write to the Postgres DB, and writes to the influx db as well
what I think it will really gain us is that rollups :sparkles: go away :sparkles:
Alex Krzos
@akrzos
Sep 29 2015 16:00
@Fryguy Well I haven't entirely figured out the rollup situation with influxdb yet, purging definitely we get for free (Retention policy)
Well not free
but perhaps at lower cost than our current purging
continuous queries is where we should get rollups from
Jason Frey
@Fryguy
Sep 29 2015 16:02
rollups are done at query time
Alex Krzos
@akrzos
Sep 29 2015 16:02
yeah but we wouldn't want super granular data to live for 6 months
Jason Frey
@Fryguy
Sep 29 2015 16:02
you tack on a rollup tie frame to the query...like TIME(30s) or whatever the syntax is
oh that's retention policies
but yeah it does live for 6 months
Alex Krzos
@akrzos
Sep 29 2015 16:03
understood, but from what I can tell the retention policy applies at a database level
Jason Frey
@Fryguy
Sep 29 2015 16:03
but supposedly, it can handle it. ¯\_(ツ)_/¯
Alex Krzos
@akrzos
Sep 29 2015 16:03
I think with 0.8 you used shard spaces
so the way I understood 0.8 was retention policies applied at shard space level thus you created continuous queries to create the rolled up data into a shard space with a longer retention period
Jason Frey
@Fryguy
Sep 29 2015 16:06
ah ok
Alex Krzos
@akrzos
Sep 29 2015 16:06
I basicly need to figure out how thats solved with 0.9
if it is
Alex Krzos
@akrzos
Sep 29 2015 17:12
Looks like retention policies can get more specific and thats what we need for downsampling to less granular data, but more historical storage of that data - https://github.com/influxdb/influxdb/blob/72f52d44c9f615b728b76d9cd06a3956963e3695/services/continuous_querier/continuous_queries.md