These are chat archives for ManageIQ/manageiq/performance

29th
Jun 2016
Keenan Brock
@kbrock
Jun 29 2016 14:05
@akrzos do we have any performance numbers around the API? Think I found a good improvement, but unsure how to be sure
Alex Krzos
@akrzos
Jun 29 2016 14:21
@kbrock We started using the REST API for various workloads and so far I'd say there is some things I think could be improved
notably response time on adding a provider
Keenan Brock
@kbrock
Jun 29 2016 14:30
@akrzos but you haven't measured throughput or anything?
Alex Krzos
@akrzos
Jun 29 2016 15:14
@kbrock I haven't been specifically asked to performance test the REST API so i'm just offering what I have currently seen thus far with using it
From a throughput perspecitve I'd say we aren't that great either we have a refresh vms workload that before we implmented the bulk method to refresh multiple tagrets was taking > 20s to push 50 or so vm refresh api requests through
could have been my workload driving code though (python) that could be making that slower
If we were to try and send 50 requests at the same time
I'm assuming httpd would likely queue those requests before hitting the worker hosted inside puma
My bets are if you send more than 256 concurrent requests you'll get 503s if they don't time out
256 == apache prefork default max processes count
Keenan Brock
@kbrock
Jun 29 2016 15:21
thanks
Keenan Brock
@kbrock
Jun 29 2016 21:30
Yay. That feeling when you move/organize code and "don't do anything",
yet you reduce memory usage by ~4% and speed up by ~4%
==> ManageIQ/manageiq#9447
I guess reducing the number of queries run by 6% sometimes helps
Jason Frey
@Fryguy
Jun 29 2016 21:31
"don't do anything" ... "9 commits" :laughing:
;) just bustin' your chops @kbrock
Keenan Brock
@kbrock
Jun 29 2016 21:32
lol
let me know when you want me to walk you through the baby steps ;)

what did you do?
@fryguy

Nothing, I didn't change anything! Honest.
@kbrock

9 commits? +53/-20 == nothing?
@fryguy

on a more serious note. I wasn't trying to speed it up, just reorging it so I could understand. and so I could start improving it
and I wasn't even touching the "slow" part of it yet
Nick LaMuro
@NickLaMuro
Jun 29 2016 21:52
So I am basically thinking on the reports page, we don't display any nodes that have > children, and just tell the user to use the interface on the left
because trying to build a 10000+ tree is all done in ruby (regardless of how I can limit the N+1 queries), and that still takes 50ish seconds...
Dan Clarizio
@dclarizio
Jun 29 2016 23:07
@NickLaMuro I've had some ideas about that in the past, where we only show a single node for the reports that have 1 or more actual report runs, then when you click on that, we show the detailed reports on the right only . . . we would have to rework some of the selection logic in the UI (I think clicking on the right actually does a tree select on the left), but that would alleviate a lot of pain in this area for sure
@NickLaMuro and we may want to put a "last run" date/time on the tree node, just so the user knows if they want to look at certain ones
@NickLaMuro and for other trees, we could consider similar treatment, but need to do that on a case by case basis
Nick LaMuro
@NickLaMuro
Jun 29 2016 23:10
Yeah, just something I am tossing around at the moment, and doing a bit of research
it is slow... when you have a large data set like that waiting between requests though :disappointed:
Dan Clarizio
@dclarizio
Jun 29 2016 23:11
doesn't help that we don't have any automatic purging for old reports . . . well, maybe, but I think it's global, so have to set it to the oldest report you want to save globally, not by report type
I nice, relatively easy, enhancement for next version might be to add a "keep_days" to the report object and have a scheduled task that goes thru and removes old reports
Nick LaMuro
@NickLaMuro
Jun 29 2016 23:13
well, don't worry, I don't plan on going crazy and making a new feature with out talking about it :)
Dan Clarizio
@dclarizio
Jun 29 2016 23:14
agreed!
but ping me if you want me to champion the "no reports in the tree" idea
I can vet it out to make sure we aren't losing any needed functionality
Nick LaMuro
@NickLaMuro
Jun 29 2016 23:14
yeah, I will ping you on that PR when I have something that actually works/helps
it will be a 20ish line change tops, and only affect a small number of screens, but ideally only one
Dan Clarizio
@dclarizio
Jun 29 2016 23:15
cool ... outa here, have a good one
Nick LaMuro
@NickLaMuro
Jun 29 2016 23:15
see yah
Keenan Brock
@kbrock
Jun 29 2016 23:33
@NickLaMuro building the json and sending it down is a bit slow. Getting rid of the children on the left sounds like a great option.