These are chat archives for ManageIQ/manageiq/performance

2nd
Jan 2018
Keenan Brock
@kbrock
Jan 02 2018 15:16
@NickLaMuro cool - yea, that one must be contributing to the issue - if it isn't the main one
Keenan Brock
@kbrock
Jan 02 2018 16:29
@dmetzger57 looking at memory of generic worker (running c&u) from 12/22 - 1/2 -- think the bump at the end is when it ran out of disk space in /var - but it looked under control
evm-all.png
Keenan Brock
@kbrock
Jan 02 2018 16:36
seeing the same with the priority worker (which will essentially be the same workload as far as generic is concerned)
evm-all.png
Dennis Metzger
@dmetzger57
Jan 02 2018 16:37
@kbrock what code version?
Keenan Brock
@kbrock
Jan 02 2018 16:38
same one nick is running - I forked his vm... looking 5.9.0.9
aah - majority of work in priority is automate and MiqServer.status_update - ignoring that, it did do 4x the number of perf_capture_timer entries (which is pretty minimal)
waiting for results from processing the c&u worker logs (script is slow - hadn't tried it on so many days)
Dennis Metzger
@dmetzger57
Jan 02 2018 16:42
@kbrock hmm, so Himanshu reported the leak on 5.9.0.4 (and provided data/chart) and with 5.9.0.9 it doesn't reproduce. Of course he was using OSP and you're on VMware
Keenan Brock
@kbrock
Jan 02 2018 16:43
I thought I was reproducing it
ugh - when the other graphs come in - I'll see if the counts of the number of C&U match what we think they should
Keenan Brock
@kbrock
Jan 02 2018 19:02
@dmetzger57 have we found a different simulator config that doesn't assign all VMs to a single storage?
Dennis Metzger
@dmetzger57
Jan 02 2018 19:08
I've not, but I've also not specifically looked / tried. I recall Alex started looking but didn't come up with a "solution".
Keenan Brock
@kbrock
Jan 02 2018 19:08
thnx
my metrics collector takes up 1.6G
and has 4million objects in memory
generic worker is only 2.5M
Dennis Metzger
@dmetzger57
Jan 02 2018 19:10
you sure about that "2.5M" for the Generic?
Keenan Brock
@kbrock
Jan 02 2018 19:10
looking at MiqGenericWorker for 12/22 - 1/03
it takes up 1.2G ram, and 2.5M live objects
old objects ~1.5M objects
good find
live + old = 4M
vs MetricsCollector, live = 4M objects, old = 1.4 objects - total = 5.3
ooh - nah - I was right before
all objects on the heap are just shy of 4 million
live = 3.3, old = 1.2 -- how does 3.3 + 1.2 == just shy of 4million objects?
evm-20180103.png
Nick told me that those really big spikes are due to the single storage issue
the storage capture code
Jason Frey
@Fryguy
Jan 02 2018 19:14
FYI ManageIQ/manageiq#16692
;)
Dennis Metzger
@dmetzger57
Jan 02 2018 19:15
:smile:
Keenan Brock
@kbrock
Jan 02 2018 19:16
cool. yea I haven't even gotten to the rollup code. still stubbing my toes on the collector
Keenan Brock
@kbrock
Jan 02 2018 22:07
@dmetzger57 is @NickLaMuro returning tomorrow?