These are chat archives for ManageIQ/manageiq/performance

15th
Sep 2015
the python talk discusses why that might occur in python, curious if ruby is also susceptible
Jason Frey
@Fryguy
Sep 15 2015 13:29
Which refresh?
Dennis Metzger
@dmetzger57
Sep 15 2015 13:35
I don't know the full refresh stack well enough at the moment to have an answer at the moment, interesting data / observtion
Alex Krzos
@akrzos
Sep 15 2015 13:36
@Fryguy Thats a provider delta refresh but I believe it is observable on initial refreshes as well
akrzos @akrzos digs up an initial refresh to see
Dennis Metzger
@dmetzger57
Sep 15 2015 13:37
is that a run using MIQ Master?
Alex Krzos
@akrzos
Sep 15 2015 13:37
5.4.2
interesting, on large vs x-large i see more switching on the x-large but hardly any on the large during an initial refresh http://pbench.perf.lab.eng.bos.redhat.com/results/dhcp23-97/2015-09-12_11:26:15.667329-Refresh-Broker/Provider-Broker-Init-vmware-large-0000/tools-default/dhcp23-97/sar/cpu.html
xlarge is painful to load in chrome, sample interval is too quick (1s)
Dennis Metzger
@dmetzger57
Sep 15 2015 13:51
@akrzos thanks for the Tuesday mystery
Alex Krzos
@akrzos
Sep 15 2015 13:51
@dmetzger57 no problem
@dmetzger57 I have also discovered on my simulated rhevm providers that the ovirt-engine-history-dwhd is adding variation to refresh on rhevm so been disabling that service for refresh tests
I think it doesn't scale for large scale rhev clusters
but yet another thing to investigate
Matthew Draper
@matthewd
Sep 15 2015 13:58
The only thing that talk mentions about processor affinity is in answer to a question: "it probably wouldn't help", "you probably don't want to do that" :confused:
Alex Krzos
@akrzos
Sep 15 2015 13:59
possibly re-run test and tune vcpus from 4 to 1 to see if similiar performance gain as shown from the beginning as an experiment
Alex Krzos
@akrzos
Sep 15 2015 16:06
@Fryguy Opened that RFE bz against RHEV - https://bugzilla.redhat.com/show_bug.cgi?id=1263357
if you have any detail to add that would be appreciated
Jason Frey
@Fryguy
Sep 15 2015 16:09
Nice
I'll add in a little detail on how the querying works so they understand why we are doing it that way
Alex Krzos
@akrzos
Sep 15 2015 16:16
Thanks
Jason Frey
@Fryguy
Sep 15 2015 16:22
ok updated
Alex Krzos
@akrzos
Sep 15 2015 19:19
Has there been any reason to have the worker processes not named to the actual worker name and displayed as such in top/ps such that per say a GenericWorker would show up in ps as MiqGenericWorker rather than /var/www/miq/vmdb/lib/workers/bin/worker.rb
or at least have a cli arguement showing the worker type in ps/top?
Matthew Draper
@matthewd
Sep 15 2015 19:20
worker.rb doesn't work without a CLI argument containing the worker type
Jason Frey
@Fryguy
Sep 15 2015 19:20
you know I was just thinking that the other day
Matthew Draper
@matthewd
Sep 15 2015 19:20
So I'd expect the latter to be there, as long as ps is showing the full command
Jason Frey
@Fryguy
Sep 15 2015 19:20
we should be able to tell the OS the process name
Matthew Draper
@matthewd
Sep 15 2015 19:21
(though I do agree that mangling $0 would be a good thing to do)
Alex Krzos
@akrzos
Sep 15 2015 19:21
[root@dhcp23-232 ~]# cat /proc/632/cmdline
/var/www/miq/vmdb/lib/workers/bin/worker.rb
Jason Frey
@Fryguy
Sep 15 2015 19:22
Yeah, we could give something descriptive like "Worker GenericWorker - ID / GUID_OF_THE_WORKER" or something like that
@matthewd is that all you do in Ruby? Just mangle $0?
Matthew Draper
@matthewd
Sep 15 2015 19:23
I'm pretty sure it will take care of finding the best way to do the thing
Fryguy @Fryguy tests that
Jason Frey
@Fryguy
Sep 15 2015 19:24
nice!
PR on it's way...this has always annoyed me :)
Jason Frey
@Fryguy
Sep 15 2015 19:25
that's what I thought needed to be done
I'll try that one too
Because I like that: Calling this method does not affect the value of $0.
it's Ruby 2.1+ though
Matthew Draper
@matthewd
Sep 15 2015 19:27
My guess would be that $0 will work in more places, while Process.setproctitle will fail silently on wrong platforms, but yeah, doesn't affect $0
OTOH, if you're inspecting $0 any time other than immediately-upon-process-start, you're probably doing something wrong ¯\_(ツ)_/¯
Alex Krzos
@akrzos
Sep 15 2015 19:27
this would help with collectd on the processes module because right now it can only collect the sum of all "ruby" processes due to the title
Jason Frey
@Fryguy
Sep 15 2015 19:28
akrzos: any recommendations on what you want to see in the proctitle?
I'm assuming some static things from evm:status
I wonder if mangling $0 would affect option parsers
Alex Krzos
@akrzos
Sep 15 2015 19:29
Worker type definitely, I'd imagine something that matches the format we see when displaying all workers through rake evm:status
Jason Frey
@Fryguy
Sep 15 2015 19:29
probably not...I'd assume they'd all use ARGV
do you have an evm:status handy?
Matthew Draper
@matthewd
Sep 15 2015 19:29
The options aren't in $0, so I doubt they're looking there ;)
Jason Frey
@Fryguy
Sep 15 2015 19:29
yeah :)
Alex Krzos
@akrzos
Sep 15 2015 19:30

Zone | Server Name | Status | ID | PID | SPID | URL | Started On | Last Heartbeat | Active Roles
---------+-------------+---------+---------------+-------+-------+-------------------------+----------------------+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
default | EVM | started | 1000000000001 | 63457 | 63463 | druby://127.0.0.1:44538 | 2015-09-09T02:42:49Z | 2015-09-15T19:18:59Z | automate:database_operations:database_owner:ems_inventory:ems_metrics_collector:ems_metrics_coordinator:ems_metrics_processor:ems_operations:event:notifier:reporting:scheduler:smartproxy:smartstate:user_interface:web_services

Worker Type | Status | ID | PID | SPID | Queue Name / URL | Started On | Last Heartbeat
------------------------------------+---------+---------------+-------+-------+-------------------------+----------------------+----------------------
MiqEmsMetricsCollectorWorkerVmware | started | 1000000000015 | 632 | 662 | vmware | 2015-09-09T02:47:42Z | 2015-09-15T19:18:51Z
MiqEmsMetricsCollectorWorkerVmware | started | 1000000000016 | 635 | 663 | vmware | 2015-09-09T02:47:42Z | 2015-09-15T19:18:51Z
MiqEmsMetricsProcessorWorker | started | 1000000000017 | 638 | 669 | ems_metrics_processor | 2015-09-09T02:47:45Z | 2015-09-15T19:18:58Z
MiqEmsMetricsProcessorWorker | started | 1000000000018 | 641 | 668 | ems_metrics_processor | 2015-09-09T02:47:45Z | 2015-09-15T19:18:58Z
MiqEmsRefreshCoreWorker | started | 1000000000011 | 64031 | 64084 | ems_1000000000001 | 2015-09-09T02:45:52Z | 2015-09-15T19:18:48Z
MiqEmsRefreshWorkerVmware | started | 1000000000139 | 51542 | 51552 | ems_1000000000001 | 2015-09-15T17:27:53Z | 2015-09-15T19:18:57Z
MiqEventCatcherVmware | started | 1000000000013 | 64042 | 64099 | ems_1000000000001 | 2015-09-09T02:45:53Z | 2015-09-15T19:18:27Z
MiqEventHandler | started | 1000000000001 | 63750 | 63820 | ems | 2015-09-09T02:43:13Z | 2015-09-15T19:18:59Z
MiqGenericWorker | started | 1000000000003 | 63756 | 63826 | generic | 2015-09-09T02:43:15Z | 2015-09-15T19:18:57Z
MiqGenericWorker | started | 1000000000002 | 63753 | 63828 | generic | 2015-09-09T02:43:16Z | 2015-09-15T19:18:57Z
MiqPriorityWorker | started | 1000000000004 | 63760 | 63803 | generic | 2015-09-09T02:43:02Z | 2015-09-15T19:18:59Z
MiqPriorityWorker | started | 1000000000005 | 63763 | 63806 | generic | 2015-09-09T02:43:04Z | 2015-09-15T19:18:59Z
MiqReportingWorker | started | 1000000000006 | 63768 | 63821 | reporting | 2015-09-09T02:43:13Z | 2015-09-15T19:18:56Z
MiqReportingWorker | started | 1000000000007 | 63771 | 63822 | reporting | 2015-09-09T02:43:13Z | 2015-09-15T19:18:57Z
MiqScheduleWorker | started | 1000000000008 | 63776 | 63815 | | 2015-09-09T02:43:08Z | 2015-09-15T19:18:35Z
MiqSmartProxyWorker | started | 1000000000141 | 13997 | 14012 | smartproxy | 2015-09-15T19:08:34Z | 2015-09-15T19:18:58Z
MiqSmartProxyWorker | started | 1000000000140 | 13994 | 14013 | smartproxy | 2015-09-15T19:08:34Z | 2015-09-15T19:18:58Z
MiqUiWorker | started | 1000000000009 | 63779 | | http://127.0.0.1:3000 | 2015-09-09T02:43:06Z | 2015-09-15T19:18:46Z
MiqVimBrokerWorker | started | 1000000000014 | 64045 | 64151 | druby://127.0.0.1:48726 | 2015-09-09T02:45:54Z | 2015-09-15T19:18:50Z
MiqWebServiceWorker | started | 1000000000010 | 637

damn
Jason Frey
@Fryguy
Sep 15 2015 19:30
wrap it in triple backticks :)
Alex Krzos
@akrzos
Sep 15 2015 19:30
too large i think
Jason Frey
@Fryguy
Sep 15 2015 19:31
ah
Matthew Draper
@matthewd
Sep 15 2015 19:31
The workers spend most of their time asleep, don't they? [running] vs [sleeping] seems useful.
Alex Krzos
@akrzos
Sep 15 2015 19:32
That at least gets worker types separated if you use collectd so long as collectd still sources that proc title correctly
Jason Frey
@Fryguy
Sep 15 2015 19:32
they are usually in started state
Alex Krzos
@akrzos
Sep 15 2015 19:32
my instance of collectd is only seeing "ruby" right now
Matthew Draper
@matthewd
Sep 15 2015 19:32
@Fryguy right, it'd be silly to update the DB every time the process wakes up… but for $0 that's not a problem
Jason Frey
@Fryguy
Sep 15 2015 19:32
but yeah, [starting] vs [started] may be useful
Alex Krzos
@akrzos
Sep 15 2015 19:33
I believe there is two titles since in top you can press 'c' to get the long cmdline
Jason Frey
@Fryguy
Sep 15 2015 19:33
yeah, long command line should give the args
@matthewd Oh you mean process state?...isn't that already in a ps output?
Alex Krzos
@akrzos
Sep 15 2015 19:33
I know if you list the processes when evmserverd is starting you can see the args
Matthew Draper
@matthewd
Sep 15 2015 19:34
@Fryguy I mean worker event loop state, I guess
Jason Frey
@Fryguy
Sep 15 2015 19:34
ah gotcha
Matthew Draper
@matthewd
Sep 15 2015 19:35
"is the worker doing work?"
Jason Frey
@Fryguy
Sep 15 2015 19:36
right