These are chat archives for ManageIQ/manageiq/performance

1st
Sep 2016
Keenan Brock
@kbrock
Sep 01 2016 00:07
so you prefer ENV['TURBO']=true? :trollface:
Jason Frey
@Fryguy
Sep 01 2016 00:27
Fun fact, the turbo button used to actually show down the CPU... https://en.m.wikipedia.org/wiki/Turbo_button#Effect
I laugh inside everytime it's suggested :laughing:
And "show down" should have been "slow down"
Nick LaMuro
@NickLaMuro
Sep 01 2016 00:43

@kbrock SERIOUSLY, LOOK AT THIS TOMORROW!!!1!

Opened up a PR related to the /vm_infra/explorer, but shouldn't conflict with what you are doing (hopefully only augment): ManageIQ/manageiq#10929

There is some fun repeated ancestry look up stuff that is the next hotspot that could use some optimization, but that is probably even less straight forward...
Nick LaMuro
@NickLaMuro
Sep 01 2016 00:54
@Fryguy @chessbyte THAT GOES FOR YOU TWO AS WELL!!1!
but nice catches, will fix when I am home
Jason Frey
@Fryguy
Sep 01 2016 00:57
OK FINE
Nick LaMuro
@NickLaMuro
Sep 01 2016 02:02

@dmetzger57 Look at what you made me do! This is what happens when you "Braaaag" about a silly 100x performance on a single route in the sprint review! Now probably my next 5 PRs are going to be duds. Thanks alot... geez...

blames other's for his inadequacies... also, seriously, kidding here...

For those following along at home, see Exibit A (ManageIQ/manageiq#10929) Fig(1) for an explaination.

still don't know where that manifest.json is hiding though...
Keenan Brock
@kbrock
Sep 01 2016 03:05
@NickLaMuro LOL. life is good
Jason Frey
@Fryguy
Sep 01 2016 14:42
IT'S ALL GOOD @NickLaMuro <3
Nick LaMuro
@NickLaMuro
Sep 01 2016 14:45
If the above wasn't a indicator enough, I my humor is based on theatrics and over exaggeration
Dennis Metzger
@dmetzger57
Sep 01 2016 14:48
:trophy:
Deven Phillips
@InfoSec812
Sep 01 2016 14:56
Hello all... Newbie to ManageIQ... Seems like when I deploy via docker the app is REALLY slow... Is there something I need to do to improve performance? Can the application be run using JRuby?
Chris Arcand
@chrisarcand
Sep 01 2016 14:58
Unsure about Docker but no, cannot be run using JRuby.
Deven Phillips
@InfoSec812
Sep 01 2016 14:59
:(
Bummer
Are there other general performance configuration options I can use to improve things?
I'm running it on a Dual E5540 server with 32GB of RAM and it's pretty clunky.
Nick LaMuro
@NickLaMuro
Sep 01 2016 15:00
@InfoSec812 I assume this is "Docker to run MIQ", not "the Docker Provider for MIQ", correct?
Probably better that this convo is done in the main room, since other's have chimed in there
Joe Rafaniello
@jrafanie
Sep 01 2016 15:05
how many cores and memory does the appliance get while running within docker? @InfoSec812. In virt land, I think we default to 4 cores, 6 GB or something in that ballpark
Deven Phillips
@InfoSec812
Sep 01 2016 15:41
I am not limiting the cores or ram on the container.
We're looking to manage about 20 VMware vcenters
Deven Phillips
@InfoSec812
Sep 01 2016 15:47
Total of about 600 hypervisors, and perhaps 5000-10000 vms
And right now performance talking to just a single vCenter and about 12 hypervisors is abysmal.
Keenan Brock
@kbrock
Sep 01 2016 16:01
Think you'll need more than 1 appliance to manage 10,000 VMs
@InfoSec812 have you tried using the VM appliance to do this?
That has been our primary release platform for years. Docker is relatively new.
Having said that, thanks for raising this issue. lets get it fixed.
Deven Phillips
@InfoSec812
Sep 01 2016 17:04
@kbrock Yeah, we definitely intend to use more than one instance of MIQ to manage the whole thing.. Right now it's a proof-of-concept.
But if we cannot resolve the issues with the PoC, it will never get that far.
Oleg Barenboim
@chessbyte
Sep 01 2016 17:06
@InfoSec812 so 2 points
  • I think you will have to be MUCH more specific than "ManageIQ is slow"
  • help here is on a best-efforts, non-paid basis
Deven Phillips
@InfoSec812
Sep 01 2016 17:18
@chessbyte Understood... I'm am just asking for general best-practices for performance tuning MIQ...
Alex Krzos
@akrzos
Sep 01 2016 17:19
@InfoSec812 that will be evironmentally dependent
Oleg Barenboim
@chessbyte
Sep 01 2016 17:32
@InfoSec812 we have NO experience yet with docker-based deployments of ManageIQ, as that is pretty new. Most large customers run our appliances as VMs in one of their on-premise infrastructures. So, in your case, I suggest trying to run the appliance as a VMware VM
Joe Rafaniello
@jrafanie
Sep 01 2016 17:46
with that said, though, enabling many roles and connecting a bunch of providers on the same appliance is a recipe for disaster ;-)
and each provider are not created equal in terms of their size, latency talking to them, or any overhead of the different APIs we use to talk to them
Keenan Brock
@kbrock
Sep 01 2016 18:09

VmInfra.all + TemplateInfra.all
Generates

SELECT "vms".*
FROM "vms"
WHERE "vms"."type" IN (
  'ManageIQ::Providers::InfraManager::Vm', 'ManageIQ::Providers::Microsoft::InfraManager::Vm',
  'ManageIQ::Providers::Redhat::InfraManager::Vm', 'ManageIQ::Providers::Vmware::InfraManager::Vm',  'VmXen')
 AND "vms"."template" = 'f'

SELECT "vms".*
FROM "vms"
WHERE "vms"."type" IN (
  'ManageIQ::Providers::InfraManager::Template', 'ManageIQ::Providers::Microsoft::InfraManager::Template',
  'ManageIQ::Providers::Redhat::InfraManager::Template', 'ManageIQ::Providers::Vmware::InfraManager::Template',
  'ManageIQ::Providers::Openstack::InfraManager::Template', 'TemplateXen')
AND "vms"."template" = 't'

Active Record generated the in list for us.
Ideas how I can generate this in list?
(I basically want VmOrTemplateInfra.all)

Nick LaMuro
@NickLaMuro
Sep 01 2016 18:11
SQL UNION?
Tim Wade
@imtayadeway
Sep 01 2016 18:12
I wonder if there's a Disaster Cookbook in the O'Reilly series
Keenan Brock
@kbrock
Sep 01 2016 18:12
We separate VmOrTemplate from Vm and Template - based upon a flag
but then in a lot of places of our code we don't want the distinction
so we end up bringing back a lot of records, then shipping back to the server with the ids for rbac
Chris Arcand
@chrisarcand
Sep 01 2016 18:13
You can do that with a union, yes.
Keenan Brock
@kbrock
Sep 01 2016 18:13
I'm not a big fan of Model.x + Model.y
union doesn't work b/c the base model is different
ooh
I tried VmInfra.all.or(TemplateInfra.all)
that isn't a "union"
Chris Arcand
@chrisarcand
Sep 01 2016 18:14
Nope, no native union support in AR. Not hard though.
Keenan Brock
@kbrock
Sep 01 2016 18:14
can I manually generate that in clause?
Chris Arcand
@chrisarcand
Sep 01 2016 18:14
More importantly, do you expect them to still have the two class distinctions in your final result list?
Keenan Brock
@kbrock
Sep 01 2016 18:14
I see a few places say subclasses.collect(&:name) but it isn't accurate
Nick LaMuro
@NickLaMuro
Sep 01 2016 18:15
Hey look, I think we have found ourselves a QueryObject, eh? (half :trollface: 'ing)
Keenan Brock
@kbrock
Sep 01 2016 18:15
hmm. think they sort afterwards, so no
lol
ems_Common, vm_common, api/base_controller/providers, all seem to do this
ci_processing...
Nick LaMuro
@NickLaMuro
Sep 01 2016 18:16
Not sure if this has to deal with "Rbac and Friends" (new series on ABC Family)
Keenan Brock
@kbrock
Sep 01 2016 18:16
ugh, ext_management_system has this on steroids
lol
Keenan Brock
@kbrock
Sep 01 2016 18:21
aah, think it is descendants. so the places that have subclasses in our code are wrong :(
Keenan Brock
@kbrock
Sep 01 2016 18:47
Can a Template be archived and/or orphaned?
Joe Rafaniello
@jrafanie
Sep 01 2016 19:44
hey @kbrock, we call role_allows just a few times per screen :trollface: