These are chat archives for ManageIQ/manageiq/performance

9th
Sep 2016
Keenan Brock
@kbrock
Sep 09 2016 14:07
@Fryguy looks like only 1 class uses FullTreeBuilder. Ok to collapse it into TreeBuilderVmsAndTemplate?
I can't remember if you were going towards FullTreeBuilder or away from it
Jason Frey
@Fryguy
Sep 09 2016 14:07
no
the point was to convert other trees to use it, so it's a standalone entity
Keenan Brock
@kbrock
Sep 09 2016 14:08
I want to go more towards the get tree custom kids model
don't want to load ALL trees for all EMS in one swoop. and the way it is written makes it tricky for me
Jason Frey
@Fryguy
Sep 09 2016 14:08
that's fine, but why collapse...you get no benefit, and if on your journey you decide other trees need it, then you have to uncollapse
Keenan Brock
@kbrock
Sep 09 2016 14:09
well. you saw what collapsing w/ rels did
ok
thanks
so the answer is we are going towards using it
I may tweak the api in there then.
(will try my best to do nothing)
Jason Frey
@Fryguy
Sep 09 2016 14:09
sort of? The answer is more that there is a nice division of class responsibilities, so why change it
Keenan Brock
@kbrock
Sep 09 2016 14:09
rewrite:
I'll keep the division
Jason Frey
@Fryguy
Sep 09 2016 14:10
the base class understands taking a tree of data and converting to UI formats...the subclass is responsible for retrieveing the data
got it
Keenan Brock
@kbrock
Sep 09 2016 14:10
the current class assumes we want to load every node of the tree up front
Jason Frey
@Fryguy
Sep 09 2016 14:10
right, that's why it's named FullTreeBuilder :sweat_smile:
Keenan Brock
@kbrock
Sep 09 2016 14:10
right, and downloading 1,000's of nodes... it is kinda slow
so we need a more granular approach
but the idea of converting a grouping of nodes is very good
Jason Frey
@Fryguy
Sep 09 2016 14:11
I'm missing something here
Keenan Brock
@kbrock
Sep 09 2016 14:11
getting reference. one sec
Jason Frey
@Fryguy
Sep 09 2016 14:12
here's my original concept on how I think the classes should be structured: https://github.com/ManageIQ/manageiq/blob/master/app/presenters/full_tree_builder.rb#L1-L9
I don't mind adding a 3rd or fourth tree builder style depending on the situation
then you just pick the right tool for the job
Jason Frey
@Fryguy
Sep 09 2016 14:12
I don't believe there's a one-size-fits-all tree builder
Keenan Brock
@kbrock
Sep 09 2016 14:13
on that line, I'd like to return a vmems object
Jason Frey
@Fryguy
Sep 09 2016 14:13
I don't understand that
Keenan Brock
@kbrock
Sep 09 2016 14:13
and then it would call into x_get_tree_custom_ems
Jason Frey
@Fryguy
Sep 09 2016 14:13
that line returns the roots of the trees
which aren't vms
Keenan Brock
@kbrock
Sep 09 2016 14:13
sorry
mistype
it gets a list of ems
THEN it gets all the nodes in the ems
seems the roots are just the ems
and if we need to expand an ems, then it can get all the folders in that ems
Jason Frey
@Fryguy
Sep 09 2016 14:14
yes, roots are emses
so you want to move it back to the walking tree builder, is what I hear
Keenan Brock
@kbrock
Sep 09 2016 14:14
but the code forces you to download everything under an ems
no
um
Jason Frey
@Fryguy
Sep 09 2016 14:14
or...on-demand tree builder or whatever
Keenan Brock
@kbrock
Sep 09 2016 14:15
um
but not on demand per level
Jason Frey
@Fryguy
Sep 09 2016 14:15
partially on-demand?
Keenan Brock
@kbrock
Sep 09 2016 14:15
it is much more efficient to lookup everything within an ems in 1 fell swoop
that concept you introduced is great
but having to do 1 fell swoop for every ems.
hmm
Jason Frey
@Fryguy
Sep 09 2016 14:16
it already does one fell swoop
partial tree builder is sort of the second bullet point in that comment
Joe Rafaniello
@jrafanie
Sep 09 2016 14:17
@dmetzger57 @NickLaMuro @kbrock Heads-up, bundler 1.13 was released which contains some resolution improvements but also the git based gem fix that I worked with @Fryguy on.... should drop idle worker memory (console and server too) by around 10-15 MB each: bundler/bundler#4618
Jason Frey
@Fryguy
Sep 09 2016 14:17
@jrafanie :heart_eyes:
We should bump ASAP
Keenan Brock
@kbrock
Sep 09 2016 14:17
I want:
  has_kids_for EmsInfra, [:x_get_tree_ems_children, :options]
  def x_get_tree_roots(count_only, options)
    objects = count_only_or_objects_filtered(count_only, EmsInfra, "name", :match_via_descendants => VmOrTemplate)
    objects.collect! { |o| x_build_node(o, nil, options.dup) } unless count_only
    root_nodes = count_only_or_objects(count_only, x_get_tree_arch_orph_nodes("VMs and Templates"))

    objects + root_nodes
  end
  def x_get_tree_ems_children(object, count_only, options)
    # if it got here, then always show the ems children (they only care if count > 0)
    if count_only
      Rbac.filtered(object.vms).size
    else
        TreeBuilderVmsAndTemplates.new(object, options).tree.children.first
    end
  end
Dennis Metzger
@dmetzger57
Sep 09 2016 14:17
:clap:
Keenan Brock
@kbrock
Sep 09 2016 14:17
instead of putting the TreeBuilder in the get roots
so we can do expand logic, only load 1 or 5 ems
Joe Rafaniello
@jrafanie
Sep 09 2016 14:18
@dmetzger57 Do you have upstream appliances at 1.12 that you can baseline and then force gem install bundler -v1.13 for comparison?
Jason Frey
@Fryguy
Sep 09 2016 14:18
@kbrock I'm very lost on what you are trying to do
Keenan Brock
@kbrock
Sep 09 2016 14:18
I have 15 ems. the user only is really in 1 of them
Jason Frey
@Fryguy
Sep 09 2016 14:18
To me it sounds like a partial preload of something in the tree, and then you want to load on demand as the user clicks
Dennis Metzger
@dmetzger57
Sep 09 2016 14:19
@jrafanie I don’t have one at the moment but I can get one
Keenan Brock
@kbrock
Sep 09 2016 14:19
@Fryguy yea
Jason Frey
@Fryguy
Sep 09 2016 14:19
ah, so you want to essentially just not load the other 14 trees
Keenan Brock
@kbrock
Sep 09 2016 14:19
+1
Jason Frey
@Fryguy
Sep 09 2016 14:19
then, when the user clicks a diffeernt EMS, load the whole tree?
Keenan Brock
@kbrock
Sep 09 2016 14:19
and you basically already wrote the infrastructure
+1
the current issue is the ondemand doesn't allow you to return 5 layers of the nodes tree
Jason Frey
@Fryguy
Sep 09 2016 14:20
ok, that should be doable as is...the TreeBuilderVmsAndTemplates only builds for 1 tree...so just skip it for trees that are not marked with [:open_nodes]
Keenan Brock
@kbrock
Sep 09 2016 14:20
ugh, typing no so good today :(
+1
but the current model
if you don't pass children back
Oleg Barenboim
@chessbyte
Sep 09 2016 14:20
@kbrock we are having meeting on Tuesday with JohnHardy, Jason, DanC, Serena and Dennis on potentially revamping the whole VM Explorer tree - so stay tuned.
Keenan Brock
@kbrock
Sep 09 2016 14:20
then it assums there are no children
oleg - that is for vms / end nodes
I'm looking at all the middle infrastructure
having similar problems over in services
Oleg Barenboim
@chessbyte
Sep 09 2016 14:21
I want to discuss all the trees
Jason Frey
@Fryguy
Sep 09 2016 14:21
what is middle infrastructure?
Keenan Brock
@kbrock
Sep 09 2016 14:21
branches not leaf nodes
Oleg Barenboim
@chessbyte
Sep 09 2016 14:21
middle earth?
Keenan Brock
@kbrock
Sep 09 2016 14:21
lol
Jason Frey
@Fryguy
Sep 09 2016 14:21
haha
Keenan Brock
@kbrock
Sep 09 2016 14:22
having the stuff display in the right also in the tree makes no sense
Jason Frey
@Fryguy
Sep 09 2016 14:22
there's your middle earth trees ;) http://www.yayboo.com/photos/4/lg/22524.jpg
Keenan Brock
@kbrock
Sep 09 2016 14:22
that is up for debate / business discussion
yay trees from middle earth
Jason Frey
@Fryguy
Sep 09 2016 14:22
@kbrock Yeah, that is the going to be discussed in the meeting
Keenan Brock
@kbrock
Sep 09 2016 14:22
@chessbyte but a less business / more technical discussion is getting the tree nodes coming out quickly
Jason Frey
@Fryguy
Sep 09 2016 14:22
alternate ideas...perhaps even dropping trees altogether for a different paradigm
Keenan Brock
@kbrock
Sep 09 2016 14:23
I'm not holding my breath
er
we're able to generate the trees (without the leafs) and get 3k vms out in < 2 seconds
I'm looking at ways of getting darga faster
Jason Frey
@Fryguy
Sep 09 2016 14:26
ok, so what I'm hearing is loading all of the folders up front, but not the VMs? If so, isn't that the second bullet I mentioned above? https://github.com/ManageIQ/manageiq/blob/master/app/presenters/tree_builder_vms_and_templates.rb#L10-L18
Oleg Barenboim
@chessbyte
Sep 09 2016 14:26
meeting will focus on 'f' release and beyond
Keenan Brock
@kbrock
Sep 09 2016 14:30
@Fryguy yes
but full tree builder prevents this
Jason Frey
@Fryguy
Sep 09 2016 14:30
yeah, it wasn't designed for that
Keenan Brock
@kbrock
Sep 09 2016 14:30
@chessbyte yes, so if I have a 'd' focused change
Jason Frey
@Fryguy
Sep 09 2016 14:30
probably need a new kind of tree builder...like a "partial on-demand" tree builder
Keenan Brock
@kbrock
Sep 09 2016 14:31
jason - the original design is good
it just doesnt' allow multiple levels in one fell swoop
and doesn't make returning hash vs object easy
to be honest, yes I'd like to merge full tree builder up
for services I'd like to load multiple levels (use ancestry)
but I can't figure it out
and we HAVE to solve this for 'd'
Jason Frey
@Fryguy
Sep 09 2016 14:33
I don't disagree with making new types of tree builders...this seems like a new one to me
Keenan Brock
@kbrock
Sep 09 2016 14:33
sounds great
Jason Frey
@Fryguy
Sep 09 2016 14:33
FullTreeBuilder has a very specific purpose...building a full tree
Keenan Brock
@kbrock
Sep 09 2016 14:33
but it is used in only 1 place. and it is a hint slow there
I am worried if we use elsewhere
Jason Frey
@Fryguy
Sep 09 2016 14:33
WalkingTreeBuilder (as I think it should be named...aka the other trees) has a very specific purpose...buliding only one node and expecting the user to go fully on demand
you want something in the middle
Keenan Brock
@kbrock
Sep 09 2016 14:33
agreed
we already have custom nodes
we return hashes OR objects for a node
Jason Frey
@Fryguy
Sep 09 2016 14:34

but it is used in only 1 place

Not a comment toward you, but I'm always confused why this is a rationale for deletion / collapsing

I hear this in general and I don't understand it...particularly when then the thing has clear separation of responsbilities
Keenan Brock
@kbrock
Sep 09 2016 14:34
if it is used in 1 place, and it isn't working in that 1 place, it is suspect that the design is good
Jason Frey
@Fryguy
Sep 09 2016 14:35
I disagree
it works, it's just not efficient
which means it needs to be made faster
Keenan Brock
@kbrock
Sep 09 2016 14:35
I don't want others to adopt it. then I'll have to fix it
aah
ok
that is why I said, I will not delete it, but possibly tweak it
but you are right
I want something different
Jason Frey
@Fryguy
Sep 09 2016 14:35
bugs/performance issues are everywhere, but that doesn't imply throwing it away
right exactly
there may very well be trees where FullTreeBuilder is the perfect answer
Keenan Brock
@kbrock
Sep 09 2016 14:36
just not when we have 50k records?
Jason Frey
@Fryguy
Sep 09 2016 14:36
but maybe it's not the right fit for VmsAndTemplates in particular (especially the way the tree is structured)
Keenan Brock
@kbrock
Sep 09 2016 14:36
we like ancestry
Jason Frey
@Fryguy
Sep 09 2016 14:37
for example a deep but narrow tree would benefit heavily from FullTreeBuilder
Keenan Brock
@kbrock
Sep 09 2016 14:37
so we need an easy way to leverage performance of ancestry and build those nodes in 1 fell swoop
Jason Frey
@Fryguy
Sep 09 2016 14:37
yeah
Keenan Brock
@kbrock
Sep 09 2016 14:37
re: deep / narrow
if we have pagination on the right hand side
then it doesn't make sense for us to download everything on the left
THAT is for the discussion next week
Jason Frey
@Fryguy
Sep 09 2016 14:40
I have a feeling that if you just tweak the current TreeBuilderVmsAndTemplates in some way you will get what you want though
like, using :of_type => EmsFolder somehow, and then letting the the UI already do it's walking thing for the VMs
Keenan Brock
@kbrock
Sep 09 2016 14:41
stupid mic failure
Jason Frey
@Fryguy
Sep 09 2016 14:41
might have an edge case on full-page-load to get the VMs (which wouldn't be triggered by a click)
I know THIS does not do vms
but if we do want vms, it is very close
Jason Frey
@Fryguy
Sep 09 2016 14:48
@kbrock Did you ever do the change we discussed about child_ids ?
Keenan Brock
@kbrock
Sep 09 2016 14:48
80 local branches and 200 local stashes
can you give me a little more context ;)
Jason Frey
@Fryguy
Sep 09 2016 14:48
haha
Keenan Brock
@kbrock
Sep 09 2016 14:49
I'm working so hard on reducing those stashes. feels like my inbox
@Fryguy but seriouly, was this with hardwares and aggregates?
Jason Frey
@Fryguy
Sep 09 2016 14:49
use git pause...I have 3 stash items
Keenan Brock
@kbrock
Sep 09 2016 14:49
This message was deleted
seriously?
how many branches
Brandon Dunne
@bdunne
Sep 09 2016 14:50
stashes are the worst, so many conflicts so often. pause / resume FTW
Jason Frey
@Fryguy
Sep 09 2016 14:50
77
Keenan Brock
@kbrock
Sep 09 2016 14:50
wow
you beat me on both numbers. I need to try this out
Jason Frey
@Fryguy
Sep 09 2016 14:50
I'm due for a cleanup though, probably 50% are already merged in some form
Oleg Barenboim
@chessbyte
Sep 09 2016 14:50
git pause/resume +1000
Jason Frey
@Fryguy
Sep 09 2016 14:50
basically, git pause keeps the stash with the branch
Keenan Brock
@kbrock
Sep 09 2016 14:50
ooh, and jason. I have 0 already merged branches
Jason Frey
@Fryguy
Sep 09 2016 14:51
which is what you want like 90% of the time
Keenan Brock
@kbrock
Sep 09 2016 14:51
yes
Jason Frey
@Fryguy
Sep 09 2016 14:51
I really only use stash to transfer partial stuff from one branch to another
Keenan Brock
@kbrock
Sep 09 2016 14:51
sold
well, in trying to keep PRs small
I often have 1 branch that renders 12 PRs
anyway
@Fryguy child_ids - can you give me 10 more words?
Jason Frey
@Fryguy
Sep 09 2016 14:52
yeah, that's my "already merged branches"...ones that I've split apart and the SHAs are different, but content wise it's basically the same...so git can't "detect" they are already merged and I have to do it manually
oh right...was looking in gitter history for the convo :P
More or less here (a little above, but I like this idea better :P )
Keenan Brock
@kbrock
Sep 09 2016 14:54
aah
ended back in there a few times
the problem is we have a scope
and we collect into the array
to then convert to the id
to then use in a query
and then we pull back to count
that work paused for me because it is most crucial for trees without vms in it
once you add vms into it, the work becomes moot
Jason Frey
@Fryguy
Sep 09 2016 14:59
gotcha
Keenan Brock
@kbrock
Sep 09 2016 15:01
ok
jason
what do you think about only 1 builder
but it allows you to return an object with children
OR
a single object
OR
a hash for the level
those are our 3 use cases
for services, I'm not sure that I want to rewrite to be full tree builder
Jason Frey
@Fryguy
Sep 09 2016 15:03
so a OneLevelDeepTreeBuilder?
Oleg Barenboim
@chessbyte
Sep 09 2016 15:05
@kbrock I think @Fryguy is trying to have different TreeBuilder classes with different requirements
Keenan Brock
@kbrock
Sep 09 2016 15:07
ok, I'll subclass and see how it goes
Jason Frey
@Fryguy
Sep 09 2016 15:07
I could also see different methods for different outputs... like TreeBuilderVmsAndTemplates.tree currently returns the tree as needed by dynatree...I don't see why it couldn't have a .to_h method that only returns the hash or a .objects method that only returns the objects or something
Keenan Brock
@kbrock
Sep 09 2016 15:08
yea, the relation_trees stuff
Jason Frey
@Fryguy
Sep 09 2016 15:08
right
Keenan Brock
@kbrock
Sep 09 2016 15:08
ok, different conversation (kinda)
I'm in services
and it is N+1 each level for ancestry
which is a waste
Jason Frey
@Fryguy
Sep 09 2016 15:09
right...because it's written as a walker, right?
Keenan Brock
@kbrock
Sep 09 2016 15:09
right
Jason Frey
@Fryguy
Sep 09 2016 15:09
right ;)
Keenan Brock
@kbrock
Sep 09 2016 15:09
I want a walker, just one that can take multiple steps at a time
Jason Frey
@Fryguy
Sep 09 2016 15:09
cool
Keenan Brock
@kbrock
Sep 09 2016 15:09
this is a subclass
Jason Frey
@Fryguy
Sep 09 2016 15:09
Leaper haha
Keenan Brock
@kbrock
Sep 09 2016 15:09
heh
Jason Frey
@Fryguy
Sep 09 2016 15:10
ParkourTreeBuilder :P
Keenan Brock
@kbrock
Sep 09 2016 15:10
so this is a subclass of walker?
Jason Frey
@Fryguy
Sep 09 2016 15:10
No, sounds completely new to me
Keenan Brock
@kbrock
Sep 09 2016 15:10
ugh. ok
Jason Frey
@Fryguy
Sep 09 2016 15:10
Though I guess you could subclass it
Keenan Brock
@kbrock
Sep 09 2016 15:11
but yea
Jason Frey
@Fryguy
Sep 09 2016 15:11
I don't think it really matters...depends
Keenan Brock
@kbrock
Sep 09 2016 15:11
has_kids_for == very slick
ok, I'll see how that goes
was hoping for less work
Jason Frey
@Fryguy
Sep 09 2016 15:12
(aside: discussion made me think of https://www.youtube.com/watch?v=29MAL8pJImQ )
Keenan Brock
@kbrock
Sep 09 2016 15:12
what part
make complete changes before going to next step?
Oleg Barenboim
@chessbyte
Sep 09 2016 15:12
classes
Jason Frey
@Fryguy
Sep 09 2016 15:12
I am not a fan of the walking tree builder, so my gut is to not subclass it but that's more feelings than technical reasons :)
Keenan Brock
@kbrock
Sep 09 2016 15:13
ok, so my memory is just rusty, but not wrong
you are transitioning away from a tree builder
I though it was the complete one
but it was the walker
ok, we'll see how this goes
Jason Frey
@Fryguy
Sep 09 2016 15:14
:) not trying to beat a dead horse, but there's no transitioning in my mind...I think there should be multiple TreeBuilders in our arsenal and we pick the right one for the job
so if anything I want to transition away from one-size-fits-all of using the Walker everywhere, and only using it where it makes sense
in that same vein, if FullTreeBuilder is the wrong choice for the VmsAndTemplates tree, we should transition off that to the right tool
Keenan Brock
@kbrock
Sep 09 2016 15:16
I'm going to see if I can do something that allows ancestry trees to be returned
Jason Frey
@Fryguy
Sep 09 2016 15:16
:+1:
Keenan Brock
@kbrock
Sep 09 2016 15:16
the tree builders are a little tricky
too many special cases handled in the base class... one size fits all...
Jason Frey
@Fryguy
Sep 09 2016 15:16
yes, the base class is horribly nasty in that respect
that was one motivation to create the FullTreeBuilder the way I did...it literally only uses 1 method from the base class
Keenan Brock
@kbrock
Sep 09 2016 15:17
the way the builders swap (e.g. tree builder returns a tree node builder) throws me. but I'll dive in
Jason Frey
@Fryguy
Sep 09 2016 15:17
and then itself has the convert_ancestry_tree_to_dynatree method
ah yeah...that's confusing
Keenan Brock
@kbrock
Sep 09 2016 15:20
lot of copy paste between many classes in tree builder. but some of them are base classes with hundred line case statements
Jason Frey
@Fryguy
Sep 09 2016 15:22
yup :/
Keenan Brock
@kbrock
Sep 09 2016 21:19
@Fryguy I was thinking about https://github.com/ManageIQ/manageiq/compare/master...kbrock:tree_hybrid to give us hybrid trees
just have trouble going form a dynahash back to the objects. (which we currently have in the main tree - and is causing build_node_tree to do all the N+1. it needs the object and only has an id
so it is not perfect, but think you understand the scope of the change I want to make now
to be honest, this will work with any ancestry
vms is different, but for a normal one, if we include [:children] in there, it can just use that instead of looking up the children
Keenan Brock
@kbrock
Sep 09 2016 21:33
@NickLaMuro did you try:
  def x_get_tree_service_kids(object, count_only)
    objects = object.direct_service_children.where(:display => true)
    count_only_or_objects_filtered(count_only, objects, 'name')
  end
granted, I want 1 query for ALL services, but that seems like a big win
Nick LaMuro
@NickLaMuro
Sep 09 2016 21:34
Again, that doesn't fix it
the problem is that there are 9k roots in this tree
so that gets fired 9k times to return nothing
so yes, it is good in other use cases
Keenan Brock
@kbrock
Sep 09 2016 21:34
ok, on it ;)
but it is prettier
no?
what does filtered do?
  def x_get_tree_roots(count_only, _options)
    roots = Rbac.filtered(Service.roots)
    MiqPreloader.preload(roots, :picture) unless count_only

    count_only_or_objects(count_only, roots, "name")
  end
why are we only passing in roots
could we just load all trees and pass THAT to rbac?
wonder if we load a tree and then use association.targets = ancestry children - it may look nice
Nick LaMuro
@NickLaMuro
Sep 09 2016 21:37
it was a quick win back in june ManageIQ/manageiq#9373 to make it faster when there was a flat services tree
Keenan Brock
@kbrock
Sep 09 2016 21:39
aah - coming back to me
Keenan Brock
@kbrock
Sep 09 2016 21:48
do we even have rbac for services?
I'm reading that it only uses tenant filtering
Nick LaMuro
@NickLaMuro
Sep 09 2016 21:53
:shrug:
probably a Chris A question at this point
Keenan Brock
@kbrock
Sep 09 2016 21:59
yea, there really isn't rbac for our db
for any user
no tenants
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:10
Discuss! ManageIQ/manageiq#11162
Keenan Brock
@kbrock
Sep 09 2016 22:14
thanks
nick - how do you call locally?
do you use in the ui?
or can you run in console?
aah - storing in a variable - keep forgetting that trick
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:18
Yeah, I am running using miq_performance. So bin/rake miq_performance:benchmark['/service/explorer']
or more specifically:
$ seq 1 5 | xargs -I {} bin/rake miq_performance:benchmark['/service/explorer']
which basically runs the benchmark on that URL 5 times in a row
Keenan Brock
@kbrock
Sep 09 2016 22:20
but when you are thinking up something, and want to use bye bug... you do in the conosole?
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:22
yeah, pretty much, minus I don't really use a debugger
Keenan Brock
@kbrock
Sep 09 2016 22:23
wow - if I were that smart...
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:24
not sure I would call it smart... more like lazy
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:34
@Fryguy I might need my dev card back...
Re-ran with ManageIQ/manageiq#10929 applied (well, a patch that fixed the code review stuff) on the /service/explorer route, and am seeing like a 10sec drop in times (I still don't know why though...)
Keenan Brock
@kbrock
Sep 09 2016 22:36
I think I'm able to do it in 1 query
but it is a 9 second query
ooh - 11 seconds
SELECT "services".* FROM "services" ORDER BY ancestry nulls first, lower(name)
Oleg Barenboim
@chessbyte
Sep 09 2016 22:37
you guys sound like guests on "Name that Tune"
"I can get those records back in 9 seconds", "no, 2 seconds", ... :-)
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:38
"1 dollar bob"
realizes he is quoting "Price is Right"... different show
Keenan Brock
@kbrock
Sep 09 2016 22:40
yea, nicks is faster
I'm going for the fences
the babe ruth approach
(high changes of striking out, but if it works...)
Oleg Barenboim
@chessbyte
Sep 09 2016 22:40
Keenan is pointing at right field -- hoping he can reduce a 38 second query to 0.62 seconds
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:41
I GOT LUCKY ONE TIME!
the pressure is real
Keenan Brock
@kbrock
Sep 09 2016 22:42
lol
THIS is what it is all about
hmm friday night at 7:00 - need to think about feeding kids.
hmm. maybe in an hour ;)
ice cream works for now
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:44
now you are going to make me want to get Ice Cream...
Keenan Brock
@kbrock
Sep 09 2016 22:50
hey, if you want to swing by... we're having it for dinner. actually, it is gelato
ms queries query (ms) rows comments
22,625.0 218 2,632.5 12,591 /service/explorer
take that!
ooh, and that is the first run, so it includes features and junk
and I think it is 1 ugly query that I'm sure you'll be able to tell me how to fix
the solution is not elegant
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:53
nice
Keenan Brock
@kbrock
Sep 09 2016 22:53
no sql written too ;)
oh and nick - for the record. You've gotten lucky more than once
Nick LaMuro
@NickLaMuro
Sep 09 2016 22:55
"George just lucky I guess" (quote from George of the Jungle... from way back in the day)
Keenan Brock
@kbrock
Sep 09 2016 22:55
wha?
what did they do to product features. it now loads all of them. then it loads a subset of them (a very big one) - beast
oh, this will be fun to optimize. there is tons of room for fun here
ms queries query (ms) rows comments
18,458.1 217 1,600.1 12,590 /service/explorer
I found out how to avoid memoization, but it is ugly
Keenan Brock
@kbrock
Sep 09 2016 23:00
hmm. so miq_features is now downloading 1135 features (1 query). But it is also downloading an additional/unnecessary 1,504 rows (in 3 queries)
I've seen this in dev, but I'm running in production mode - keep that in mind when running numbers. only is taking 40ms, but it will mess with GC

ok, so @Fryguy merged delegates belongs_to bugfix, and delegate default values (the scarry one)

it may be time to embed virtual attributes into the primary query instead of the n+1 we do now

we are in the icon view, but we sure are downloading a lot of data. maybe we're downloading the data for the other tabs. I know we do this for the accordion, probably for the left hand icons vs table view
Keenan Brock
@kbrock
Sep 09 2016 23:08
@NickLaMuro how fast is your approach? is it beating 13s?
Mine is uglier than yours
Nick LaMuro
@NickLaMuro
Sep 09 2016 23:08
no, mine are just 10s improvements a piece
:point_up: September 9, 2016 5:34 PM That though, will probably help yours as well
Keenan Brock
@kbrock
Sep 09 2016 23:16
ok
ms queries query (ms) rows comments
12,601.1 209 385.1 9,951 /service/explorer
I like my query more this time around
and all of the 209 are basically n+1
Keenan Brock
@kbrock
Sep 09 2016 23:38
ManageIQ/manageiq#11166 -- 9k queries -> 209