Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Keenan Brock
    @kbrock
    aah - source_location
    Joe Rafaniello
    @jrafanie
    Are we even using CimProfiles?
    Keenan Brock
    @kbrock
    I find myself asking that question about a bunch of stuff :)
    Joe Rafaniello
    @jrafanie
    It's so frustrating looking at perf stuff and random other classes are being loaded
    Joe Rafaniello
    @jrafanie
    @kbrock @dmetzger57 did we discuss MiqExpression.model_details?
    Dennis Metzger
    @dmetzger57
    @jrafanie not that i recall
    Joe Rafaniello
    @jrafanie
    7% of the first visit to vm_infra/explorer on ruby 2.1 on 5.4.z (without the sort fix) is in building cached data for 2 models
    we used to pre-populate this information at startup in all environments... I don't see why we couldn't do it a UI worker context at startup of that process
    Dennis Metzger
    @dmetzger57
    that would at least partially explain the extra slow render on the first visit to the page, but it won’t help with the every other visit
    Joe Rafaniello
    @jrafanie
    Screen Shot 2016-02-03 at 1.59.42 PM.png
    Yes, that would only help the first visit
    Dennis Metzger
    @dmetzger57
    not sure if its worth the work/risk for the current BZ, definitely another going forward perf improvement
    Joe Rafaniello
    @jrafanie
    yeah, it would depend if the initial visit is what they're most disliking
    Dennis Metzger
    @dmetzger57
    it’s the 30-40 seconds everytime they visit the page that is the current complaint
    Joe Rafaniello
    @jrafanie
    ok, the change is minor, just wanted to point it out
    diff --git a/vmdb/lib/workers/mixins/web_server_worker_mixin.rb b/vmdb/lib/workers/mixins/web_server_worker_mixin.rb
    index 1a2874d..eea775c 100644
    --- a/vmdb/lib/workers/mixins/web_server_worker_mixin.rb
    +++ b/vmdb/lib/workers/mixins/web_server_worker_mixin.rb
    @@ -11,6 +11,11 @@ module WebServerWorkerMixin
       def do_before_work_loop
         @worker.release_db_connection
    
    +    MiqExpression.base_tables.each do |model|
    +      MiqExpression.get_relats(model)
    +      MiqExpression.build_lists(model)
    +    end
    +
         # Since thin traps interrupts, log that we're going away and update our worker row
         at_exit { do_exit("Exit request received.") }
       end
    or something like that
    Oleg Barenboim
    @chessbyte
    Why do we have to build anything at all? Why can't it be built on first reference via standard memoization pattern??
    Less is More
    Chris Arcand
    @chrisarcand
    Joe Rafaniello
    @jrafanie
    It's built on demand right now, I don't really know what it does
    ok, @Fryguy tell me again why we need to build all 3 trees when we only have 1 visible?
    Oleg Barenboim
    @chessbyte
    I am so lost on all this pre-building for the UI so we dont spend extra time if and only if user clicks somewhere
    suggest getting back to basics
    Joe Rafaniello
    @jrafanie
    we call VmShowMixin#build_vm_tree 3 times, once per tree... each build_vm_tree uses relationships to build a relationship tree, fetching 10k+ things from the database... we then filter that tree by doing a different search using Rbac.search (10k mostly duplicate things)
    the trees are all closed initially and only 1 is visible
    once these enormous trees, with data we don't actually see at first, are built, we convert all three 3 to json so we can have a format the UI can then draw I assume
    if we're lucky, we might be able to carve off 10-20% by finding slow code
    Oleg Barenboim
    @chessbyte
    @jrafanie this is where javascripting that page over REST API to get what is needed when it is needed would help, right?
    Joe Rafaniello
    @jrafanie
    well, yes, I think the "dividing into parts that we need when we need them" is the key, the vehicle of delivering these parts is immaterial
    If a single vm changes power state, we'd have to reload the 3 whole trees to see it
    if REST is the vehicle to force the carving up of things into parts, then yes, REST all the things
    Matthew Draper
    @matthewd
    Doesn't that put us right back where we were before @Fryguy fixed it? :confused:
    Joe Rafaniello
    @jrafanie
    @dmetzger57 what's the expectations?
    I might have missed it, have we done comparisons of master to 5.4?
    I haven't yet migrated to master... I'm wondering if it's worth doing that
    Joe Rafaniello
    @jrafanie
    @kbrock looks like a valid test failure on your rbac.search sort fix: https://travis-ci.org/ManageIQ/manageiq/jobs/106761189
    Dennis Metzger
    @dmetzger57
    @jrafanie @akross has run 5.4.4.2, 5.4.5.0 and 5.5.0.13 with the customer database. results in BZ 1281999. The number are better on 5.5.0, on the Haswell hardware the page loads in 12-17 seconds as opposed to 27-20 wih 5.4.42 and on Sandy Bridge its 21-27 seconds vs 49-63 seconds. Master to be tried. I’ve asked if the times on 5.5.0 would be acceptable, though I’ve been told that an upgrade is not viable in the short term.
    Keenan Brock
    @kbrock
    @jrafanie thanks for the reminder
    Joe Rafaniello
    @jrafanie
    np
    Joe Rafaniello
    @jrafanie
    @dmetzger57 @akrzos in the comments regarding the timing in the bugzilla, https://bugzilla.redhat.com/show_bug.cgi?id=1281999#c21, is that the initial visit to vm_infra/explorer and is that from the rails log/production.log?
    Keenan Brock
    @kbrock
    the initial visit has a bunch of junk in there - including that one part that you mentioned lj
    Joe Rafaniello
    @jrafanie
    Yeah, I can't tell what the numbers mean
    Keenan Brock
    @kbrock
    I'd prefer to ignore the first, and just look at the others
    yea
    Dan Clarizio
    @dclarizio
    @chessbyte We could not build the second and third accordion trees until clicked on, but then those should NOT be the ones taking all the time as those are just trees of filters (maybe dozens) not thousands like the VM & Templates tree, which mimics the VMware hierarchy. That first tree is the biggest offender in large environments.
    Alex Krzos
    @akrzos
    @jrafanie Yes thats from the production.log
    The initial visit to the explorer is the first timing
    (always higher)
    Dan Clarizio
    @dclarizio
    @chessbyte We used to add nodes to the trees as the user clicked thru, but then as @Fryguy pointed out, after quite a few nodes were open, leaving and revisiting the tree would cause n+1 type fetches to fill in the parts the user had opened prior