These are chat archives for ManageIQ/manageiq/performance

4th
Feb 2016
Joe Rafaniello
@jrafanie
Feb 04 2016 00:54
Yes, but that's about it
Joe Rafaniello
@jrafanie
Feb 04 2016 13:58
The Rbac.search sort fix is also nice since everything goes through rbac... although not sure how many places pass in targets
Keenan Brock
@kbrock
Feb 04 2016 15:37

@matthewd @Fryguy that sorting method isn't so simple

tenant_ids = [1,2]
tenants = []

tenants.sort_by { |x| tenant_ids.index(x)} # => []
# vs
tenants_by_id = tenants.index_by(&:id)
tenant_ids.map { |t| tenants_by_id[t]} # => [nil, nil]

Should I add a compact? or should I go through tests and try and track down bad tests?

Matthew Draper
@matthewd
Feb 04 2016 15:41
I did mention that caveat ;)
I'd look very suspiciously at any such mismatch, but maybe just go with the sort_by so you can move on for now?
Keenan Brock
@kbrock
Feb 04 2016 15:42
@matthewd thanks - yea. just "commit it already"
I also find the sort_by easier to read.
Keenan Brock
@kbrock
Feb 04 2016 16:07
@jrafanie long read, but some good information about how to implement unicorn worker systems in ruby ref - I only got through a bit. it is dense.
ooh, and there is a movie version.
Jason Frey
@Fryguy
Feb 04 2016 16:09
Unicorn: The Movie
Keenan Brock
@kbrock
Feb 04 2016 16:10
+1
it is basic college knowledge from C - but good review since we are deep in here
Joe Rafaniello
@jrafanie
Feb 04 2016 16:12
cool, I'll take a look
Keenan Brock
@kbrock
Feb 04 2016 16:14
@Fryguy @jrafanie but warning. you have to do a shot every time you hear the word "socket"
Keenan Brock
@kbrock
Feb 04 2016 16:24
@jrafanie @dmetzger57 There is also rackrabbit - great example of a forking worker queue and the code is very approachable. we may be able to get something out of it gh (but year stale)
Matthew Draper
@matthewd
Feb 04 2016 16:25
Do we have plans to do that sort of forking? :confused:
Joe Rafaniello
@jrafanie
Feb 04 2016 16:29
yikes, 4k LOC
Keenan Brock
@kbrock
Feb 04 2016 16:30
seems the direction of our workers stuff
Joe Rafaniello
@jrafanie
Feb 04 2016 19:28
wow, @kbrock have you run the db against master?
I think I did something wrong: (ruby 2.2) Completed 200 OK in 58005ms (Views: 660.3ms | ActiveRecord: 2435.7ms)
time to check my Gemfile.dev.rb
Joe Rafaniello
@jrafanie
Feb 04 2016 20:07
oh yay, master is slow because of sprockets
Joe Rafaniello
@jrafanie
Feb 04 2016 20:13
don't do this thousands of times: ActionController::Base.helpers.image_path("100/#{icon}")
how much do you want to bet we calculate the same icon image_path many times?
oh well, this is master
Dennis Metzger
@dmetzger57
Feb 04 2016 20:15
that would be silly
Joe Rafaniello
@jrafanie
Feb 04 2016 20:15
I'll try to get the relevant rubyprof html
Matthew Draper
@matthewd
Feb 04 2016 20:19
Yeah, that's bad
You need to cache ActionController::Base.helpers somewhere
Rails used to cache it internally, but we no longer do
(No need to cache the individual image path mappings: those are fast… it's just slow to rebuild the whole set of helpers every time)
Joe Rafaniello
@jrafanie
Feb 04 2016 20:25
Screen Shot 2016-02-04 at 3.24.24 PM.png
Again, above is master only
Jason Frey
@Fryguy
Feb 04 2016 20:28
so what I'm seeing is that everything is slow because TreeNodeBuilder#build is slow ;)
Joe Rafaniello
@jrafanie
Feb 04 2016 20:43

(No need to cache the individual image path mappings: those are fast… it's just slow to rebuild the whole set of helpers every time

@matthewd so, I did SOME_CONSTANT = ActionController::Base.helpers early and replaced the helpers calls to use SOME_CONSTANT, didn't really help much 56575ms instead of around 60s

are you sure we don't have to cache the result of image paths?
Matthew Draper
@matthewd
Feb 04 2016 20:44
Yeah, I was noticing that seemed likely from your ruby-prof :confused:
You certainly shouldn't have to cache them
I think I would expect them to be cached inside sprockets, even in development
(this is development, I assume?)
Joe Rafaniello
@jrafanie
Feb 04 2016 20:46
production
Rails 4.2.5.1 application starting in production
ruby 2.2.4
Matthew Draper
@matthewd
Feb 04 2016 20:46
I thought we recently turned off asset compilation in production
(which should make it a simple hash lookup)
Joe Rafaniello
@jrafanie
Feb 04 2016 20:48
let me recompile my assets and try again, maybe my assets were old
I don't remember if we disabled asset compilation in production for all assets
Matthew Draper
@matthewd
Feb 04 2016 20:49
Could also try Draper::ViewContext.current.image_path("100/#{icon}"), just to check whether it's something weird about where we're getting the helper from
Joe Rafaniello
@jrafanie
Feb 04 2016 20:50
sprockets is :sparkles:
Me and sprockets
It helps to run rake assets:precompile with RAILS_ENV=production
stand down @matthewd, thanks
Joe Rafaniello
@jrafanie
Feb 04 2016 20:55
I think it's working now...
I'll profile now that it's back to 14/10 seconds in production
Keenan Brock
@kbrock
Feb 04 2016 21:08
@jrafanie sorry you had to look into this. Thought I had posted those findings before
Joe Rafaniello
@jrafanie
Feb 04 2016 21:09
no worries
Keenan Brock
@kbrock
Feb 04 2016 21:09
I am playing with that trace to reduce the number of frames displayed
so much noise
Joe Rafaniello
@jrafanie
Feb 04 2016 21:14
@kbrock I think you can do this (min_percent):
config.middleware.use Rack::RubyProf, :path => Rails.root.join('tmp/profile'), :printers => {RubyProf::CallStackPrinter => 'call_stack.html'}, :min_percent => 5
Keenan Brock
@kbrock
Feb 04 2016 21:14
I changed printers - goes much faster
also hacked it to use timestamp in the name
think you can do RubyProf::CallStackPrinter => lambda { "#{Time.now.strftime("%M")}-call_stack_.html" }
but I hacked directly in the source
@jrafanie what is min_percent - the size of each frame?
I want them there, just the whole 5 children that have no other children that are all 100% to collapse into the same one.
Keenan Brock
@kbrock
Feb 04 2016 21:17
yea, I'll have to play with this. they don't collapse parents into children
maybe modify expand to expand all children that are 100% of parent on click
@matthewd that OR clause in rbac is to add the "owned vms" onto the list. so you see all those vms that you have privs to see AND the ones you own
also looks like MS SQL server had issues with limits and sub selects - so we implemented our joins in ruby instead of sql :(
THAT is causing issues
Matthew Draper
@matthewd
Feb 04 2016 21:56
@kbrock that sounds like it's just a more refined definition of "all those you have privs to see", then
(which is good, because that sounds like something we can work with)
Joe Rafaniello
@jrafanie
Feb 04 2016 22:09
TIL: Don't use case statements a lot now that we use draper
Screen Shot 2016-02-04 at 5.08.50 PM.png
When all else fails, blame Draper cc @matthewd :wink:
Matthew Draper
@matthewd
Feb 04 2016 22:14
hah
Seems like that'd be better as an is_a?(Draper::Decorator) :confused:
Joe Rafaniello
@jrafanie
Feb 04 2016 22:32
yeah, i'll take a look