These are chat archives for ManageIQ/manageiq/performance

2nd
Dec 2016
Keenan Brock
@kbrock
Dec 02 2016 02:15
@Fryguy did you say that you and @chessbyte spoke with someone who wrote stored procedures in ruby? possibly PL/Ruby? Do you have a link to that project?
(I found a link to PL/Ruby) but it has had a mixed life and with mruby I though there may be better options
@Fryguy also, I stumbled across a sql query that converts yaml to json here - would be nice to get our serialized columns into a format that is postgres friendly. :)
although, postgres supports xml and json. and "explain plans" are returned in text, xml, json, and yaml. wonder if they would have a yamlb column or something
Jason Frey
@Fryguy
Dec 02 2016 03:59
@kbrock I don't remember talking to anyone about that
Keenan Brock
@kbrock
Dec 02 2016 04:33
heh. PL/Ruby is old, but looks like https://github.com/franckverrot/holycorn is the mruby version using foreign data wrapper. I had expected him to go the PL/Ruby route, but looks like there are 3 ways to go and his talk explains them
Joe Rafaniello
@jrafanie
Dec 02 2016 15:49
@NickLaMuro check out the fun script I was using with the malloc env variable to see if made a difference...
require './config/environment'
require 'sys/proctable'

def print_percent_shared(parent_or_child)
  smaps = Sys::ProcTable.ps(Process.pid).smaps
  puts "#{parent_or_child} #{((smaps.rss - smaps.uss)/smaps.rss.to_f) * 100} % shared.  #{smaps.rss / 1.megabyte} MB RSS"
end

fork do
  30.times do
    sleep 0.5
    print_percent_shared(:child)
    sleep 0.5
  end
  sleep 180
end

30.times do
  print_percent_shared(:parent)
  sleep 1
end

sleep 180
Cool story: we get down to 30-40% between parent/child shared memory within those 3 minutes
Keenan Brock
@kbrock
Dec 02 2016 15:52
@jrafanie is this running any of our code? or even rails?
Joe Rafaniello
@jrafanie
Dec 02 2016 15:52
yeah, I did that on an upstream appliance
loading config environment loads rails
Keenan Brock
@kbrock
Dec 02 2016 15:52
aah
yea see it now
Joe Rafaniello
@jrafanie
Dec 02 2016 15:52
I should check if it's using nagayoshi_fork
Keenan Brock
@kbrock
Dec 02 2016 15:52
think the COW is dead
Joe Rafaniello
@jrafanie
Dec 02 2016 15:53
^ this
Jason Frey
@Fryguy
Dec 02 2016 15:53
it should be using nakayoshi by default
Joe Rafaniello
@jrafanie
Dec 02 2016 15:53
yeah, fork gives us fast process creation and a little shared memory
that's about it
Keenan Brock
@kbrock
Dec 02 2016 15:53
I want the shepard gem (foreman for COW)
(doesn't exist)
Nick LaMuro
@NickLaMuro
Dec 02 2016 15:56
neat
(sorry, was fixing my PR that is now being vetted by the entire team I am sure...)
Keenan Brock
@kbrock
Dec 02 2016 15:57
lol
Joe Rafaniello
@jrafanie
Dec 02 2016 15:57
My gut is telling me that some of the various malloc/jemalloc tunings/options would be more helpful the moment ruby starts organizing the heap in anyway... as it is right now, it seems like things are thrown together randomly in contiguous blocks on a page and free/malloc will eventually cause most of the CoW memory to be copied