These are chat archives for ManageIQ/manageiq/performance

6th
Feb 2018
Daniel Berger
@djberg96
Feb 06 2018 00:23
@NickLaMuro what inspired the latest PR for sys-proctable? were you seeing performance/memory issues?
or just felt like it?
Keenan Brock
@kbrock
Feb 06 2018 00:23
lol
Daniel Berger
@djberg96
Feb 06 2018 00:24
oh, nm, i forgot we were looking at this before
Keenan Brock
@kbrock
Feb 06 2018 00:24
it came to light when nick was looking for memory leaks
Nick LaMuro
@NickLaMuro
Feb 06 2018 00:24
@djberg96 more of the latter than anything. Nothing is dependent on it, just thought it wouldn't hurt.

it came to light when nick was looking for memory leaks

Well, yeah, at a certain point, everything was suspect...

even my sanity :worried:
Keenan Brock
@kbrock
Feb 06 2018 00:25
I always think everything could be a trap, which is why I'm still alive
--Prince Humperdinck (in Princess Bride) [ who was directly quoting @NickLaMuro]
Daniel Berger
@djberg96
Feb 06 2018 00:26
well, i wouldn't mind adding an options argument hash in general, but it does mean (potential) breakage for the windows version
but it would be good for Linux, too, since we could then let users skip smaps, etc
Keenan Brock
@kbrock
Feb 06 2018 00:27
ugh - ok. fixed quote. sorry about that
Nick LaMuro
@NickLaMuro
Feb 06 2018 00:28
@djberg96 yeah, this was just the easiest for me to quickly iterate on to start without a VM. Going to see if I can do some similar things with the other OS's as well
Daniel Berger
@djberg96
Feb 06 2018 00:28
I would vote to keep the lazy stuff separate
Nick LaMuro
@NickLaMuro
Feb 06 2018 00:28

--Prince Humperdinck (in Princess Bride) [ who was directly quoting @NickLaMuro]

Not the intent, but I will take credit as if I was anyway

@djberg96 cool, I will rebase that out then and put it in a separate branch for later
Also, regarding the "why now" question from early, this was also a bit of a respite from trying to get java openjdk-8-jre installed on a TCL (Tiny Core Linux) VM...
they use Mercurial for there source... it displeases me...
Daniel Berger
@djberg96
Feb 06 2018 00:33
is the gist of the overall savings basically avoiding repeated .to_sym calls?
Nick LaMuro
@NickLaMuro
Feb 06 2018 00:34
For the PROC_STRUCT_FIELD_MAP, yeah
but also the split, and other junk that is done for that as well (.join, .to_s)
The stuff with .ps "turning into a two line method" was more that even when a pid was passed in, we still called a method to collect and loop over all the pids
even though it was a no-op for all but one
Daniel Berger
@djberg96
Feb 06 2018 00:38
actually...i should check the C API and see if there's a function that will get the info we need for a single pid
oh, there's already proc_pidinfo
dunno why i stuck that in the loop
we could just use that if a pid argument is passed
maybe not.....need to dig
been too long
Nick LaMuro
@NickLaMuro
Feb 06 2018 00:45
(coffee shop WiFi just died on me. Might relocate...)
((On phone))
Daniel Berger
@djberg96
Feb 06 2018 00:45
Didn't buy enough coffee, they cut you off
Daniel Berger
@djberg96
Feb 06 2018 03:56
@NickLaMuro nm, that's basically what you did, and refactored out the struct creation into its own method.
Sorry if I'm slow on the uptake ;)
Anyway, made a comment
Keenan Brock
@kbrock
Feb 06 2018 14:31
@Fryguy yea - right?
Jason Frey
@Fryguy
Feb 06 2018 14:31
pretty cool

The underlying fundamental issue is that Prometheus, unlike Graphite/Statsd is centered around the concept pulling metrics as opposed to pushing metrics.

I think this summs up why we couldn't use prometheus for our internal metrics

Jason Frey
@Fryguy
Feb 06 2018 14:32
yup exactly
Keenan Brock
@kbrock
Feb 06 2018 14:33
but good to know sam has some solutions for that - so if we want to use that for metrics of our vms / app...
@Fryguy do we use AMS?
last I remember, oj was "the thing"
Jason Frey
@Fryguy
Feb 06 2018 14:33
no, we don't
but I thought that was pretty cool regardless :)
Keenan Brock
@kbrock
Feb 06 2018 14:33
(oj is no longer flavor of the month for a while)
oh yea
it looks fast
Jason Frey
@Fryguy
Feb 06 2018 14:35
wow that's cool
Keenan Brock
@kbrock
Feb 06 2018 14:35
we have something similar for counting queries
Keenan Brock
@kbrock
Feb 06 2018 15:26
@NickLaMuro generically parsing tables:
top -c -b -n 1 | ruby untable.rb --skip 6 --verbose
#   01234567891123456789212345678931234567894123456789512345678961234567897123456789812
      PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
    11325 root      20   0  157744   2208   1548 R   5.9  0.0   0:00.02 top -c -b -n 1
0   00000
1         111111111
2                   222
3                       33
4                          4444444
5                                  555555
6                                         666666
7                                                77
8                                                   8888
9                                                        9999
10                                                             000000000
11                                                                       ...

  0 ====
  0 PID                  11325                                   
  0 USER                 root                                    
  0 PR                   20                                      
  0 NI                   0                                       
  0 VIRT                 157744                                  
  0 RES                  2208                                    
  0 SHR                  1548                                    
  0 S                    R                                       
  0 %CPU                 5.9                                     
  0 %MEM                 0.0                                     
  0 TIME+                0:00.02                                 
  0 COMMAND              top -c -b -n 1
it looks at a table output (first 6 rows of top aren't a table) and turns it into a hash
same code works with ps, top, df, and others
even though the command has spaces in it - it groups it together
still would probably be better to just directly hit proc / sys-proctable
current appliance...
Daniel Berger
@djberg96
Feb 06 2018 23:19
@NickLaMuro's first PR for sys-proctable for Darwin merged - reduces symbol creation
if I've learned anything from Nick lately, it's that I shouldn't call .to_sym in a loop :)
Nick LaMuro
@NickLaMuro
Feb 06 2018 23:35
well, not if it returns the same results every time I guess...
Also, filled out the Benchmarks section in the description, even though it has already been merged
Nick LaMuro
@NickLaMuro
Feb 06 2018 23:43
Script I used for the IPS testing added in a PR here: djberg96/sys-proctable#67