These are chat archives for ManageIQ/manageiq/performance

20th
Feb 2018
Nick LaMuro
@NickLaMuro
Feb 20 2018 00:02

for the record... I ended up googling gssapi... "Generic Security Service Application Program Interface"

(so ignore my stupid guess at it above)

Keenan Brock
@kbrock
Feb 20 2018 14:11
@rvalente those would help. thanks for the reminder - I'll see if I can modify those scripts as part of that PR
@NickLaMuro thnx :point_up: February 19, 2018 4:21 PM
Ron Valente
@rvalente
Feb 20 2018 14:39
@kbrock thanks! Hope you had a good PTO. Running the purge now, will follow up with periodic_vacuum_fulltables and see where we stand from a DB footprint. Current metrics* partition tables are now at 45GB :fire:
Keenan Brock
@kbrock
Feb 20 2018 14:39
thnx
something happened recently
it used to be abe to keep up with purging the metrics, now it is having troubles
hence the drastic route of truncate
Ron Valente
@rvalente
Feb 20 2018 14:40
yea
makes sense
Keenan Brock
@kbrock
Feb 20 2018 14:40
thanks to @NickLaMuro for the comments to make it more apparent. (I didn't write the original stuff, just my job to make faster :) )
Ron Valente
@rvalente
Feb 20 2018 15:05
@kbrock @NickLaMuro do you want me to update a BZ or the PR with the PostgreSQL tuning I did to allow the WAL to run clean? (wal_buffers -> 32MB, max_wal_size -> 4GB)
(Defaults are 16MB and 1GB respectively).
Joe Rafaniello
@jrafanie
Feb 20 2018 15:22
@rvalente can you open a BZ with the tuning parameters you used and how it changed the problem you were seeing?
thanks
if it makes sense, we can make it the default
Ron Valente
@rvalente
Feb 20 2018 15:24
Will do! Thanks
Joe Rafaniello
@jrafanie
Feb 20 2018 15:25
:+1:
Keenan Brock
@kbrock
Feb 20 2018 15:35
thnx
Keenan Brock
@kbrock
Feb 20 2018 15:42
@NickLaMuro um - what hour is daylight savings going to eat?
oooh - we are doing utc, so no daylight savings issues. but I'll put in some tests
Nick LaMuro
@NickLaMuro
Feb 20 2018 15:44
@kbrock your main final times are UTC, but I fear that since you are converting from local time, there might be issues
Keenan Brock
@kbrock
Feb 20 2018 15:44
thnx
but yea, it does seem dangerous
all of this is dangerous
should I rename PR to running with scisors ?
Nick LaMuro
@NickLaMuro
Feb 20 2018 15:46
s/scisors/scissors/
Keenan Brock
@kbrock
Feb 20 2018 15:46
ug
guess that is why we have PRs
Nick LaMuro
@NickLaMuro
Feb 20 2018 15:46
or in emojis: :running: :scissors:
Keenan Brock
@kbrock
Feb 20 2018 15:46
+1
you know, I thought everyone knew there were triggers for metrics table
I'm glad there was a few(?) "me too"'s in the PR from beni
I do remember how long it took me to debug a performance problem in the trigger ;)
or how I found out how the low level deletion worked. the explain is very funky when working with those tables
ok, I'll go document it
think I hit the code changes
djberg96 @djberg96 was not aware we had triggers, thought they were verboten in rails
Ron Valente
@rvalente
Feb 20 2018 16:13
@kbrock are the triggers in the metrics table ON DELETE?
Joe Rafaniello
@jrafanie
Feb 20 2018 16:14
Thanks @rvalente
Keenan Brock
@kbrock
Feb 20 2018 16:51
@rvalente no. On insert into ‘metrics’. That table is actually always empty. Sub tables get the values. Nothing on delete
Daniel Berger
@djberg96
Feb 20 2018 17:02
select prosrc from pg_trigger,pg_proc where pg_proc.oid=pg_trigger.tgfoid and pg_trigger.tgname = 'metrics_inheritance_before';
or just view them in pgadmin :)
Keenan Brock
@kbrock
Feb 20 2018 18:00
@NickLaMuro ManageIQ/manageiq#17017
You want me to throw together a delete version that uses with_table_for_time ?
Nick LaMuro
@NickLaMuro
Feb 20 2018 18:03

@kbrock not sure it is entirely necessary. Again, I really don't have a problem using truncate, since it makes sense for how the data is put together.

I do, however, think we need to be extra sure that the new code works the same, before and after, and that those walking in to this code for the first time understand what is being done

Hence why I suggested possibly making the tests their own PR, so we validate they work on the old code, then apply them to master and rebase them into what you have done with truncate
Keenan Brock
@kbrock
Feb 20 2018 18:25
ok - sounds like a plan
not sure if we need to backport these - so I'd like to try and keep them as back portable as possible (reduced number of commits)
Keenan Brock
@kbrock
Feb 20 2018 18:34
@NickLaMuro this work for you - or you have other use cases as well? (before daylight savings, ...)
    it "deletes mid day" do
    it "deletes just after midnight" do
    it "deletes just after daylight savings (fallback)" do
    it "deletes just after daylight savings (spring forward)" do
Nick LaMuro
@NickLaMuro
Feb 20 2018 18:41
I was thinking that the new year might also be an issue
Keenan Brock
@kbrock
Feb 20 2018 18:41
ugh - right
Nick LaMuro
@NickLaMuro
Feb 20 2018 18:41
but maybe not
Keenan Brock
@kbrock
Feb 20 2018 18:41
jan 1 2:00am?
Nick LaMuro
@NickLaMuro
Feb 20 2018 18:41
should be fine
... timezones make my head hurt...
Keenan Brock
@kbrock
Feb 20 2018 18:42
I'll check out parse to see if we pass in different date formats
lol
er - pass in different timezones
Nick LaMuro
@NickLaMuro
Feb 20 2018 18:42
do you want to pass in different settings as well (4.hours, 8.hours, etc.)?
Keenan Brock
@kbrock
Feb 20 2018 18:43
ok
sounds good
don't think I can test interval
that timeframes do you want to test 8.hours? I was thinking just 2am - I think that is the perfect time to screw things up
Keenan Brock
@kbrock
Feb 20 2018 19:31
@NickLaMuro think I'm cheating on the tests. the old purge was granular to the minute. the new one is granular to the hour
the old one will properly delete records 4:01 hours ago - the new one keeps that one - rounding down to the hour. so if run at 8:01 - it will keep 4:59 hours worth
the 4 hour window is arbitrary to begin with - just putting that out there
Nick LaMuro
@NickLaMuro
Feb 20 2018 19:44
fair enough
@kbrock I would just say make sure that there is stuff close to and within the 4.hour mark that then passes with both codebases
Daniel Berger
@djberg96
Feb 20 2018 19:48
"This is the Emergency Broadcast System. The Purge will begin in 21 minutes."
Nick LaMuro
@NickLaMuro
Feb 20 2018 19:49
@djberg96 you are the "totally prepared for the zombie apocalypse" type, aren't ya?
Daniel Berger
@djberg96
Feb 20 2018 19:51
I've seen all the movies by now, so i should be ;)
Keenan Brock
@kbrock
Feb 20 2018 22:48
@Ladas https://pganalyze.com/blog/parse-postgresql-queries-in-ruby.html - maybe we can get our pretty print from here