These are chat archives for ManageIQ/manageiq/performance

14th
Jun 2017
Eric Hayes
@hayesr
Jun 14 2017 00:15
Hey I tried Bootsnap for kicks. ManageIQ/manageiq#15366 Useful? 🤷‍♂️
Nick LaMuro
@NickLaMuro
Jun 14 2017 00:19
@hayesr I commented :D
Eric Hayes
@hayesr
Jun 14 2017 00:22
@NickLaMuro I see now, thanks :)
Keenan Brock
@kbrock
Jun 14 2017 01:42
@NickLaMuro I'm so confused. I remember adding alises for joins in arel and stuff - but what do I do with that gem?
Nick LaMuro
@NickLaMuro
Jun 14 2017 01:44
huh?
what gem?
Keenan Brock
@kbrock
Jun 14 2017 01:44

sorry, "gem"

Not sure if this is the most appropriate room for this, but ran across this "gem" today:
https://github.com/rails/arel/blob/master/test/visitors/test_to_sql.rb#L554-L556

Nick LaMuro
@NickLaMuro
Jun 14 2017 01:46
...
Jason Frey
@Fryguy
Jun 14 2017 14:20
@kbrock you mean this?
Keenan Brock
@kbrock
Jun 14 2017 17:42
@Fryguy I reformatted that PR per matthew's request, but then it just languished :(
Jason Frey
@Fryguy
Jun 14 2017 17:43
opensores ;)
Keenan Brock
@kbrock
Jun 14 2017 17:43
lol
too soon ;)
@NickLaMuro ok, c&U: I commented out the part that "saves" the row to the database.
And I noticed that it removes a bunch of BEGIN and END COMMIT statements
not sure if just doing an insert (without the begin / end) would save us that much
@Fryguy is rails/rails#17937 achievable?
Or is it just a pipe dream?
and I guess my use case is different. if it is a single statement I want no begin / commit at all
Keenan Brock
@kbrock
Jun 14 2017 17:54
@blomquisg c&u has 2 components. 1 which is global, and 1 which is mixed into each object. capture.rb vs ci_mixin/capture.rb
In the past, I moved a bunch of the ci_mixin stuff into capture.rb.
Thinking that we may want to move more into the global part.
ok with you?
Jason Frey
@Fryguy
Jun 14 2017 17:54
yeah achievable
just someone needs to spend time to do it
and I frankly don't have time at the moment...but it would save us a TON in certain parts of our app
Keenan Brock
@kbrock
Jun 14 2017 17:54
yea, I don't know how to do without modifying all connectors (adapters?)
and that potentially can break the various plugins - seems "dangerous"
Greg Blomquist
@blomquisg
Jun 14 2017 17:55
@kbrock: Living Dangerously
Jason Frey
@Fryguy
Jun 14 2017 17:55
¯\_(ツ)_/¯
Keenan Brock
@kbrock
Jun 14 2017 17:56
ok, may look into that
Jason Frey
@Fryguy
Jun 14 2017 17:56
I think it can be done by introducing new methods in the adapters that are private, like being_db_transaction_delayed, then call those directly from transaction
Keenan Brock
@kbrock
Jun 14 2017 17:56
yea, it is the emit part - doesn't work so easily
there are also 2 or 3 places that can hold the state of "have emitted 'BEGIN' yet"
would be very cool if the code knew about callbacks / children and could handle transactions accordingly
Jason Frey
@Fryguy
Jun 14 2017 17:58
I'm not sure what you mean
that's exactly what I implemented in the Rails2 patch in the issue
it just needs to be "reimplemented" in the newer Rails5 code base
Keenan Brock
@kbrock
Jun 14 2017 17:59
yea
in rails, if it were smarter about callbacks, we could just use destroy everywhere. and if there were no callbacks, it would do delete under the covers vs downloading all objects
same thing goes for save. if it knows there are no nested objects or other validations, then it could just send the update or create sql statement and not deal with transactions at all
Jason Frey
@Fryguy
Jun 14 2017 18:02
I'm lost :/
Keenan Brock
@kbrock
Jun 14 2017 18:02
doesn't matter - ok
think fixing this bug which has been open for a while is too big right now
already in 2 context switches deep
Chris Arcand
@chrisarcand
Jun 14 2017 18:15

c&u has 2 components. 1 which is global, and 1 which is mixed into each object. capture.rb vs ci_mixin/capture.rb
In the past, I moved a bunch of the ci_mixin stuff into capture.rb.
Thinking that we may want to move more into the global part.

I don’t know anything about this but…does this have anything to do with the write stuff I’m working on?

Keenan Brock
@kbrock
Jun 14 2017 20:56
nah
@chrisarcand of interest, I merged all the collectors with the coordinator - so it is a single process.
Commenting out the write into Metric sped up 160s -> 60s (12k writes)
so that suggests that writing into the metrics table is slow
we do something strange there, where we write into a view or something.
Looking into writing to a table instead of the view. Think that will speed us up by a bunch - maybe 50% overall. because an insert into a table should be quite quick
Jason Frey
@Fryguy
Jun 14 2017 20:59
it writes into a table
but it's an inherited table with partitioning
Gregg Tanzillo
@gtanzillo
Jun 14 2017 21:00
bundle install
:D
Keenan Brock
@kbrock
Jun 14 2017 21:01
ok, but 12k writes should not take 100seconds
Jason Frey
@Fryguy
Jun 14 2017 21:01
man, @chrisarcand 's gif would hve been a perfect response to @gtanzillo 's misfire
Keenan Brock
@kbrock
Jun 14 2017 21:01
lol
Gregg Tanzillo
@gtanzillo
Jun 14 2017 21:01
lol, yeah!
Keenan Brock
@kbrock
Jun 14 2017 21:02
@Fryguy so is metrics and metrics_00 a different table? or can I think of it as a view?
Jason Frey
@Fryguy
Jun 14 2017 21:02
I guess you can think of it as a view, but it's not a literal view
metrics, metrics_01, etc. are completely separate tables leveraging stuff in those two links above
partitioning allows the purger to delete from one table while the writers access a completely different table, allowing them not to collide
eventually, we had thought we could just TRUNCATE TABLE instead of purging, but never got there
Keenan Brock
@kbrock
Jun 14 2017 21:03
can I insert directly into metrics_01?
Jason Frey
@Fryguy
Jun 14 2017 21:04
@kbrock technically, yes, but I don't know what side effects that will have
Keenan Brock
@kbrock
Jun 14 2017 21:04
hmm