These are chat archives for opal/opal

2nd
May 2015
Forrest Chang
@fkchang
May 02 2015 06:35
I was using thor, had a hard time figuring out how to use it, surprisingly not documented given how mature it is, IMO. If urs is easier to grok, I'll switch
Vais Salikhov
@vais
May 02 2015 09:16
@elia, you there?
Elia Schito
@elia
May 02 2015 09:19
Hey
@vais sup?
Vais Salikhov
@vais
May 02 2015 09:21
So, I've been thinking about benchmarking again
There are many meanings to the word
  1. Opal vs MRI vs other rubies - interesting but useless
  1. Vanity benchmarks
I.e. how fast is my ruby vs other rubies
We are not interested in that (at least I'm not)
Then there is the whole Opal vs JavaScript performance thing
Again, I'm not interested in that.
So, this rules out (for me) running any of the Ruby benchmark suites out there
Vais Salikhov
@vais
May 02 2015 09:26
What I think makes the most sense for Opal development is to benchmark the specs
Elia Schito
@elia
May 02 2015 09:27
Makes sense to me
Vais Salikhov
@vais
May 02 2015 09:27
Again, the problem I want to solve is not knowing how a change to a method implementation of one class may affect the performance of that method AND performance of other seemingly unrelated methods. Then, with pinpoint precision, be able to go in and fix the performance problem.
Elia Schito
@elia
May 02 2015 09:27
i sometimes looked at the time taken on Travis in that sense
Vais Salikhov
@vais
May 02 2015 09:28
That's what a generic benchmark suite does not provide - the pinpoint precision to fix the problem
Yes, that's right! The solution is right there in the test suite!
mspec even has an option to run each it block multiple times
So, I think the solution is to leverage the spec suite and mspec itself
The issue that I'm struggling with is the general workflow for running this performance check
Logically, there's no escaping the fact that the suite would have to be run twice:
Elia Schito
@elia
May 02 2015 09:30
oh that's cool, we can add an allow failure entry that runs those block multiple times and set up some kind of boy that reports performance
Vais Salikhov
@vais
May 02 2015 09:31
once for the before, and once for the after, then the two can be compared
Elia Schito
@elia
May 02 2015 09:31
that would work locally of course but can give immediate feedback on IRC or on github
Vais Salikhov
@vais
May 02 2015 09:31
Right
But here's the problem: what about this before and after thing?
I mean, integrating it with git - there be dragons
Elia Schito
@elia
May 02 2015 09:32
for Travis I'm not sure is very easy
but we can keep it for local checks
Vais Salikhov
@vais
May 02 2015 09:33
Ok, so, lets go through a hypothetical workflow
Elia Schito
@elia
May 02 2015 09:33
say we have a env car for enabling bms on mspec with the number of times it should run
Vais Salikhov
@vais
May 02 2015 09:33
I checkout an older version of the code and run the performance rake task
Elia Schito
@elia
May 02 2015 09:34
we can use it on both local and travis
Vais Salikhov
@vais
May 02 2015 09:35
Right, so, back to the first step, I checkout an older version and run the task
Then I checkout the newer version and run the task
Elia Schito
@elia
May 02 2015 09:35
right
Vais Salikhov
@vais
May 02 2015 09:35
Now I want to compare them
The problem is that we'd need to store the intermediate results somewhere between runs, you see?
I mean there is a git break in the middle of the workflow
Elia Schito
@elia
May 02 2015 09:36
dump the results to json
Vais Salikhov
@vais
May 02 2015 09:36
So the thing cannot be in-process
Right, that's what I was thinking too
Elia Schito
@elia
May 02 2015 09:36
kry would be example name
key
Vais Salikhov
@vais
May 02 2015 09:36
Right
Elia Schito
@elia
May 02 2015 09:36
value the time
Vais Salikhov
@vais
May 02 2015 09:36
Right
Now I guess I need to dig into mspec
I hacked some things directly into it to have a proof of concept
Elia Schito
@elia
May 02 2015 09:37
heroku app can compare stuff for travis
Vais Salikhov
@vais
May 02 2015 09:38
Oh, nice
You mean the rake task would upload to a heroku app
?
Elia Schito
@elia
May 02 2015 09:38
yep
brb
Vais Salikhov
@vais
May 02 2015 09:38
Cool, then that's it!
I just worry my mspec-fu is zero right now
there's a ton of hooks
the ones I need must be there already, but I just don't know
But now I got the general sense of the direction where we want this to go, so I can start working on it
Thanks, @elia!
Elia Schito
@elia
May 02 2015 09:45
Last thing, you might want to ask brixen for tips as well
Vais Salikhov
@vais
May 02 2015 09:46
Do you know what's the best way to reach out to him?
I have a PR sitting on mspec for a couple weeks with no response, so not sure if opening an issue to ask for tips is gonna work...
Elia Schito
@elia
May 02 2015 09:47
Oh, I'd try IRC first and then with an issue on gh
i think that the problem is that now he has a day job
Kurtis Rainbolt-Greene
@krainboltgreene
May 02 2015 17:07
@fkchang: That's exactly why I created architecture.