These are chat archives for dropbox/pyston

1st
Jul 2015
Marius Wachtler
@undingen
Jul 01 2015 14:57
looks like travis-ci is extremely slow today..
Chris Toshok
@toshok
Jul 01 2015 17:56
       django_template.py             4.6s (2)             0.7s (2)  -85.5%
            pyxl_bench.py             3.8s (2)             1.2s (2)  -67.9%
sqlalchemy_imperative2.py             4.9s (2)             0.7s (2)  -86.2%
        django_migrate.py             1.8s (2)             0.6s (2)  -65.8%
      virtualenv_bench.py             7.8s (2)             2.0s (2)  -74.5%
                  geomean                 4.1s                 0.9s  -77.7%
looks good, ship it
Travis Hance
@tjhance
Jul 01 2015 17:56
++
Marius Wachtler
@undingen
Jul 01 2015 17:59
did you remove the sleep call inside the main routine?
Chris Toshok
@toshok
Jul 01 2015 18:02
added a new compiler pass that is able to reduce the entire program to a single x86 instruction
Marius Wachtler
@undingen
Jul 01 2015 18:03
sounds legit, would invest :-D
Rudi Chen
@rudi-c
Jul 01 2015 21:42
Under what conditions will a Box have an attr_list?
From b->getHCAttrsPtr()->attr_list
Kevin Modzelewski
@kmod
Jul 01 2015 22:10
I think it's b->getHCAttrsPtr()->attributeArraySize() > 0
Rudi Chen
@rudi-c
Jul 01 2015 22:12
Sorry I meant, is there always a list, or do HCs get created as an optimization?
Kevin Modzelewski
@kmod
Jul 01 2015 22:18
err not sure what you mean
if b->cls->instancesHaveHCAttrs(), then you can do b->getHCAttrsPtr()
but attrs starts as NULL, and will only get initialized when it needs to have non-zero size
oh, sorry my earlier comment should have been
b->getHCAttrsPtr()->hcls->attributeArraySize() > 0
Rudi Chen
@rudi-c
Jul 01 2015 22:22
Unrelated : the following code
def generateClassAndInstances():           
    for i in xrange(5000):                 
        class NormalClass(object):         
            def __init__(self):            
                self.lst = [1, 2, "banana"]
            def __add__(self, other):      
                return 42                  
            def empty(self):               
                self.lst = []              

        normal = NormalClass()
triggers
../../src/codegen/irgen/hooks.cpp:189: pyston::CompiledFunction *pyston::compileFunction(pyston::CLFunction *, pyston::FunctionSpecialization *, pyston::EffortLevel, const pyston::
OSREntryDescriptor *): Assertion `f->versions.size() < 20' failed: __init__ 20
With the -n option
Kevin Modzelewski
@kmod
Jul 01 2015 22:35
oh hmm
maybe we should just have it bail on compiling the function at that point
rather than aborting
Chris Toshok
@toshok
Jul 01 2015 22:36
does -n really make sense anymore?
especially when the new jit pass goes in
actually with that example it should be possible to lift the class outside of the loop (outside of the function, really)
it doesn’t close over anything, does it?
Rudi Chen
@rudi-c
Jul 01 2015 22:38
Not in this case
Chris Toshok
@toshok
Jul 01 2015 22:39
does that actually work in python? would decorators leak that info somehow?
Rudi Chen
@rudi-c
Jul 01 2015 22:39
Lifting the class outside the loop?
Chris Toshok
@toshok
Jul 01 2015 22:41
yeah. i’m guessing that these two loops are not equivalent:
@mydecorator class Foo(object): pass … for i in xrange(5000): /* do something with Foo in here */
for i in xrange(5000): @mydecorator class Foo(object): pass /* do something with foo in here */
Travis Hance
@tjhance
Jul 01 2015 22:43
do you even need the decorator for it to be different
Rudi Chen
@rudi-c
Jul 01 2015 22:43
I can't think of an edge case with decorators, but I can think of an edge case where the function returns the class (and the callee expects the returned classes to be different instances or something), would be hard to detect.
Travis Hance
@tjhance
Jul 01 2015 22:43
like if it’s inside the loop, you get a different object each time you create the class, you can modify them in arbitrary ways
Chris Toshok
@toshok
Jul 01 2015 22:44
but that can be done similarly if classes get hoisted. just depends how things are desugared
Travis Hance
@tjhance
Jul 01 2015 22:45
also the definition of a class can contain arbitrary execution
Chris Toshok
@toshok
Jul 01 2015 22:45
like, if “class Foo(object) pass” gets desugared into two different things, a function that creates class Foo, and an assignment to a local, then the function can be hoisted
actually i suppose that would work even with decorators
Travis Hance
@tjhance
Jul 01 2015 22:45
oh, perhaps
Kevin Modzelewski
@kmod
Jul 01 2015 22:47
I think we kind of do what you're saying
we know that all the functions for the different classes
(including the function that creates them)
are all the same CLFunction
but we do need to create the class each time through the loop
Chris Toshok
@toshok
Jul 01 2015 22:49
oh, right, and that generates (presumably) a new CompiledFunction for __init__ specialized to the different type object
Chris Toshok
@toshok
Jul 01 2015 23:32
finally got quick_test passing with bitmap + run allocator, man that was a pain :/
was trying to keep using claimBlock and the multiple lists of blocks
but that doesn’t really make sense in a run allocator world - we have a list of runs of given size. if we’re out, we need to allocBlock
Marius Wachtler
@undingen
Jul 01 2015 23:38
what do you guys think about another attempt at integrating the v8 assembler classes? I think we pretty much hit the limitations of our own one now. e.g we miss instructions like xor, we don't support 32bit regs (would be helpfully if we assign a small number to a register which can get split in half e.g. mov 3, eax instead of mov 3, rax)
Travis Hance
@tjhance
Jul 01 2015 23:38
i don’t htink the v8 assembly is actually, like, comprehensive in this regard either
Marius Wachtler
@undingen
Jul 01 2015 23:39
I think even if we remove stuff we don't need and can't easily support it may be less work to have a modified fork of the few files in tree than our own assembler classes..
Kevin Modzelewski
@kmod
Jul 01 2015 23:40
I think even if the v8 one isn't the one to go with, it feels silly to build our own :/
Travis Hance
@tjhance
Jul 01 2015 23:40
yeah it does
Marius Wachtler
@undingen
Jul 01 2015 23:40
mmh ok.... I saw that the support the 32bit assignment trick. but I didn't look much further so yeah if they don't support much more than it doesn't make sense.
Travis Hance
@tjhance
Jul 01 2015 23:41
there was also the thing marius showed me, asmjit, https://github.com/kobalicek/asmjit
it might be a better choice due to being not highly coupled to an existing jit
the v8 assembler was more tangled up in js specific stuff than i had hoped
… I guess llvm doesn’t have anything useful (or we’d be using it)?
That’s kind of surprising
Chris Toshok
@toshok
Jul 01 2015 23:43
anyone look at dynasm before?
Travis Hance
@tjhance
Jul 01 2015 23:43
I mean, their disassembler works perfectly well
and llvm, well, definitely emits assembly code somehow
so, like, what’s the deal here?
Marius Wachtler
@undingen
Jul 01 2015 23:45
if think using llvm to go from asm string to machine code is also easy but going directly from code to machine code is hard
@toshok is that the luajit one?
Chris Toshok
@toshok
Jul 01 2015 23:46
yeah
i can’t recall if it’s more about emitting assembly or more about mixing c++/asm in the same file
likely the latter
Travis Hance
@tjhance
Jul 01 2015 23:47
as a counterpoint, we could stick with our own and make it good enough to be the One True Jit Assembler Library for everybody else to use:P
Chris Toshok
@toshok
Jul 01 2015 23:58
:+1: