These are chat archives for dropbox/pyston

15th
May 2015
Chris Toshok
@toshok
May 15 2015 00:03
sweet
import __pyston__

class Foo(object):
    pass


class Bar(object):
    pass

__pyston__.dumpTotalShape(Foo)
__pyston__.dumpTotalShape(Bar)
prints:
TotalShape(0x127004eae8)
TotalShape(0x127004eae8)
i don’t have a clean way of deterministically replaying attribute addition from the root hcls, so I always sort each level
but that won’t work for attributes added in different order
Chris Toshok
@toshok
May 15 2015 00:21
actually, where are class attributes stored? BoxedClass::attrs seems to only have __base__ for the classes I’m testing
Travis Hance
@tjhance
May 15 2015 00:22
wat really?
Chris Toshok
@toshok
May 15 2015 00:24

class attributes are defined like:

class Foo(object):
     class_attribute = 5

right?

Travis Hance
@tjhance
May 15 2015 00:25
yeah
is BoxedClass::attrs not what I think it is
BoxedClass::attrs is the same thing as Box::attrs, right?
Chris Toshok
@toshok
May 15 2015 00:26
i’m guessing i might have to look at b_cls->getHCAttrsPtr() ?
not sure. will verify
Travis Hance
@tjhance
May 15 2015 00:26
yeah getHCAttrsPtr()
should definitely be correct
I don’t know what BoxedClass::attrs is, come to think of it
but it’s probably what getHCAttrsPtr() points to
and Box::attrs can’t possibly be a thing, since the attrs are at a variable offset (so what I said above is probably wrong)
i should probably look at code before I say anything else:P
Kevin Modzelewski
@kmod
May 15 2015 00:28
yeah I think getHCAttrsPtr should return ::attrs
Chris Toshok
@toshok
May 15 2015 00:28
yeah, i figured type_cls’s hcattrs offset would point to ::attrs
Kevin Modzelewski
@kmod
May 15 2015 00:29
and hmm the class attributes should be stored in ::attrs
Chris Toshok
@toshok
May 15 2015 00:38
(gdb) print b->getHCAttrsOffset()
$3 = 384
(gdb) print (uintptr_t)&b->attrs - (uintptr_t)b
$5 = 384
b is a BoxedClass*
how does the AttrwrapperChild play into things?
Rudi Chen
@rudi-c
May 15 2015 18:12
Do we run pip under Pyston? For example, when in virtualenv_test.py we call 'pip install'.
Marius Wachtler
@undingen
May 15 2015 18:17
yes we can. virtualenv installs pip and distutils by default.
oh I think I meant setuptools not distutils
if 'pip search' or 'pip uninstall' does not work you may have to do a git submodule update.
Rudi Chen
@rudi-c
May 15 2015 18:27
I get that "pip install bcrypt==1.1.0 python-gflags==2.0 sqlalchemy==1.0.0" fails with "Could not find any downloads that satisfy the requirement bcrypt==1.1.0
No distributions at all found for bcrypt==1.1.0"
Due to some changes I made for indexing/slicing
But it's hard to figure out what :P
Chris Toshok
@toshok
May 15 2015 18:33
are any of the test/tests/* failing?
Rudi Chen
@rudi-c
May 15 2015 18:35
I can pass all of them now
Marius Wachtler
@undingen
May 15 2015 18:41
@toshok this should remove the need for calling PyGC_AddRoot when storing python objects inside static vars.
But this won't fix extensions which allocate memory and then store Python object inside it. Eg: alloc C++ class which has fields to Python objects with new. But I did not yet encounter that
Kevin Modzelewski
@kmod
May 15 2015 20:23
@rudi-c we run . test_env/bin/activate in that test; activate is a virtualenv helper script that sets your path to include test_env/bin/
so python resolves to test_env/bin/python, and pip resolves to test_env/bin/pip, which has a shebang line of something like test_env/bin/python
Chris Toshok
@toshok
May 15 2015 21:25
wow, so:
(gdb) print EmptyKey
$3 = 18446744073709551615
(gdb) print Val
$4 = (const unsigned long &) @0x7fffffff8d38: 18446744073709551615
$4 is the hash I compute for the local shape. EmptyKey is llvm::DenseMap’s “this slot is empty" value
what are the odds?
Kevin Modzelewski
@kmod
May 15 2015 21:32
>>> hex(18446744073709551615)
'0xffffffffffffffffL'
:P
Chris Toshok
@toshok
May 15 2015 21:32
it’s much more impressive in decimal :)
although that does point to a problem in my hashing...
sweet:
   0x208c8a5:   mov    (%rdi),%rax; load obj->cls
   0x208c8a8:   cmpq   $0x208dd00,0x1d0(%rax); compare obj->cls->total_shape to expected shape
   0x208c8b3:   jne    0x208c944; doesn’t match
   0x208c8b9:   mov    0x150(%rax),%rax; load tp_mro into rax
   0x208c8c0:   mov    0x10(%rax),%rax; load &tp_mro->elts[0] into rax
   0x208c8c4:   mov    (%rax),%rax; load the class at elts[0]
that’s from a callattr for int’s hash
shape compare, then a load of the mro and a load of the first element in the mro - since we know that for that shape the hash method is in mro[0] (there might be too many ()’s in there)
actually it looks right
Chris Toshok
@toshok
May 15 2015 22:10
okay cool. the generic ic for createBoxedIterWrapperIfNeeded only has 2 slots in it (instead of going megamorphic)
Kevin Modzelewski
@kmod
May 15 2015 22:45
hey chris, am I reading this right --
if we call Heap::free() on an object that has weakrefs to it
we will free the result regardless of whether it had weakrefs to it?
ie destructContents doesn't check the return value of _doFree
oh I guess it will assert in that case
Chris Toshok
@toshok
May 15 2015 22:48
hm, yeah that bit is rather unclean
we should probably just assert that ::free isn’t called on PYTHON objects?
Chris Toshok
@toshok
May 15 2015 22:56
hm, the following python code:
class IterableSub(Iterable):
    pass
gets compiled such that there’s a runtimeCall to <IterableSub_e2_3>
what does that function correspond to? does it create the BoxedClass?
Chris Toshok
@toshok
May 15 2015 23:04
hm
it looks like runtimeCall(or something below) currently guards on the instance you’re calling
if ($rdi == specific-pointer) { callq-specific-address } ...
Kevin Modzelewski
@kmod
May 15 2015 23:21
I believe it's guarding on the BoxedObject since it's going to embed pointers to the closure and/or default arguments
Chris Toshok
@toshok
May 15 2015 23:21
hrm, yeah that makes sense. we do all the guarding on argument types/defaults/kwargs/etc/etc along with guarding on the functino instance itself.
yeah
Kevin Modzelewski
@kmod
May 15 2015 23:21
we could maybe also guard on the CLFunction and then fetch the defaults/closure
Chris Toshok
@toshok
May 15 2015 23:22
i don’t even think that would help this particular (pathological) microbenchmark