These are chat archives for dropbox/pyston

11th
Apr 2015
Travis Hance
@tjhance
Apr 11 2015 00:11
I didn’t follow that strain of logic at all, can you elaborate?
like it went from ‘exceptions and iterobject.cpp and tp_as_sequence’ to ‘custom libgcc’ and I got lost somewhere in between
:P
Chris Toshok
@toshok
Apr 11 2015 00:12
the hasnext methods in iterobject rely on exceptions being thrown by getitem (and caught in iterobject.cpp)
turns out django throws a ton of those
Travis Hance
@tjhance
Apr 11 2015 00:13
right
Chris Toshok
@toshok
Apr 11 2015 00:13
so I figured maybe we could relieve ourselves of all those exceptions by instead using tp_as_sequence->sq_length and tp_as_sequence->sq_item from the given iterable’s cls
but that doesn’t work because __len__ is called at the wrong time, etc
Travis Hance
@tjhance
Apr 11 2015 00:14
okay
Chris Toshok
@toshok
Apr 11 2015 00:15
so I can’t reduce # exceptions thrown as much as I’d like, so I’m going to try and see if I can reduce the cost of them
Travis Hance
@tjhance
Apr 11 2015 00:15
oh ic
Chris Toshok
@toshok
Apr 11 2015 00:15
_Unwind_Find_FDE is showing up pretty hot in django (~30-40% of cpu time over 10,000 requests)
which is kinda nuts
Travis Hance
@tjhance
Apr 11 2015 00:16
hm isn't arntz working on a custom unwinder that should help?
Kevin Modzelewski
@kmod
Apr 11 2015 00:16
why does that cause us to call __len__ at the wrong time?
Travis Hance
@tjhance
Apr 11 2015 00:16
how does that tie in
Chris Toshok
@toshok
Apr 11 2015 00:16
the iterobject code just increments its idea of the current index and calls getitem()
never calls __len__ at all
calling tp_as_sequence->sq_length = calling __len__
wow, lovely formatting there, gitter :)
Kevin Modzelewski
@kmod
Apr 11 2015 00:18
hmm but that's how CPython does it
those are their fields after all :P
Chris Toshok
@toshok
Apr 11 2015 00:19
test/tests/unpacking_exceptions.py shows no len in the output though
just:
__getitem__ 0
__getitem__ 1
Kevin Modzelewski
@kmod
Apr 11 2015 00:19
I guess what I'm saying is that the move from using the Python attributes (which throw Pyston exceptions) to the C API ones (which throw C API [non-unwinding] exceptions) shouldn't change the semantics
I guess I'm not sure what else got changed
Chris Toshok
@toshok
Apr 11 2015 00:20
oh hm
i had added the check for if the index was >= the length and would return false there
won’t tp_as_sequence->sq_item just call into __getitem__ and catch the exception if thrown?
and return NULL with the C api exception set
Kevin Modzelewski
@kmod
Apr 11 2015 00:22
sorry, we're talking about two different things at the same time and I think I'm getting confused
one is the change between Pyston exceptions and C API exceptions
and the other is changing the iterator protocol in a specific place
I guess I'm curious what kinds of objects we create seqiters for even if we know the length
since those are just for supporting old-style iteration, but if the class declares a length it probably supports new-style __iter__ iteration?
Chris Toshok
@toshok
Apr 11 2015 00:25
not in the test, but code in the wild may
Kevin Modzelewski
@kmod
Apr 11 2015 00:29
oh hmm, what objects are they?
Chris Toshok
@toshok
Apr 11 2015 01:05
in the unpacking_exceptions.py test it’s just an object subclass with __getitem__ and __len__ defined