These are chat archives for dropbox/pyston

Jun 2015
Chris Toshok
Jun 09 2015 00:01 UTC
is there any reason to have a public/internal thread state?
i mean, any reason they couldn’t be the same struct
Marius Wachtler
Jun 09 2015 00:04 UTC
It converts just a loop. But the overhead to generate machine code should be extremely small and it uses the interpreter frame so aborting is very cheap. But on the other hand maybe it will fail to speed up the interpreter by much if it's so similar.
Kevin Modzelewski
Jun 09 2015 00:09 UTC
I think the original reason for the split was because we had a thread state, and then we added support for PyThreadState
I guess it's more "pyston-specific" than internal
Chris Toshok
Jun 09 2015 00:09 UTC
we’ll end up needing another filler char[] (for the std::vector<> in ThreadStateInternal) if we were merge them
Kevin Modzelewski
Jun 09 2015 00:09 UTC
though I guess there are some things in there that shouldn't really be exposed?
hmm I'm not sure we should merge them
Chris Toshok
Jun 09 2015 00:10 UTC
yeah, i expect if they were merged we’d end up wanting some more opaque PyThreadState structure
okay, will move the class decl to threading.h (if I can, haven’t looked at all the references within threading.cpp) and add inlinable accessors for the hot stuff
Chris Toshok
Jun 09 2015 00:59 UTC
weird, even adding inlined (static) methods it’s still .2 seconds slower
Travis Hance
Jun 09 2015 02:14 UTC
Time to read the compiled assembly? :D
Chris Toshok
Jun 09 2015 17:22 UTC
adding inline appears to have done the trick. the compiler’s ways are mysterious
the body of the function was return current_thread_state->unwind_state; that seems like something that would never fail to inline :)
Rudi Chen
Jun 09 2015 18:18 UTC
Is it a problem that GC allocations can happen during garbage collection?
#465 0x00000000015b7397 in gc_alloc (bytes=24, kind_id=pyston::gc::CONSERVATIVE)
    at /home/vagrant/pyston/src/gc/gc_alloc.h:47
#466 0x0000000000803c9c in pyston::StlCompatAllocator<std::_List_node<_PyWeakReference*> >::allocate (
    this=0x7ffffffed5b8, n=1) at /home/vagrant/pyston/src/gc/heap.h:54
#467 0x0000000000803bbf in std::_List_base<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_get_node (this=0x7ffffffed5b8)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:334
#468 0x0000000000803ae0 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_create_node<_PyWeakReference* const&> (this=0x7ffffffed5b8, __args=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:502
#469 0x0000000000803a64 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_insert<_PyWeakReference* const&> (this=0x7ffffffed5b8, __position=..., __args=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:1561
#470 0x0000000000803805 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::push_back
    (this=0x7ffffffed5b8, __x=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:1016
#471 0x000000000080348c in pyston::gc::runCollection () at /home/vagrant/pyston/src/gc/collector.cpp:606
#472 0x000000000080b77d in pyston::gc::registerGCManagedBytes (bytes=74785)
    at /home/vagrant/pyston/src/gc/heap.cpp:135
#473 0x000000000080e5b0 in pyston::gc::LargeArena::alloc (this=0x1e85668 <pyston::gc::global_heap+456>, size=74785)
    at /home/vagrant/pyston/src/gc/heap.cpp:730
#474 0x00000000006a4c2d in pyston::gc::Heap::alloc (this=0x1e854a0 <pyston::gc::global_heap>, bytes=74785)
    at /home/vagrant/pyston/src/gc/heap.h:545
#475 0x00000000015b73b7 in gc_alloc (bytes=74777, kind_id=pyston::gc::PYTHON)
    at /home/vagrant/pyston/src/gc/gc_alloc.h:69
#476 0x00000000008b9a70 in pyston::BoxedString::operator new (size=24, nitems=74752)
    at /home/vagrant/pyston/src/runtime/types.h:456
Specifically, adding to a std::list can lead to a call to gc_alloc inside the "should_not_reenter" portion.
Chris Toshok
Jun 09 2015 18:47 UTC
Which std::list is this?
Rudi Chen
Jun 09 2015 18:48 UTC
std::list<PyWeakReference*, StlCompatAllocator<PyWeakReference*>> weak_references; (the one that gets populated during the sweep phase with weakly-referenced objects)
In runCollection()
Chris Toshok
Jun 09 2015 18:50 UTC
Yeah, I don't think it's a good thing for the gc to be using gc allocations like that. We should probably be doing the same thing as with the objects to finalize list - keep it in a static and traverse it at mark time (instead of using stlcompatallocator)
And also use a while !list.empty() loop to traverse it, since callbacks can add more to it
Rudi Chen
Jun 09 2015 18:54 UTC
The list only gets populated during sweeping though, I don't think the list can grow while being cleared (it's in the gc reenter section).
*no reenter section