These are chat archives for dropbox/pyston

9th
Jun 2015
Chris Toshok
@toshok
Jun 09 2015 00:01
is there any reason to have a public/internal thread state?
i mean, any reason they couldn’t be the same struct
Marius Wachtler
@undingen
Jun 09 2015 00:04
It converts just a loop. But the overhead to generate machine code should be extremely small and it uses the interpreter frame so aborting is very cheap. But on the other hand maybe it will fail to speed up the interpreter by much if it's so similar.
Kevin Modzelewski
@kmod
Jun 09 2015 00:09
I think the original reason for the split was because we had a thread state, and then we added support for PyThreadState
I guess it's more "pyston-specific" than internal
Chris Toshok
@toshok
Jun 09 2015 00:09
we’ll end up needing another filler char[] (for the std::vector<> in ThreadStateInternal) if we were merge them
Kevin Modzelewski
@kmod
Jun 09 2015 00:09
though I guess there are some things in there that shouldn't really be exposed?
hmm I'm not sure we should merge them
Chris Toshok
@toshok
Jun 09 2015 00:10
yeah, i expect if they were merged we’d end up wanting some more opaque PyThreadState structure
yeah
okay, will move the class decl to threading.h (if I can, haven’t looked at all the references within threading.cpp) and add inlinable accessors for the hot stuff
Chris Toshok
@toshok
Jun 09 2015 00:59
weird, even adding inlined (static) methods it’s still .2 seconds slower
Travis Hance
@tjhance
Jun 09 2015 02:14
Time to read the compiled assembly? :D
Chris Toshok
@toshok
Jun 09 2015 17:22
adding inline appears to have done the trick. the compiler’s ways are mysterious
the body of the function was return current_thread_state->unwind_state; that seems like something that would never fail to inline :)
Rudi Chen
@rudi-c
Jun 09 2015 18:18
Is it a problem that GC allocations can happen during garbage collection?
#465 0x00000000015b7397 in gc_alloc (bytes=24, kind_id=pyston::gc::CONSERVATIVE)
    at /home/vagrant/pyston/src/gc/gc_alloc.h:47
#466 0x0000000000803c9c in pyston::StlCompatAllocator<std::_List_node<_PyWeakReference*> >::allocate (
    this=0x7ffffffed5b8, n=1) at /home/vagrant/pyston/src/gc/heap.h:54
#467 0x0000000000803bbf in std::_List_base<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_get_node (this=0x7ffffffed5b8)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:334
#468 0x0000000000803ae0 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_create_node<_PyWeakReference* const&> (this=0x7ffffffed5b8, __args=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:502
#469 0x0000000000803a64 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::_M_insert<_PyWeakReference* const&> (this=0x7ffffffed5b8, __position=..., __args=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:1561
#470 0x0000000000803805 in std::list<_PyWeakReference*, pyston::StlCompatAllocator<_PyWeakReference*> >::push_back
    (this=0x7ffffffed5b8, __x=@0x7ffffffed588: 0x1270b0d048)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../../include/c++/4.8/bits/stl_list.h:1016
#471 0x000000000080348c in pyston::gc::runCollection () at /home/vagrant/pyston/src/gc/collector.cpp:606
#472 0x000000000080b77d in pyston::gc::registerGCManagedBytes (bytes=74785)
    at /home/vagrant/pyston/src/gc/heap.cpp:135
#473 0x000000000080e5b0 in pyston::gc::LargeArena::alloc (this=0x1e85668 <pyston::gc::global_heap+456>, size=74785)
    at /home/vagrant/pyston/src/gc/heap.cpp:730
#474 0x00000000006a4c2d in pyston::gc::Heap::alloc (this=0x1e854a0 <pyston::gc::global_heap>, bytes=74785)
    at /home/vagrant/pyston/src/gc/heap.h:545
#475 0x00000000015b73b7 in gc_alloc (bytes=74777, kind_id=pyston::gc::PYTHON)
    at /home/vagrant/pyston/src/gc/gc_alloc.h:69
#476 0x00000000008b9a70 in pyston::BoxedString::operator new (size=24, nitems=74752)
    at /home/vagrant/pyston/src/runtime/types.h:456
Specifically, adding to a std::list can lead to a call to gc_alloc inside the "should_not_reenter" portion.
Chris Toshok
@toshok
Jun 09 2015 18:47
Which std::list is this?
Rudi Chen
@rudi-c
Jun 09 2015 18:48
std::list<PyWeakReference*, StlCompatAllocator<PyWeakReference*>> weak_references; (the one that gets populated during the sweep phase with weakly-referenced objects)
In runCollection()
Chris Toshok
@toshok
Jun 09 2015 18:50
Yeah, I don't think it's a good thing for the gc to be using gc allocations like that. We should probably be doing the same thing as with the objects to finalize list - keep it in a static and traverse it at mark time (instead of using stlcompatallocator)
And also use a while !list.empty() loop to traverse it, since callbacks can add more to it
Rudi Chen
@rudi-c
Jun 09 2015 18:54
The list only gets populated during sweeping though, I don't think the list can grow while being cleared (it's in the gc reenter section).
*no reenter section