These are chat archives for dropbox/pyston

31st
Aug 2015
Rudi Chen
@rudi-c
Aug 31 2015 18:19
With all the custom unwinding stuff, are we still guaranteed that destructors for local variables are called?
I added a destructor to ASTInterpreter but in one particular instance of a call to astInterpretFunction, the destructor is never called.
I think an exception may be thrown in ASTInterpreter::execute but I'm not sure.
Marius Wachtler
@undingen
Aug 31 2015 18:34
I don't see a reason why they should not get called. It's not like we use setjump/longjump now instead or something like this.
Did your example involve generators?
Rudi Chen
@rudi-c
Aug 31 2015 18:40
How can I tell?
And how do they work?
Marius Wachtler
@undingen
Aug 31 2015 18:42
by context switching, but the problem is that if they exit we will just unmap the memory instead of exiting all frames and thereby calling the desturctors
Sun
@Daetalus
Aug 31 2015 20:20
Hi, @rudi-c will you continue to work for Pyston?
Rudi Chen
@rudi-c
Aug 31 2015 20:20
I can definitely keep contributing.
Won't be full time, but a PR may show up from time to time :P
Rudi Chen
@rudi-c
Aug 31 2015 20:29
@undingen Yup it was a generator. So it's not that the destructor isn't getting called, but it gets called out of order.
Sun
@Daetalus
Aug 31 2015 20:29
I see. I am busy until the end of the week. Hope I can dive deeper rather than only in the runtime. Thanks Rudi!
Rudi Chen
@rudi-c
Aug 31 2015 21:31
@undingen Destructors get called with munmap? How?
Marius Wachtler
@undingen
Aug 31 2015 21:31
no, they don't get called
Rudi Chen
@rudi-c
Aug 31 2015 21:32
Oh nvm I parsed your sentence wrong.
Marius Wachtler
@undingen
Aug 31 2015 21:32
at least not the c++ one, the GC registered destructor funcs should get called
Rudi Chen
@rudi-c
Aug 31 2015 21:33
So here's what I want to do: register some stack-allocated objects when they are created, because they need to have a gc visit function, and visit the registered objects during gc.
I'm currently trying to add those objects to a global unordered_map when they get created and remove them when they get destroyed.
But that doesn't work with generators because the destructor might not be called.
And the object will remain in the map and get scanned, which segfaults.
Is there some way I can make this work?
Marius Wachtler
@undingen
Aug 31 2015 21:39
that reminds me of the astinterpreter map we had and besides the performance the other reason why I removed it and replaced it with the "stackmem only" astrinterpreter instances were this problems.
I think what could work is: if a generator gets freed, iterating over your unordered_map and searching if any inststances are inside the memory range of the generator which should get freed
Rudi Chen
@rudi-c
Aug 31 2015 21:42
I see
Rudi Chen
@rudi-c
Aug 31 2015 21:49
Is there a way to create some per-thread-per-generator global object?
And easily access the one corresponding to the current thread & generator.
Kevin Modzelewski
@kmod
Aug 31 2015 21:52
that sounds like a good approach
we could stick the per-generator state on the generator object itself
and then have the generator gchandler visit that state
for the non-generator stuff maybe something in threading.cpp would be the right place to put it?
Marius Wachtler
@undingen
Aug 31 2015 21:53
yes, that sounds good :-)
Rudi Chen
@rudi-c
Aug 31 2015 21:59
One of the stack objects that I want to scan are rewriter objects. But it seem that the overhead of adding them to a list of objects to scan alone is 1-3% -_-
Marius Wachtler
@undingen
Aug 31 2015 22:06
scanning the pointers which are embedded in the generated machine code or scanning the rewriter instances introduces this overhead?
are you intercepting all rewriter::loadConst() calls and adding them to the list?
Rudi Chen
@rudi-c
Aug 31 2015 22:08
I'm not even scanning the rewriters right now. I'm just making the GC aware of their existance.
By adding and removing them from a list of objects to scan as they get created and removed from the stack.
Kevin Modzelewski
@kmod
Aug 31 2015 22:11
are you able to use a vector for that?
Rudi Chen
@rudi-c
Aug 31 2015 22:12
Yes, using a vector. With an unordered_map, it's closer to 7% perf hit.
Kevin Modzelewski
@kmod
Aug 31 2015 22:17
oh hmm that's surprising
could we avoid it if we didn't create a rewriter object?
ie avoid push/popping anything if it's just NULL (the vastly common case)
Rudi Chen
@rudi-c
Aug 31 2015 22:18
I already guard on if (obj)
Hmm let me just make sure that the perf hit does come from the push/popping the rewriter though. Now that I think about it there's a slight chance it comes from somewhere else.
Rudi Chen
@rudi-c
Aug 31 2015 22:38
Oh it's not as bad if I move everything into the header file (I guess the compiler then inlines the constructor/destructor?). In that case, the pushing/popping of the rewriter makes a consistent ~0.5% perf hit which is a lot better.