These are chat archives for dropbox/pyston

28th
May 2015
Rudi Chen
@rudi-c
May 28 2015 00:14
Hmm what do you guys do if a test fails on Travis but not local machine?
Kevin Modzelewski
@kmod
May 28 2015 00:27
oof that looks annoying
I guess there are some ways that travis-ci differs from local machines
I think in this case it's that it starts with an empty pip cache
maybe try doing rm -rf ~/.cache/pip?
I guess as for your actual question, it's usually a matter of trying to see where it dies (adding more output if needed) and why it would behave differently there
oh man there's a lot going on in that traceback :)
alloc -> gc collection -> destructor -> raised exception -> failed attempt to get a traceback -> abort -> another failed attempt to get a traceback
Kevin Modzelewski
@kmod
May 28 2015 00:33
probably not the issue here, but we might need to not call destructors inside gc collections
or, need to change when we are allowed to do gc collections
I think there are cases that we can trigger a collection when it is not safe to run arbitrary python code
(or extension code)
Rudi Chen
@rudi-c
May 28 2015 00:35
Yeah rm -rf makes the test fail on my machine too
So an exception raised inside a finalizer is supposed to be ignored according the the Python website
and print a warning to stderr
I was actually going to do that next (on a simple test, it looks like we do ignore it, just need a warning message), but I'm surprised there's an exception at all.
In the test, that is.
Next on the TODO list is also to move some finalizer calls outside GC for performance purposes.
Chris Toshok
@toshok
May 28 2015 03:15
is the pip cache in $HOME?
Kevin Modzelewski
@kmod
May 28 2015 05:19
I think so?
Marius Wachtler
@undingen
May 28 2015 18:29
Concerning the additional integration tests: what should we do if not all tests pass (true for most of the libs): output the number of failures and make sure it's not increasing?
Chris Toshok
@toshok
May 28 2015 19:09
I don’t understand how it’s possible that two injections have left my face this numb (back from dentist :)
Chris Toshok
@toshok
May 28 2015 19:40
having a problem with unwinding through interpreter frames. it’s really confusing how the same code works for getTraceback() but not when used in the context of the c++ unwinder
it manifests itself as the ip (-1, using ip unchanged results in 0 stack frames showing up in the traceback) matching the interpreter frame, but the bp not existing in the map
Chris Toshok
@toshok
May 28 2015 19:53
ohh, ugh, I think I understand how it’s possible. RegisterHelper has already cleared the entry from s_interpreterMap (due to the exception unwinding through that frame)
then we resume unwinding, which finds the interpreter frame again, but there’s nothing in the map, so we crash
Marius Wachtler
@undingen
May 28 2015 19:56
mmh it's a "simple" example without generators?
Chris Toshok
@toshok
May 28 2015 19:56
yeah, doesn’t require generators at all. just throwing an exception that unwinds through an interpreter frame
the reason that it doesn’t crash getTraceback() is that we aren’t simultaneously unwinding (and calling c++ destructors)
the c++ unwind basically walks up the stack until it finds a spot where it needs to call something. that something can either be an explicit catch block or something the compiler emits to clean up automatically allocated C++ objects
if it’s the latter, we resume unwinding afterward
and that unwinding actually starts with the stack frame that cleaned up the c++ objects on the stack still. the unwinder banks on that IP not having a cleanup block of its own
man, so much magic
Chris Toshok
@toshok
May 28 2015 20:03
hm, the unwinder isn’t banking on there no cleanup blocks, since you could envision using an automatic storage c++ instance (StatCounter, etc) in a catch block.
but there isn’t a way to have an infinite number of them - they’re bounded by program size
Chris Toshok
@toshok
May 28 2015 20:47
(gdb) print *linf_infoSegmentation fault (core dumped)
$
Kevin Modzelewski
@kmod
May 28 2015 20:50
hmm what version of gdb?
Chris Toshok
@toshok
May 28 2015 20:51
7.6.2
very weird behavior when printing out some things while unwinding
Kevin Modzelewski
@kmod
May 28 2015 21:02
yeah I would definitely move to a newer gdb
:)
Chris Toshok
@toshok
May 28 2015 21:02
heh
hm, more wrinkles wrt incremental tracebacks
we never make it to the toplevel ASTInterpreter::execute frame
if the exception doesn’t also originate there, that is
so toplevel: try: raise Exception() except: … works and we’d get the right traceback
but def f(): raise Exception() … f() won’t work
because we catch the exception in ASTInterpreter::visit_invoke
Kevin Modzelewski
@kmod
May 28 2015 21:04
maybe we should call the equivalent of PyTraceBack_Here?
I guess I'm saying, maybe it's actually nice because we can do things the way cpython does in that case :)
Chris Toshok
@toshok
May 28 2015 21:06
hm, yeah we can just manually add the calling frame in the catch block
Marius Wachtler
@undingen
May 28 2015 21:08
or we maybe could workaround it if we remove the catch inside visit_invoke() and instead add a custom class which does in the destructor what currently the catch handler does but only if uncaught_exception return true
just as a last resort if there is now cleaner solution.
and not sure if this will even work :-D
Chris Toshok
@toshok
May 28 2015 21:13
yeah I don’ t think we can tell its uncaught until we’ve already unwound past that frame
so the dtor will have already been called
the additional if() here seems to work (and is pretty much the equivalent to PyTraceBack_Here): gist.github.com/toshok/9075ae87c90edf85adbf
Kevin Modzelewski
@kmod
May 28 2015 22:05
@undingen for the integration tests, yeah that seems like a good approach
(maybe assert on the exact number of failures to detect if new tests start passing)
Marius Wachtler
@undingen
May 28 2015 22:10
will do. It's really annoying that most projects don't include tests when installed thru pip from the repo:-(
Marius Wachtler
@undingen
May 28 2015 22:16
ok put this may come handy: I just saw that pip supports installing directly from git repos
Marius Wachtler
@undingen
May 28 2015 22:21
maybe better than adding a bunch of git submodules
Kevin Modzelewski
@kmod
May 28 2015 22:32
oh nice :)
yeah it's annoying to have to clone so many misc projects to build pyston
Chris Toshok
@toshok
May 28 2015 23:24
oh man.. raise0 (the raise line) shows up in incremental tracebacks, but only in jit-mode
adding another boolean flag, so we’re up to was_osr, was_interpreter, and skip_next_if_jitted
Chris Toshok
@toshok
May 28 2015 23:44
love modern compilers:
'memset' call operates on objects of type 'unw_context_t' (aka 'ucontext') while the size is based on a different type 'unw_context_t *' (aka 'ucontext *')
Chris Toshok
@toshok
May 28 2015 23:53
yay, looks like incremental tracebacks are working
Marius Wachtler
@undingen
May 28 2015 23:54
yeah!!!
Chris Toshok
@toshok
May 28 2015 23:54
real 0m7.921s for django-template.py now
which is actually pretty close to what it was (7.5s) with tracebacks disabled altogether