These are chat archives for dropbox/pyston

8th
May 2015
Rudi Chen
@rudi-c
May 08 2015 00:36
Ah wops didn't pull and didn't realize intern() had been implemented already (#506)
Rudi Chen
@rudi-c
May 08 2015 00:57
Wait no the existing implementation uses a stub.
Chris Toshok
@toshok
May 08 2015 01:09
ugh:
Program received signal SIGSEGV, Segmentation fault.
pyston::sigprof_handler (unused=<optimized out>) at src/core/stats.cpp:162
that seems to happen if we get SIGPROF from inside swapContext
Kevin Modzelewski
@kmod
May 08 2015 01:47
maybe that's why there's normally a sigprocmask call there :/
Chris Toshok
@toshok
May 08 2015 02:05
yeah heh
i added it back around the calls to swapContext, but then it’s the thing showing up hottest :/
Travis Hance
@tjhance
May 08 2015 04:44
hey how do I make pyston with the profiling? when I try make pyston_profile I get
$ make pyston_profile
make: *** No rule to make target `stdlib.release.bc.o', needed by `pyston_profile'.  Stop.
Chris Toshok
@toshok
May 08 2015 05:01
we should really keep a thread local “top python frame"
instead of relying on libunwind for that
Marius Wachtler
@undingen
May 08 2015 07:49
@rudi-c Yes the 'intern()' I added is just a stub. (and comes without tests) I think the problem with yours is that it has to intern in place aka return the same object. I have a finished implementation around because babel uses it. Sorry for the duplication didn't want to steal the intern the intern function away :-D.
Marius Wachtler
@undingen
May 08 2015 07:58
@toshok the sigprocmask thing is unfortunate. If we have to add this back we will be again nearly as slow as makecontext :-(
Kevin Modzelewski
@kmod
May 08 2015 08:05
@tjhance you would probably have to do make pyston_profile USE_CMAKE=0
that's gprof-based profiling which I at least personally don't use any more
I would try doing make perf_testname
which has JIT support
Marius Wachtler
@undingen
May 08 2015 15:28
yeah finally found this stupid resource_filename() only supported for .egg, not .zip bug. We just set module.__file__ inside PyImport_ExecCodeModuleEx to late to the real filepath...
Travis Hance
@tjhance
May 08 2015 16:38
Well what I really wanted to do is make with the Stat allocations defined
How do i do that
Marius Wachtler
@undingen
May 08 2015 16:51
If I understood correctly what you want then you IMHO have to change manually stats.h STAT_ALLOCATIONS.
Chris Toshok
@toshok
May 08 2015 16:52
yup
Travis Hance
@tjhance
May 08 2015 16:52
Ohhh
Chris Toshok
@toshok
May 08 2015 16:53
it’s 0 && !STATS_DISABLED normally, s/0/1 and should be good to go
Rudi Chen
@rudi-c
May 08 2015 17:06
@undingen No worries! Let me know when the implementation is done, and I can merge in test file only?
Marius Wachtler
@undingen
May 08 2015 17:07
ok will do
Chris Toshok
@toshok
May 08 2015 17:31
so using setitimer to implement our own runtime sampling (to show area where the chosen_cf_body_jitted time was being spent) isn’t really giving all that much actionable intel :/
might need to work more on removing timer overhead, since we’ve got so many of them (~8,000,000,000 pause/resumes last time I looked, and I’ve added a lot more) even if the overhead is ~1us per swap...
Rudi Chen
@rudi-c
May 08 2015 18:11
@undingen I'll take a look at the setslice task.
Marius Wachtler
@undingen
May 08 2015 18:15
Cool thanks. getslice and delslice is also missing.
Chris Toshok
@toshok
May 08 2015 20:55
?value_e3
err
oops :)
Rudi Chen
@rudi-c
May 08 2015 21:33
Is there an easy way to print the AST corresponding to a python file?
Travis Hance
@tjhance
May 08 2015 21:34
pysotn’s ast or the True AST?
Rudi Chen
@rudi-c
May 08 2015 21:35
pyston
Travis Hance
@tjhance
May 08 2015 21:35
i think there’s a print or dump or something in codegen/ast.cpp
Rudi Chen
@rudi-c
May 08 2015 21:41
Ah there's a print_ast method. Output looks weird though.
Kevin Modzelewski
@kmod
May 08 2015 21:41
yeah I don't think we got that to support full files very well
I usually only look at it after the cfg phase
Rudi Chen
@rudi-c
May 08 2015 21:43
Does Pyston get an AST directly from parsing or does it get a C(oncrete)ST first?
Kevin Modzelewski
@kmod
May 08 2015 21:48
hmm I'm not sure what form libpypa returns, or how you would classify it
maybe just AST directly?
Rudi Chen
@rudi-c
May 08 2015 21:48
Oh wait, CFG = Context Free Grammar or Control Flow Graph in this context?
Michael Arntzenius
@rntz
May 08 2015 21:48
control flow graph
Rudi Chen
@rudi-c
May 08 2015 21:49
kk then probably AST directly
CST is a structure that has the same structure as the grammar usually
AST is trimmed down
Marius Wachtler
@undingen
May 08 2015 22:15
Our AST nodes are also kind of a bytecode for us. The AST will contain nodes which the parse tree does not contain e.g. AST_LangPrimitive.
When generating the CFG from the parsed nodes we will uses the same AST nodes (with a few exceptions) but they are in a form which is easier to interpret / jit / etc...
But there is sadly no clear barrier how a node looks like before and after CFG generation.
Marius Wachtler
@undingen
May 08 2015 22:21
For example: AST_Assign can have a std::vector of targets where it stores the value.
But after the CFG is generated it will only ever contain one entry....
Just mentioning it because you may find it a little bit confusing at first.
Marius Wachtler
@undingen
May 08 2015 23:23
ok enough for today. there are still a few workarounds in place which I want to get rid of over the weekend. but with this workarounds in place: the majority of the testcases of babel, protobuf, cheetah and mysql_python are passing.
maybe we should start next with getting pypy running inside pyston :-P
Chris Toshok
@toshok
May 08 2015 23:27
Is rpython actually a strict subset? Because that would be pretty hilarious :)
Marius Wachtler
@undingen
May 08 2015 23:28
I think so, I read some where that one can run it with cpython but it's really slow...
19.3. The Python Interpreter
Since RPython is a strict subset of Python, the PyPy Python interpreter can be run on top of another Python implementation untranslated. This is, of course, extremely slow but it makes it possible to quickly test changes in the interpreter. It also enables normal Python debugging tools to be used to debug the interpreter. Most of PyPy's interpreter tests can be run both on the untranslated interpreter and the translated interpreter. This allows quick testing during development as well as assurance that the translated interpreter behaves the same as the untranslated one.