These are chat archives for dropbox/pyston

17th
Jun 2015
Chris Toshok
@toshok
Jun 17 2015 05:20
I added a timer for the time spent unwinding, and it’s stupidly low on the incremental-traceback branch (235us)
Travis Hance
@tjhance
Jun 17 2015 05:49
I’m getting this when I run make pyston_dbg
ninja -C /home/tjhance/pyston/build/Debug pyston copy_stdlib copy_libpyston sharedmods ext_pyston ext_cpython 
ninja: Entering directory `/home/tjhance/pyston/build/Debug'
[4/4] Generating basic_test.pyston.so
FAILED: cd /home/tjhance/pyston/test/test_extension && /home/tjhance/pyston/build/Debug/pyston setup.py build --build-lib /home/tjhance/pyston/build/Debug/test/test_extension
Segmentation fault (core dumped)
[4/4] Generating ../lib_pyston/_multiprocessing.pyston.so, ../lib_pyston/pyexpat.pyston.so, ../lib_pyston/_elementtree.pyston.so
FAILED: cd /home/tjhance/pyston/from_cpython && /home/tjhance/pyston/build/Debug/pyston setup.py build --build-lib /home/tjhance/pyston/build/Debug/lib_pyston
Segmentation fault (core dumped)
ninja: build stopped: subcommand failed.
make: *** [pyston_dbg] Error 1
why is it calling setup.py with pyston rather than python? (I’m not turning on SELF_HOST, or anything, as far as I know)
Chris Toshok
@toshok
Jun 17 2015 06:00
you can change that in from_cpython/CMakeLists.txt
it unconditionally uses pyston there
Travis Hance
@tjhance
Jun 17 2015 06:01
ah
why?
it seems pretty silly that we need a working pyston before we can run make check
Chris Toshok
@toshok
Jun 17 2015 06:02
not sure - the toplevel CMakeLists.txt does:
# pyston self host mode
if(ENABLE_SELF_HOST)
  set(PYTHON_EXE "pyston")
else()
  find_program(PYTHON_EXE python)
endif()
maybe we need a similar thing in from_cpython's
Travis Hance
@tjhance
Jun 17 2015 06:07
something is really weird
n file included from /usr/include/python2.7/Python.h:128:0,
                 from Modules/_multiprocessing/multiprocessing.h:12,
                 from Modules/_multiprocessing/socket_connection.c:9:
Modules/_multiprocessing/socket_connection.c: In function ‘conn_poll’:
/usr/include/python2.7/ceval.h:133:54: error: ‘_save’ undeclared (first use in this function)
 #define Py_BLOCK_THREADS        PyEval_RestoreThread(_save);
                                                      ^
Modules/_multiprocessing/socket_connection.c:214:9: note: in expansion of macro ‘Py_BLOCK_THREADS’
         Py_BLOCK_THREADS
         ^
/usr/include/python2.7/ceval.h:133:54: note: each undeclared identifier is reported only once for each function it appears in
 #define Py_BLOCK_THREADS        PyEval_RestoreThread(_save);
                                                      ^
Modules/_multiprocessing/socket_connection.c:214:9: note: in expansion of macro ‘Py_BLOCK_THREADS’
         Py_BLOCK_THREADS
         ^
Kevin Modzelewski
@kmod
Jun 17 2015 06:17
so yeah, we unconditionally self-host right now
or at least, I think we should, I'm not sure if there are still hooks in the cmakelists
Travis Hance
@tjhance
Jun 17 2015 06:18
okay, so I’m not crazy:)
Kevin Modzelewski
@kmod
Jun 17 2015 06:18
this is what CPython does as well
I'm not sure if there's any way to build those parts with python instead of pyston
because distutils will try to build for the runtime that is running it
Travis Hance
@tjhance
Jun 17 2015 06:18
yeah but… I need to have a working pyston in order to run the tools for debugging pyston?
Kevin Modzelewski
@kmod
Jun 17 2015 06:19
as a workaround, you can turn off the stuff that gets built post-building pyston
by passing CMAKE_SHAREDMODS= RUN_DEPS= to your make line
we should find something better
Travis Hance
@tjhance
Jun 17 2015 06:19
okay
Kevin Modzelewski
@kmod
Jun 17 2015 06:19
I imagine this has to be much worse for projects that are much more integrally self-hosted
I wonder what they do
Travis Hance
@tjhance
Jun 17 2015 06:19
yeah like we should have pyston compile those only for the test cases that need them, or something
Kevin Modzelewski
@kmod
Jun 17 2015 06:20
hmm, I don't know how to have it know which tests will need which modules
I agree that this is an issue
even with the workaround it's really annoying to have to do that
Travis Hance
@tjhance
Jun 17 2015 06:20
maybe they have a stable version?
Kevin Modzelewski
@kmod
Jun 17 2015 06:21
so we have some build-ordering options that might help (or make it worse)
that right now, we rebuild all of the extension modules if pyston gets rebuilt
Travis Hance
@tjhance
Jun 17 2015 06:22
setting those variables didn’t help
Kevin Modzelewski
@kmod
Jun 17 2015 06:22
I think it's super rare that we make any changes that change the extension module output
with the BLOCK_THREADS thing?
sorry, haven't gotten to responding to that
Travis Hance
@tjhance
Jun 17 2015 06:22
no the BLOCK_THREADS thing is what i got when i replaced pyston with python
i presume that just own’t work
Kevin Modzelewski
@kmod
Jun 17 2015 06:22
oh
yeah
Travis Hance
@tjhance
Jun 17 2015 06:22
but I get
$ CMAKE_SHAREDMODS= RUN_DEPS= make check
make lint
make[1]: Entering directory `/home/tjhance/pyston'
pyston: linting...
Lint checks passed
make[1]: Leaving directory `/home/tjhance/pyston'
make check_format
make[1]: Entering directory `/home/tjhance/pyston'
ninja -C /home/tjhance/pyston/build/Release check-format
ninja: Entering directory `/home/tjhance/pyston/build/Release'
[1/1] cd /home/tjhance/pyston/src && /home/tjhance/pyston/tools/check_format.sh /home/tjhance/pyston/build/Release/llvm/./bin/clang-format
Format checks passed
make[1]: Leaving directory `/home/tjhance/pyston'
make pyston_dbg 
make[1]: Entering directory `/home/tjhance/pyston'
ninja -C /home/tjhance/pyston/build/Debug pyston copy_stdlib copy_libpyston sharedmods ext_pyston ext_cpython 
ninja: Entering directory `/home/tjhance/pyston/build/Debug'
[32/32] Generating ../lib_pyston/_multiprocessing.pyston.so, ../lib_pyston/pyexpat.pyston.so, ../lib_pyston/_elementtree.pyston.so
FAILED: cd /home/tjhance/pyston/from_cpython && /home/tjhance/pyston/build/Debug/pyston setup.py build --build-lib /home/tjhance/pyston/build/Debug/lib_pyston
Segmentation fault (core dumped)
[32/32] Generating basic_test.pyston.so
FAILED: cd /home/tjhance/pyston/test/test_extension && /home/tjhance/pyston/build/Debug/pyston setup.py build --build-lib /home/tjhance/pyston/build/Debug/test/test_extension
Segmentation fault (core dumped)
ninja: build stopped: subcommand failed.
make[1]: *** [pyston_dbg] Error 1
make[1]: Leaving directory `/home/tjhance/pyston'
make: *** [check] Error 2
Kevin Modzelewski
@kmod
Jun 17 2015 06:23
sorry, those should be command line arguments not environment variables
actually I thought it would use both
but anyway, it's worth trying it the other way
Travis Hance
@tjhance
Jun 17 2015 06:23
ooh
ah
this works
Kevin Modzelewski
@kmod
Jun 17 2015 06:24
so anyway, one option is to use the last-built pyston regardless of whether it is out of date
I think we can get the build system to do that
I had it accidentally working that way for a short while
I don't know if that will cause more issues or fewer
I think it would overall help
Travis Hance
@tjhance
Jun 17 2015 06:28
thanks
I opened an issue for this
Marius Wachtler
@undingen
Jun 17 2015 14:29
[LLVMdev] design question on inlining through statepoints and patchpoints
http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-June/086865.html
Chris Toshok
@toshok
Jun 17 2015 21:08
I’m really liking logByCurrentPythonLine. calls to gc_alloc by line of python: gist.github.com/toshok/baf68296735f9173321f
the -1 line numbers are when we can’t find a current statement