These are chat archives for dropbox/pyston

28th
May 2016
Marius Wachtler
@undingen
May 28 2016 14:46
Yes this is the patch. But it evolved by mainly trying to get lxml and numpy working and adding small workarounds (and removing some) for stuff which did not work etc... I did not inspect the cython code base and not even run the tests yet. So I expect that there are some bugs lurking around :-(. This definitely needs to get improved in the future see dropbox/pyston#1213 We are currently defining that we are pypy but we should create a pyston mode because for some of the code we are closer to cpython than pypy etc..
Sun
@Daetalus
May 28 2016 14:48
I am looking at the Cython.
Marius Wachtler
@undingen
May 28 2016 14:58
cool, did you try a different version of cython on scipy?
Nathaniel J. Smith
@njsmith
May 28 2016 15:01
PyPy is also a very fast moving target here, it's possible the cython pypy ifdefs aren't even appropriate for pypy now
It might even make sense to have some joint discussion with them about which cython hacks should be kept
Marius Wachtler
@undingen
May 28 2016 15:05
it also a bit unfortunate that the numpy and lxml packages contain the generated cython files which requires us currently to pip install directly from the git repo. I guess if pypy is changing in this area a lot too that they will run into similar problems with the pregenerated files.
Nathaniel J. Smith
@njsmith
May 28 2016 15:07
I don't think pypy needs any non-upstreamed patches to cython right now, but it's likely that there are #ifdef PYPY workarounds in upstream cython that are no longer necessary, as they've improved their c API support
Anyone at pycon btw?
Marius Wachtler
@undingen
May 28 2016 15:16
I think kmod is :-)
Marius Wachtler
@undingen
May 28 2016 18:38
@Daetalus so I have been able to install scipy after applying some hacks :+1: . I can execute some simple functions from the scipy tutorial but running the tests crashes immediately.
to workaround the cython crash I used the newest cython from the there gitrepo and applied our patch except of the first part which disables generating custom code object
I than removed the RELEASE_ASSERT(is_dummy, "not implemented"); in PyCode_New
Sun
@Daetalus
May 28 2016 18:40
Ah, thank you!
Marius Wachtler
@undingen
May 28 2016 18:41
than I only needed to workaround some build error because of some -Werror warning unreleated to pyston
and add a typedef PyIntObject PyBoolObject; plus remove the const from Py_buffer::format in object.h
hope this helps you to get started
the crash is related to the code object so maybe you should apply whole cython patch and not leave the first part out if it works with the patch.
Sun
@Daetalus
May 28 2016 18:54
Thank you, let me see...
Marius Wachtler
@undingen
May 28 2016 19:02
I think the crash could maybe also just be an issue of missing the call to PyType_Ready on the base class
Sun
@Daetalus
May 28 2016 19:11

to workaround the cython crash I used the newest cython from the there gitrepo and applied our patch except of the first part which disables generating custom code object

BTW, will it broken NumPy(I am checking it)?

Seems not.
Marius Wachtler
@undingen
May 28 2016 19:12
I did not test the whole numpy testsuite again so I don't know but uncommenting the assert is a hack because It makes it look like we support the PyCode_New api but we don't do...
It's just to get further I don't recommend this as a actual solution :-D
Sun
@Daetalus
May 28 2016 19:13
I see.
Marius Wachtler
@undingen
May 28 2016 19:16
but maybe you can even leave it in I just left it stock because I thought this could be why cython was crashing but I did two changes at the same time switch the cython version and remove this part of the patch so I don't know if it was only the version change which fixed the issue or both. But I did test removing it in the 0.22 version and it still crashed. So maybe it's just the version change which is necessary. But regenerating all the cython output for scipy takes a quite some time in a debug build so I don't want to try this again :-D
Sun
@Daetalus
May 28 2016 19:18
I see. How did you solved the errors like "error: implicit declaration of function ‘zcopy_’ [-Werror=implicit-function-declaration]"?
Marius Wachtler
@undingen
May 28 2016 19:23
I could not figure out where this flag comes from (CFLAGS did not allow me to overwrite it)
Sun
@Daetalus
May 28 2016 19:24
I unset it too. But not working.
Marius Wachtler
@undingen
May 28 2016 19:24

so I hacked in a

for arg in cc_args:
     if "error=implicit-function-declaration" not in arg:
       new_args.append(arg)
cc_args = new_args

inside CCompiler_compile in site-packages/numpy-1.11.0-py2.7-linux-x86_64.egg/numpy/distutils/ccompiler.py

after that it build :-D
so it's probably better to figure out where it get's this flag from instead of removing it like I did :-D
Sun
@Daetalus
May 28 2016 19:28
I will take a look, thanks!
Marius Wachtler
@undingen
May 28 2016 19:28
np
:-)
Kevin Modzelewski
@kmod
May 28 2016 20:25
dropbox/pyston@949dfeb
implicit function declaration is very scary :P
Sun
@Daetalus
May 28 2016 20:30
Thanks for point it out.