These are chat archives for dropbox/pyston

15th
Jul 2016
Marius Wachtler
@undingen
Jul 15 2016 00:00
fyi: I tagged 0.5.1 and published a short blog post https://blog.pyston.org/2016/07/14/pyston-0-5-1-released/ There are no docker images available yet
Kevin Modzelewski
@kmod
Jul 15 2016 00:09
Oh man, those are some pretty nice speedups!
Hi @mhsjlw! what's up?
Can we interest you in resurrecting the OSX-support project?
mhsjlw
@mhsjlw
Jul 15 2016 01:09
Hi, I'm back sorry about that
No promises that I can make it work, but I can definitely look into OSX-support
I'll dust off the old Mac Mini
:D
mhsjlw
@mhsjlw
Jul 15 2016 01:33
Ok, so I just wanted to try building it on Ubuntu 15.10 and got this error:
../../from_cpython/Modules/_ssl.c:318:33: error: implicit declaration of function 'SSLv3_method' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        self->ctx = SSL_CTX_new(SSLv3_method()); /* Set up context */
                                ^
../../from_cpython/Modules/_ssl.c:318:33: error: incompatible integer to pointer conversion passing 'int' to parameter of type 'const SSL_METHOD *' (aka 'const struct ssl_method_st *') [-Werror,-Wint-conversion]
        self->ctx = SSL_CTX_new(SSLv3_method()); /* Set up context */
                                ^~~~~~~~~~~~~~
/usr/include/openssl/ssl.h:2131:40: note: passing argument to parameter 'meth' here
SSL_CTX *SSL_CTX_new(const SSL_METHOD *meth);
                                       ^
2 errors generated.
[3348/3459] Linking CXX executable llvm/bin/clang-3.7
ninja: build stopped: subcommand failed.
Makefile:671: recipe for target 'pyston_dbg' failed
make: *** [pyston_dbg] Error 1
Weird... ?!
Kevin Modzelewski
@kmod
Jul 15 2016 03:23
oh interesting, I thought we had fixed the ssl issues
this is on master?
or maybe better yet, just copy the latest _ssl.c from CPython
We copied it from 2.7.7 but there have been changes to the ssl module in the later 2.7 releases
Kevin Modzelewski
@kmod
Jul 15 2016 03:30
I thought we had dealt with this but I guess now non-sslv3 openssl modules are getting rolled out?
mhsjlw
@mhsjlw
Jul 15 2016 11:04
Yeah it's on master
I'll give that a try
mhsjlw
@mhsjlw
Jul 15 2016 11:22
@kmod yup, that did it. Just replace the _ssl.c file, I used this GitHub commit: https://github.com/python/cpython/blob/5e086d53ae0edd771818f95ac12614f98878ad73/Modules/_ssl.c
Hmm, after a bit it seems to break though, because it needs some _ssl_data.h, so I added that
Then it gets upset that clinic/_ssl.c doesn't exist.
mhsjlw
@mhsjlw
Jul 15 2016 11:28
I've tried a bunch of things and I just can't figure it out :(
I'll just give it a try on mac
and I'll open a notepad to document my steps
mhsjlw
@mhsjlw
Jul 15 2016 11:42
Ok I've managed to install the libunwind headers, now I'll try running and example program
Marius Wachtler
@undingen
Jul 15 2016 11:46
:-)
mhsjlw
@mhsjlw
Jul 15 2016 11:48
I'm mostly following JuliaLang's installation and Makefile at this point
because they fully support Windows and OS X and they use libunwind
Marius Wachtler
@undingen
Jul 15 2016 11:50
Did you copy the ssl module from the 2.7 branch? https://github.com/python/cpython/blob/2.7/Modules/_ssl.c
I think you copied the one from the 3.x because of the clinic/ problem
mhsjlw
@mhsjlw
Jul 15 2016 11:50
:P
Ah yeah the master is 3.x
Ok, I'll give that a try
:D
Marius Wachtler
@undingen
Jul 15 2016 11:54
:+1:
mhsjlw
@mhsjlw
Jul 15 2016 11:57
Ah oops
I install libunwind into my path, I didn't know you distributed it with pyston
*installed
mhsjlw
@mhsjlw
Jul 15 2016 12:03
When you say patched for pyston
what changed did you make?
*changes
It's early :P
mhsjlw
@mhsjlw
Jul 15 2016 12:08
Well, there are a bunch of patches ... hmm
well the good news is that the headers look a lot like Ubuntu's
I'm using a newer version of that ^ but it's the idea
mhsjlw
@mhsjlw
Jul 15 2016 12:14
But I really need to know what your patches are, and if they'll break this
I'm trying to find them
Marius Wachtler
@undingen
Jul 15 2016 12:28
I think our patches are only too improve performance, I don't think you want to (and very likely can't) apply them on apples libunwind
mhsjlw
@mhsjlw
Jul 15 2016 12:32
Ok good :D
Marius Wachtler
@undingen
Jul 15 2016 12:33
In general I think getting pyston running on os x may be quite complicated. I suggest starting with getting the interpreter running by setting FORCE_INTERPRETER = true in src/core/options.cpp and setting ENABLE_RUNTIME_ICS = 0
and ENABLE_BASELINEJIT = false
this will disable all JIT compilation and will only use the interpreter
mhsjlw
@mhsjlw
Jul 15 2016 12:35
sounds good, thanks
still cracking away at getting libunwind to link properly without destroying your cmake files
Marius Wachtler
@undingen
Jul 15 2016 12:39
:-(, sadly can't really help with the cmake stuff
mhsjlw
@mhsjlw
Jul 15 2016 12:43
no problem
I've really only written CMake for Linux before and these dylibs are certainly a pain in the ass :P
mhsjlw
@mhsjlw
Jul 15 2016 13:06
@undingen hmm, by using the 2.7 branch's _ssl.c and _ssl_data.h it still breaks
../../from_cpython/Modules/_ssl.c:1470:9: error: implicit declaration of function 'Py_SETREF' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        Py_SETREF(self->ctx, (PySSLContext *)value);
        ^
1 error generated.
Ah lepton is trending today again!
Marius Wachtler
@undingen
Jul 15 2016 13:08
we don't have this macros yet: you could add
#define Py_SETREF(op, op2)                      \
    do {                                        \
        PyObject *_py_tmp = (PyObject *)(op);   \
        (op) = (op2);                           \
        Py_DECREF(_py_tmp);                     \
    } while (0)

#define Py_XSETREF(op, op2)                     \
    do {                                        \
        PyObject *_py_tmp = (PyObject *)(op);   \
        (op) = (op2);                           \
        Py_XDECREF(_py_tmp);                    \
    } while (0)
mhsjlw
@mhsjlw
Jul 15 2016 13:09
By the way, awesome work on that
Marius Wachtler
@undingen
Jul 15 2016 13:09
yeah lepton got a lot of good repsonses :-)
mhsjlw
@mhsjlw
Jul 15 2016 13:09
@undingen I'll give it a try, thanks
Now you just have to compress video :P
Ok I think it's building now
should I make a pr for this?
also wait what? why did it build a debug?
the INSTALLING.md says 'defaults to Release'
weird
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 13:17
Is there an easy way to use Pyson with travis-ci?
*Pyston :D
I was thinking it would be fun to start testing Pyston compatibility against my own libraries.
Marius Wachtler
@undingen
Jul 15 2016 13:20
I'm never using the cmake directly with pyston I always use our makefile wrapper make pyston_release and make pyston_dbg
mhsjlw
@mhsjlw
Jul 15 2016 13:21
@eandersson i'm sure you could write a quick script to build it
Marius Wachtler
@undingen
Jul 15 2016 13:21
mmhh I'm not sure I think if travis-ci supports it you can load one of our docker images otherwise you will probably have to write a script which download the release package.
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 13:22
@mhsjlw: I am sure I can! I am just lazy and was hoping someone would have a quick script already. :smile:
Marius Wachtler
@undingen
Jul 15 2016 13:22
I'm happy to hear about how your libs are running under pyston :-)
mhsjlw
@mhsjlw
Jul 15 2016 13:23
@eandersson once I figure out how to build pyston in release I can probably whip one up
ah wait ok pyston_release
Marius Wachtler
@undingen
Jul 15 2016 13:23
Currently it's more the other direction: we run some popular python libs in our travis-ci build in order to make sure pyston patches don't brake them.
mhsjlw
@mhsjlw
Jul 15 2016 13:23
i'll give that a try
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 13:23
Thanks. I'll try it locally with the docker image for now.
mhsjlw
@mhsjlw
Jul 15 2016 13:24
Maybe it's time for someone to write a roswell of python
choose a python implementation, install it and use it
with a portable working enviroment
or just use docker ;)
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 13:25
That would be pretty sweet.
mhsjlw
@mhsjlw
Jul 15 2016 13:25
@undingen do you upload your pyston artifacts to Travis CI? that could be an option
then you could just pull a certain travis build's pyston executable
if it builds on travis, it runs on travis ;)
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 13:26
How is compatibility with the general tools; such as PIP?
Marius Wachtler
@undingen
Jul 15 2016 13:26
@mhsjlw I don't think we do this currently.
mhsjlw
@mhsjlw
Jul 15 2016 13:26
I'm still trying to install numpy >_<
@undingen unfortunately, you cant. I don't actually use travis I prefer CircleCI and travis can't upload aftifacts but circleci can
Marius Wachtler
@undingen
Jul 15 2016 13:27
we have a patched pip because we needed to change some library directories/ etc and we did not upstream our patches yet...
mhsjlw
@mhsjlw
Jul 15 2016 13:27
i'm not sure if you guys have an S3 server (unlikely since you're dropbox :D) but you can setup travis to upload to S3
hmm
Marius Wachtler
@undingen
Jul 15 2016 13:28
If you just want to try out numpy you can use: docker run -it pyston/pyston-numpy:0.5.1
mhsjlw
@mhsjlw
Jul 15 2016 13:29
ah thanks :D
@eandersson if you're interested, I could just build a pyston executable on Ubuntu and send it to you if you want to upload it somewhere and use it for builds?
Marius Wachtler
@undingen
Jul 15 2016 13:30
our binary release and the docker image comes with virtualenv and pip
mhsjlw
@mhsjlw
Jul 15 2016 13:30
wait there are binary releases?
:D
mhsjlw
@mhsjlw
Jul 15 2016 13:31
ah nice, thanks
Marius Wachtler
@undingen
Jul 15 2016 13:31
:-)
mhsjlw
@mhsjlw
Jul 15 2016 13:31
WOAH
Marius Wachtler
@undingen
Jul 15 2016 13:31
compiling it from scratch is a pain because compiling llvm takes so long...
mhsjlw
@mhsjlw
Jul 15 2016 13:31
my computer monitor just flickered and ubuntu said clang failed
then I open the terminal and clang segfaulted
xD
Marius Wachtler
@undingen
Jul 15 2016 13:31
:-P
mhsjlw
@mhsjlw
Jul 15 2016 13:32
clang: error: unable to execute command: Segmentation fault (core dumped)
that hasn't happened to me before :P
I really wanted to try making a high-preformance server-side flatbuffer game server
that's why I'm here
and now I'm trying to make pyston build on mac osx
which by the way I think I figured out the libunwind
Marius Wachtler
@undingen
Jul 15 2016 13:44
unfortunately getting pyston working on os x is a big project and not only a build problem AFAIK.
but I have not tried it and don't have a os x so maybe I'm wrong :-)
mhsjlw
@mhsjlw
Jul 15 2016 14:06
yeah i'm sure that if i even get it build it will be disasterous
*built
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 18:17
So my library runs fine using the latest docker image. Which is awesome. I just need to figure out how to set it up with travis now.
Erik Olof Gunnar Andersson
@eandersson
Jul 15 2016 19:16
Only had a single unit test fail, something about ipv6.