These are chat archives for dropbox/pyston

22nd
Jul 2015
Chris Toshok
@toshok
Jul 22 2015 00:11
okay that explains the “bug” - that’s the gcov for the first comparison/branch
yup
ugh
.L2:
        .cfi_def_cfa 7, 8
        .cfi_restore 6
        addq    $1, __gcov0._ZN6pystonL21_checkUnpackingLengthEll(%rip)
        ret
is the assembly
if I change that to:
.L2:
        addq    $1, __gcov0._ZN6pystonL21_checkUnpackingLengthEll(%rip)
        .cfi_def_cfa 7, 8
        .cfi_restore 6
        ret
the test passes
Rudi Chen
@rudi-c
Jul 22 2015 00:20
Two compiler bugs in a day yay
Rudi Chen
@rudi-c
Jul 22 2015 01:16
I gathered up the things needed to compile NumPy https://notes.dropbox.com/Get-NumPy-to-compile-pmZinTAhxvdWEnKGnZQSw
Kevin Modzelewski
@kmod
Jul 22 2015 01:59
after that it compiles?
Sun
@Daetalus
Jul 22 2015 02:06
Could you allow me to access the NumPy doc on Notes?
Kevin Modzelewski
@kmod
Jul 22 2015 03:04
hi daetalus, could you rebase #719?
it looks like there are some merge issues, and also that will get it to rebuild against the hopefully-now-fixed master
Sun
@Daetalus
Jul 22 2015 03:06
Sorry for that, I thought I already rebased. I rebase again!
And I will create PR about #598, now I am running the test.
Rudi Chen
@rudi-c
Jul 22 2015 03:17
I copied it over to my personal notes, the work notes doesn't allow public sharing
@kmod It passes setup.py build and setup.py install
Chris Toshok
@toshok
Jul 22 2015 03:18
nice
Sun
@Daetalus
Jul 22 2015 03:19
Thanks @rudi-c !
Rudi Chen
@rudi-c
Jul 22 2015 03:20
oh wait I get an email when someone external requests access
Well, either way, doesn't matter.
Kevin Modzelewski
@kmod
Jul 22 2015 03:26
oh nice! that's a lot farther than I would have thought
you could also try running it
depending on how many hacks you had to put in place to get it to build
Kevin Modzelewski
@kmod
Jul 22 2015 03:49
I think the build is fixed :)
Chris Toshok
@toshok
Jul 22 2015 03:49
:fireworks:
that is a lame :fireworks: :)
Ishan Khare
@ishankhare07
Jul 22 2015 06:28
i don't know if pyston has a mailing list like core-mentorship for newcomers who want to contribute
so i thought to just ask here itself
what are the best resources for start contributing into pyston?
and what might be the prerequisites for that?(llvm, compilers...?)
i'm fairly good at python and c/c++ stuff
Rudi Chen
@rudi-c
Jul 22 2015 06:29
The easiest way to get started with the codebase is to add missing features.
They tend not to require too much background.
Little things like this: dropbox/pyston#688
There's tons of these edge cases in Python that need to be supported, because there's always someone somewhere who relies on it.
Newly updated docs fresh from today :D
Sun
@Daetalus
Jul 22 2015 10:09
Does there has a way that just run one part of test? For example, just run the 8th part of test:pyston_defaults_integration ?
Marius Wachtler
@undingen
Jul 22 2015 10:20
I don't think there is a special build target. But you could manually run the tester.py and pass the test/integration directory
But I don't know if you have to pass special environments vars etc to get the same behavior as a travis-ci run
Sun
@Daetalus
Jul 22 2015 10:24
Thanks!
Marius Wachtler
@undingen
Jul 22 2015 10:43
I too managed to crash a compiler: clang 3.6 :-D.
  1. Running pass 'Function Pass Manager' on module 'eh_bjit.cpp'.
  2. Running pass 'X86 DAG->DAG Instruction Selection' on function '@_Z4bjitv'
    clang: error: unable to execute command: Segmentation fault (core dumped)
    clang: error: clang frontend command failed due to signal (use -v to see invocation)
    Ubuntu clang version 3.6.0-2ubuntu1 (tags/RELEASE_360/final) (based on LLVM 3.6.0)
Chris Toshok
@toshok
Jul 22 2015 17:23
what version of gcc are people running?
Travis Hance
@tjhance
Jul 22 2015 17:23
system gcc is 4.8.2
Chris Toshok
@toshok
Jul 22 2015 17:24
and of those running 4.8, which people have gmp <= 5.1.3?
Travis Hance
@tjhance
Jul 22 2015 17:24
how do I check my gmp version?
Chris Toshok
@toshok
Jul 22 2015 17:24
$ dpkg -l | grep gmp
Travis Hance
@tjhance
Jul 22 2015 17:24
5.1.3
Chris Toshok
@toshok
Jul 22 2015 17:25
okay
so the pgo code/ehframe-gen bug appears to not be in 4.9, but switching to 4.9 causes breakages with that version of gmp. just requires a few one line additions to pyston/libpypa. was hoping nobody had a system config like me, but I’m guessing ubuntu 14.04 is built such that it’s going to hit a lot of people
guessing you’re on 14.04 too @tjhance?
Travis Hance
@tjhance
Jul 22 2015 17:28
yep
14.04.1 to be specific
Marius Wachtler
@undingen
Jul 22 2015 17:32
I have 15.04 with 4.9.2 and gmp 6.0. Should I test if it's fixed with this configuration?
Chris Toshok
@toshok
Jul 22 2015 17:52
hrm, OSError: [Errno 28] No space left on device: ‘sqlalchemy_test_env_pyston'
@undingen: i think you’ll be fine right out of the box
Rudi Chen
@rudi-c
Jul 22 2015 18:25
So I tried running NumPy on a simple program (I added it to the notes doc). There's a couple problems involving PyType_Ready where NumPy does things we don't expect (like call it on a primitive type). Those shouldn't be hard to fix.
Then I get a segfault and don't know how to proceed from there.
(Or rather, get to a point where there's no easy workaround to move to the next bug)
Marius Wachtler
@undingen
Jul 22 2015 18:29
sketchily remembering that was also the point where I stopped...
where does the segfault raise?
Rudi Chen
@rudi-c
Jul 22 2015 18:32
add_scalarmath () at numpy/core/src/umath/scalarmath.c.src:1622
@name@_as_number.nb_index = Py@NAME@ArrType_Type.tp_as_number->nb_index;
Marius Wachtler
@undingen
Jul 22 2015 18:40
:-(. No idea, they only think I remember is seeing this @name@ names ...
Kevin Modzelewski
@kmod
Jul 22 2015 20:22
@Daetalus there are some helpers, you can try make check_dbg ARGS=testtorun
or check_release if it's happening in the release configuration
Chris Toshok
@toshok
Jul 22 2015 20:23
why is :shipit: this :shipit: ?
Kevin Modzelewski
@kmod
Jul 22 2015 20:23
???
Chris Toshok
@toshok
Jul 22 2015 20:31
as of 8d204ed, thanks to @rudi-c, we now properly destroy python objects
Marius Wachtler
@undingen
Jul 22 2015 20:35
:+1:
Travis Hance
@tjhance
Jul 22 2015 20:44
I don’t know but it’s my new favorite emoji
:shipit:
Chris Toshok
@toshok
Jul 22 2015 20:44
haha
Chris Toshok
@toshok
Jul 22 2015 21:20
after switching pgo to generating .gcda files in a different directory, gcc is now spitting this out during the -fprofile-use build:
../../src/runtime/cxx_unwind.cpp:708:1: warning: Missing counts for called function void pyston::panic()/4918
pretty sure that function wasn’t called in the training run...
Marius Wachtler
@undingen
Jul 22 2015 21:21
could someone please abort: build 2045. it's unnecessary...
Chris Toshok
@toshok
Jul 22 2015 21:21
done
Marius Wachtler
@undingen
Jul 22 2015 21:21
thanks
Marius Wachtler
@undingen
Jul 22 2015 21:44

looks like #706 + #720 + #736 will result in:

                           8d204ed19eb7469b2c:  d914b97c2665f6db95:
       django_template.py             3.3s (4)             3.2s (4)  -3.4%
            pyxl_bench.py             3.3s (4)             3.1s (4)  -5.3%
sqlalchemy_imperative2.py             3.8s (4)             3.6s (4)  -4.3%
                  geomean                 3.5s                 3.3s  -4.3%

:-)

Chris Toshok
@toshok
Jul 22 2015 21:44
niiiice
Marius Wachtler
@undingen
Jul 22 2015 21:46
  • pgo -> short week :-P
Chris Toshok
@toshok
Jul 22 2015 21:46
hah
Chris Toshok
@toshok
Jul 22 2015 21:56
what’s the normal workflow for adding workarounds/fixes/hacks to libpypa?
oh hm, our git submodule isn’t our fork
Marius Wachtler
@undingen
Jul 22 2015 21:58
yeah normally we create PR on https://github.com/vinzenz/libpypa
Kevin Modzelewski
@kmod
Jul 22 2015 22:31
would it be easy to get a newer gmp?
Kevin Modzelewski
@kmod
Jul 22 2015 22:53
hey marius did you confirm that your test function (the one the eh frame is for) and our assembly ends up spilling r12 to the same location?
Chris Toshok
@toshok
Jul 22 2015 23:43
man that was a pain to track down
the g++ build is invoking the linker with —as-needed
and for some reason gold is deciding that jemalloc isn’t needed
adding -Wl,—no-as-needed fixes the gcc build
Kevin Modzelewski
@kmod
Jul 22 2015 23:46
heh awesome :)