These are chat archives for dropbox/pyston

23rd
Jul 2015
Chris Toshok
@toshok
Jul 23 2015 00:07
any bets on how long https://travis-ci.org/dropbox/pyston/jobs/72215486 will take? :) that should build pyston_release_gcc with PGO
i’m guessing > 1 hour
Chris Toshok
@toshok
Jul 23 2015 00:25
ugh, looks like ccache is defeating the pgo .o -> .gcda path magic :/
Kevin Modzelewski
@kmod
Jul 23 2015 00:37
hmm maybe we could give up on getting travis-ci to test the pgo configuration
Chris Toshok
@toshok
Jul 23 2015 00:38
yeah we might want to regardless
33 minutes and [1145/3122] through the first build
that’s applying the profile flags to llvm as well, though
once I get a good build locally I’ll compare timings +/- llvm
Chris Toshok
@toshok
Jul 23 2015 01:03
timings are great after I resolved the ccache problem
except for django_template2.py - for some reason it barely helps with that. my only guess is that since it’s not in combined.py, there’s some bit of runtime/llvm that’s not being exercised
@kmod are the comparison numbers your sent in email all done under perf (including cpython/pypy)?
ah, looks like no. the meeting_* table is investigate.py, and the other is the original benchmarking comparisons
Chris Toshok
@toshok
Jul 23 2015 01:20
pgo builds might be something that’s reserved for final testing for releases?
the number really are good, though, it’d be a shame to miss out on them day-to-day:
              pyston (calibration)                      :   1.00s clang: 1.01 ( -0.7%)
              pyston django_template2.py                :   3.78s clang: 4.19 ( -9.8%)
              pyston pyxl_bench.py                      :   2.72s clang: 2.99 ( -9.0%)
              pyston sqlalchemy_imperative2.py          :   3.20s clang: 3.52 ( -9.0%)
              pyston django_template.py                 :   2.76s clang: 3.11 (-11.4%)
              pyston (geomean-df76)                     :   3.21s clang: 3.53 ( -9.3%)
Kevin Modzelewski
@kmod
Jul 23 2015 01:32
nice :)
Travis Hance
@tjhance
Jul 23 2015 01:45
Is it possible to use profile information from one build for another build?
If so, you might be able to capture some amount of the pgo gain by caching the profiles
Without having an absurd build time
But maybe this isn't possible or too janky top be worth
Chris Toshok
@toshok
Jul 23 2015 01:47
Hm, not sure at what granularity gcov works (file or function) - but I think gcc can detect when profiling data doesn't match.
Ccache would need to be disabled but I'm already doing that
Sun
@Daetalus
Jul 23 2015 02:54
Hi @kmod , do you mind me to occupy your time? I have to questions about #739.
1st, How to check we found a factkey in kwargs, maybe to check whether it is callable?
2nd, I the third comment of #739, I don't understand why we shouldn't call PyObject_RichCompareBool if has extremElement.
Sorry, the first problem solved.
Kevin Modzelewski
@kmod
Jul 23 2015 03:49
oh sorry I think I had a typo in my comment
I meant !extremElement
(forgot the !)
ie when we look at the first element
we will assign it to extremElement
and then immediately compare it to itself
Sun
@Daetalus
Jul 23 2015 03:53
Got it, thanks!
And I don't quite understand "also a test that has a key function that prints when it gets evaluated". What does this test will look like?
Kevin Modzelewski
@kmod
Jul 23 2015 04:11
oh, I guess I meant to have a test that will make sure that we call the key function the right number of times with the right arguments
ex ```
def key(a, b):
  print a, b
print min(range(10), key=key)
Sun
@Daetalus
Jul 23 2015 04:20
Ok, thanks!
Marius Wachtler
@undingen
Jul 23 2015 05:51

yes I checked the assembly of the test function and our one:
start of our bjit function:

push   %r12
sub    $0x110,%rsp
mov    %rdi,%r12

start of the test func:

.cfi_startproc
    pushq    %r12
    .cfi_def_cfa_offset 16
    subq    $272, %rsp              # imm = 0x110
    .cfi_def_cfa_offset 288
    .cfi_offset %r12, -16
should be good / the same:-)
there are instructions after this ones (call to foo) but they don't add any EH entries so it's exactly the same as our bjit case
Sun
@Daetalus
Jul 23 2015 07:05

Hi, In CPython:

>>> ast.dump(ast.parse("1 ** -2"))
'Module(body=[Expr(value=BinOp(left=Num(n=1), op=Pow(), right=Num(n=-2)))])'
>>> ast.dump(ast.parse("-1 ** 2"))
'Module(body=[Expr(value=UnaryOp(op=USub(), operand=BinOp(left=Num(n=1), op=Pow(), right=Num(n=2))))])'

Negative sign in different place has different meaning . It is a bug or a feature? Pyston treat -1 ** 2 as (-1) ** 2, not -(1 ** 2).

Marius Wachtler
@undingen
Jul 23 2015 07:12
mmh but cpython outputs: -4 for -2 ** 2 so it also has to treat it as -((2) ** 2)
oh sorry miss read it...
I can't really reproduce the problem with pyston is that because I don't have your pow changes? If there is a problem it's pretty sure in libpypa and not intentional
you could also try running pyston with -x which uses cpythons parser (it executes a cpython with a script which serializes the ast)
Travis Hance
@tjhance
Jul 23 2015 07:18
I was thinking about writing some unit tests for libpypa, it could use more test love, I think
it should be really easy too, just take huge python files, dump them to AST with cpython, compare it to pyston
maybe generate some random python programs from the grammar, to get edge cases
anyway, my findings match marius’s:
>> -1 ** 2
-1
(with pyston)
Sun
@Daetalus
Jul 23 2015 07:22

I believe it is not related to my pow change. And the output that use libpypa is correct.

>>> ast.dump(ast.parse("-2 ** 2"))
'Module(body=[Expr(value=UnaryOp(op=USub(), operand=BinOp(left=Num(n=2), op=Pow(), right=Num(n=2))))])'

Even the result of -2 ** 2 is correct in CPython. But it shouldn't be according its ast. Correct me if I am wrong.

In CPython:

>>> -1 ** 2
-1
Travis Hance
@tjhance
Jul 23 2015 07:23
So above, you showed that the ast parses it as -(1 ** 2)
So this seems fine
Sun
@Daetalus
Jul 23 2015 07:23
Yes
Marius Wachtler
@undingen
Jul 23 2015 07:26
Yeah pypa has only very few tests :-(. But AFAIK the ast comparison stuff exists so it's only a matter of adding more tests.
Travis Hance
@tjhance
Jul 23 2015 07:27
I’m thinking just add all the cpython tests:)
Marius Wachtler
@undingen
Jul 23 2015 07:27
:-)
Sun
@Daetalus
Jul 23 2015 07:29
In CPython, ast also parses -2 ** 2 as -(2 ** 2), but return 4. It should parses it as (-2) ** 2. I believe the parse result of libpypa is same as CPython( But it is wrong).
Travis Hance
@tjhance
Jul 23 2015 07:32
interesting… -2 ** 2 evaluates to -4 for me, with cpython
what version of cpython are you using?
(although I can’t imagine that cpython would have changed this behavior between versions)
Sun
@Daetalus
Jul 23 2015 07:34
Sorry, my mistake...
You are correct
Travis Hance
@tjhance
Jul 23 2015 07:34
(unrelated) on jitdev I’m getting
/mnt/tjhance/pyston/src/runtime/types.h:817:1: error: static_assert failed “” static_assert(sizeof(BoxedDict) == sizeof(PyDictObject), "");
did something change on jitdev?
@toshok did you upgrade gcc or something?
Marius Wachtler
@undingen
Jul 23 2015 07:57
have to figure out what stuff I want to send to dublin :-(, guy from shipping company is calling later.
Sun
@Daetalus
Jul 23 2015 08:25
Hi guys, are you ok with curses_test in master branch? Which is failed on my computer.
Kevin Modzelewski
@kmod
Jul 23 2015 08:29
hi daetalus you need to do find -name '*.pyston.so' -delete
(in the pyston directory)
Sun
@Daetalus
Jul 23 2015 08:29
Ok, thank you!
Sun
@Daetalus
Jul 23 2015 10:41
@kmod #739 and #719 was updated. Please review if you have time.
Kevin Modzelewski
@kmod
Jul 23 2015 10:46
ok will do :)
fyi it's almost 4am here so I won't be able to get to it right away :/
Sun
@Daetalus
Jul 23 2015 10:49
That's OK. Have enough rest.
Chris Toshok
@toshok
Jul 23 2015 14:02
@tjhance ack, yes I did. I thought we had all split to different jitdevs
Chris Toshok
@toshok
Jul 23 2015 14:38
@tjhance does that happen on a fresh gcc build, or one with an existing build/ directory? it should be enough to remove the build.ninja file and rebuild, since we autodetect sizeof(std::unordered_map) now
Travis Hance
@tjhance
Jul 23 2015 15:08
I've been on the same jitdev since I first stated using it
Travis Hance
@tjhance
Jul 23 2015 16:22
that didn’t help
I removed build.ninja, ran make clean and then make pyston_release and got the same thing
Chris Toshok
@toshok
Jul 23 2015 16:32
wait make pyston_release is failing?
or pyston_release_gcc?
can you gist the failure messages? is it the std::max_align_t error? or something else (still the static_assert)?
Travis Hance
@tjhance
Jul 23 2015 16:36
make pyston_release is failing. It’s the BoxedDict static_assert
FAILED: cd /mnt/tjhance/pyston/build/Release/src/runtime/inline && /usr/bin/ccache /mnt/tjhance/pyston/build/Release/llvm/./bin/clang++ -DNVALGRIND -DTHREADING_USE_GIL=1 -DTHREADING_USE_GRWL=0 -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DDEFAULT_PYTHON_MAJOR_VERSION=2 -DDEFAULT_PYTHON_MINOR_VERSION=7 -DDEFAULT_PYTHON_MICRO_VERSION=6 -DLLVMREV=230300 -Qunused-arguments -fcolor-diagnostics -Qunused-arguments -fcolor-diagnostics -Wall -Wextra -Werror -Wreturn-type -Wno-sign-compare -Wno-unused -Wno-unused-parameter -fno-omit-frame-pointer -g -std=c++11 -fno-rtti -fexceptions -fvisibility-inlines-hidden -ffunction-sections -fdata-sections -Woverloaded-virtual -Wno-invalid-offsetof -Wcast-qual -Wno-sign-conversion -Wnon-virtual-dtor -Winit-self -Wmissing-include-dirs -Wstrict-overflow=5 -Wpointer-arith -Wtype-limits -Wwrite-strings -Wempty-body -Waggregate-return -Wmissing-field-initializers -Wredundant-decls -Winline -Wint-to-pointer-cast -Wlong-long -Wvla -Wno-attributes -g -Qunused-arguments -fcolor-diagnostics -Wimplicit-int -Wstrict-prototypes -Wold-style-definition -Wnested-externs -Wpointer-to-int-cast -Wno-mismatched-tags -Wno-extern-c-compat -Qunused-arguments -fcolor-diagnostics -Wimplicit-int -Wstrict-prototypes -Wold-style-definition -Wnested-externs -Wpointer-to-int-cast -Wno-mismatched-tags -Wno-extern-c-compat -O3 -DNDEBUG -fstrict-aliasing -enable-tbaa -DNVALGRIND -DBINARY_SUFFIX=_release -DBINARY_STRIPPED_SUFFIX= -I/mnt/tjhance/pyston/build/Release/from_cpython/Include -I/mnt/tjhance/pyston_deps/llvm-trunk/include -I/mnt/tjhance/pyston/build/Release/llvm/include -I/mnt/tjhance/pyston/build/Release -I/mnt/tjhance/pyston/build/Release/libunwind/include -I/mnt/tjhance/pyston/libpypa/src -I/mnt/tjhance/pyston/lz4/lib -I/mnt/tjhance/pyston/src -c /mnt/tjhance/pyston/src/runtime/inline/xrange.cpp -o xrange.cpp.bc -emit-llvm
In file included from /mnt/tjhance/pyston/src/runtime/inline/xrange.cpp:17:
/mnt/tjhance/pyston/src/runtime/types.h:817:1: error: static_assert failed ""
static_assert(sizeof(BoxedDict) == sizeof(PyDictObject), "");
^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Chris Toshok
@toshok
Jul 23 2015 16:39
sec, checking as well
Chris Toshok
@toshok
Jul 23 2015 16:50
hm, I don’t get that error - I am getting the max_align_t error, which I can fix by pulling out the change and getting a PR merged quickly (since vinzenz merged my PR there, I can bump everything)
are you using system gcc or a gcc from pyston_deps?
Chris Toshok
@toshok
Jul 23 2015 17:01
yeah with that fix pyston_release also builds here.
branch is here: https://github.com/toshok/pyston/tree/gcc-4.9-gmp-fix - waiting on quick_check before opening a PR
Travis Hance
@tjhance
Jul 23 2015 17:12
um, is there anything else I can try deleting?
Chris Toshok
@toshok
Jul 23 2015 17:15
can you gist the full build output until it fails, by any chance?
you could blow away the entire build/Release directory, but that shouldn’t change things
if build.ninja is missing cmake should re-run.. but maybe it caches enough that it won’t redo the sizeof calc
Travis Hance
@tjhance
Jul 23 2015 17:21
tjhance@jitdev:~/pyston (master f1129a84) $ make clean
tjhance@jitdev:~/pyston (master f1129a84) $ find . -name build.ninja
./build/Release/build.ninja
tjhance@jitdev:~/pyston (master f1129a84) $ rm build/Release/build.ninja 
tjhance@jitdev:~/pyston (master f1129a84) $ make pyston_release
make[1]: Entering directory `/mnt/tjhance/pyston'
make[1]: Leaving directory `/mnt/tjhance/pyston'
make[1]: Entering directory `/mnt/tjhance/pyston'
make[1]: Leaving directory `/mnt/tjhance/pyston'
cd /mnt/tjhance/pyston/build/Release; CC='clang' CXX='clang++' cmake -GNinja /mnt/tjhance/pyston -DTEST_THREADS=1 -DCMAKE_BUILD_TYPE=Release
-- found ccache /usr/bin/ccache
-- found the gold linker /usr/bin/ld.gold
-- Target triple: x86_64-unknown-linux-gnu
-- Native target architecture is X86
-- Threads enabled.
-- Doxygen disabled.
-- Sphinx disabled.
-- Go bindings disabled.
-- Could NOT find OCaml (missing:  OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- Could NOT find OCaml (missing:  OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- OCaml bindings disabled.
-- Building with -fPIC
-- Constructing LLVMBuild project information
-- Targeting X86
-- Could NOT find LibXml2 (missing:  LIBXML2_LIBRARIES LIBXML2_INCLUDE_DIR) 
-- Clang version: 3.7.0
-- 64 bit architecture detected size of void * is 8
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/tjhance/pyston/build/Release
ninja -C /mnt/tjhance/pyston/build/Release pyston copy_stdlib copy_libpyston sharedmods ext_pyston ext_cpython 
ninja: Entering directory `/mnt/tjhance/pyston/build/Release'
[7/116] Building LLVM bitcode dict.cpp.bc
FAILED: cd /mnt/tjhance/pyston/build/Release/src/runtime/inline && /usr/bin/ccache /mnt/tjhance/pyston/build/Release/llvm/./bin/clang++ -DNVALGRIND -DTHREADING_USE_GIL=1 -DTHREADING_USE_GRWL=0 -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DDEFAULT_PYTHON_MAJOR_VERSION=2 -DDEFAULT_PYTHON_MINOR_VERSION=7 -DDEFAULT_PYTHON_MICRO_VERSION=6 -DLLVMREV=230300 -Qunused-arguments -fcolor-diagnostics -Qunused-arguments -fcolor-diagnostics -Wall -Wextra -Werror -Wreturn-type -Wno-sign-compare -Wno-unused -Wno-unused-parameter -fno-omit-frame-pointer -g -std=c++11 -fno-rtti -fexceptions -fvisibility-inlines-hidden -ffunction-sections -fdata-sections -Woverloaded-virtual -Wno-invalid-offsetof -Wcast-qual -Wno-sign-conversion -Wnon-virtual-dtor -Winit-self -Wmissing-include-dirs -Wstrict-overflow=5 -Wpointer-arith -Wtype-limits -Wwrite-strings -Wempty-body -Waggregate-return -Wmissing-field-initializers -Wredundant-decls -Winline -Wint-to-pointer-cast -Wlong-long -Wvla -Wno-attributes -g -Qunused-arguments -fcolor-diagnostics -Wimplicit-int -Wstrict-prototypes -Wold-style-definition -Wnested-externs -Wpointer-to-int-cast -Wno-mismatched-tags -Wno-extern-c-compat -Qunused-arguments -fcolor-diagnostics -Wimplicit-int -Wstrict-prototypes -Wold-style-definition -Wnested-externs -Wpointer-to-int-cast -Wno-mismatched-tags -Wno-extern-c-compat -O3 -DNDEBUG -fstrict-aliasing -enable-tbaa -DNVALGRIND -DBINARY_SUFFIX=_release -DBINARY_STRIPPED_SUFFIX= -I/mnt/tjhance/pyston/build/Release/from_cpython/Include -I/mnt/tjhance/pyston_deps/llvm-trunk/include -I/mnt/tjhance/pyston/build/Release/llvm/include -I/mnt/tjhance/pyston/build/Release -I/mnt/tjhance/pyston/build/Release/libunwind/include -I/mnt/tjhance/pyston/libpypa/src -I/mnt/tjhance/pyston/lz4/lib -I/mnt/tjhance/pyston/src -c /mnt/tjhance/pyston/src/runtime/inline/dict.cpp -o dict.cpp.bc -emit-llvm
In file included from /mnt/tjhance/pyston/src/runtime/inline/dict.cpp:17:
In file included from /mnt/tjhance/pyston/src/runtime/dict.h:19:
/mnt/tjhance/pyston/src/runtime/types.h:817:1: error: static_assert failed ""
static_assert(sizeof(BoxedDict) == sizeof(PyDictObject), "");
^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Chris Toshok
@toshok
Jul 23 2015 17:25
hrm
what’s $ grep SIZEOF_UNORDEREDMAP build/Release/CMakeCache.txt show?
Travis Hance
@tjhance
Jul 23 2015 17:28
HAVE_SIZEOF_UNORDEREDMAP:INTERNAL=TRUE
SIZEOF_UNORDEREDMAP:INTERNAL=48
Chris Toshok
@toshok
Jul 23 2015 17:28
yeah that’s wrong. should be 56
have you tried rm -r build/Release + rebuild?
Travis Hance
@tjhance
Jul 23 2015 17:33
should i remove CMakeCache.txt?
or i’ll just remove teh whole thing
I would have thought make clean would do that
Chris Toshok
@toshok
Jul 23 2015 17:35
same, but i’m not sure our make clean does the right thing with cmake
nuke the site from orbit, it’s the only way to be sure
Chris Toshok
@toshok
Jul 23 2015 17:42
you’ll definitely want the changes on that branch though, or you’ll fail in a different way
Chris Toshok
@toshok
Jul 23 2015 18:19
interesting.. @rudi-c: pyston: /home/travis/build/dropbox/pyston/src/gc/heap.cpp:131: bool pyston::gc::hasOrderedFinalizer(pyston::BoxedClass *): Assertion!cls->tp_del' failed.`
Rudi Chen
@rudi-c
Jul 23 2015 18:19
What's the cls?
Chris Toshok
@toshok
Jul 23 2015 18:19
dunno :/
Rudi Chen
@rudi-c
Jul 23 2015 18:19
k lemme try to repro that
Chris Toshok
@toshok
Jul 23 2015 18:20
maybe change the assert to RELEASE_ASSERT(!cls->tp_del, “class %s has a tp_del”, cls->tp_name) ?
at least, i think that works
Daniel Agar
@dagar
Jul 23 2015 18:26
make clean should probably be deleting build/
Travis Hance
@tjhance
Jul 23 2015 18:40
okay, now I get the max_align_t error
Chris Toshok
@toshok
Jul 23 2015 18:41
okay cool
that’s fixed by that branch. kicked the clang build and hopefully that gc problem doesn’t happen again
Rudi Chen
@rudi-c
Jul 23 2015 18:43
?
Chris Toshok
@toshok
Jul 23 2015 18:43
thankfully either merging/cherry-picking/reproducing those changes won’t require a rebuild of much (libpypa/parser.cc, some stuff in src/runtime)
@rudi-c the tp_del assert
Rudi Chen
@rudi-c
Jul 23 2015 18:43
What do you mean by "fixed by that branch"?
Chris Toshok
@toshok
Jul 23 2015 18:44
oh, the max_align_t error that @tjhance is having
(fixed by #751 )
Rudi Chen
@rudi-c
Jul 23 2015 18:44
Are you saying the gc error might be a side effect of another error?
Chris Toshok
@toshok
Jul 23 2015 18:44
no. i mean, i can’t imagine it is...
it’s just bumping libpypa a couple revisions and adding #include <cstddef> in two places
it wouldn’t have been the side effect of another error - it might be the side effect of the fix, but i’d be very much surprised if that were the case
Rudi Chen
@rudi-c
Jul 23 2015 18:48
Oh I misunderstood your sentence
Rudi Chen
@rudi-c
Jul 23 2015 18:54
Hmm I duplicated your branch adding the release assert and the gc bug doesn't happen. Weird.
Just curious, have we tried using all of CPython's huge setup.py at once in from_cpython before?
Kevin Modzelewski
@kmod
Jul 23 2015 18:57
I think I tried it once
and quickly gave up (didn't look into it that much)
Chris Toshok
@toshok
Jul 23 2015 19:01
@kmod mind checking out #751 so @tjhance won’t be so broken by my 4.9 upgrade :/
going to check if bumping gmp also fixes the issue (and is possible on 14.04), but this will fix anyone who ends up in a similar state (gcc 4.9 and gmp 5.1.3)
Sun
@Daetalus
Jul 23 2015 19:04

Hi, in test_format, there has a custom __oct__

        class Foobar(long):
            def __oct__(self):
                # Returning a non-string should not blow up.
                return self + 1

        test_exc('%o', Foobar(), TypeError,
                 "expected string or Unicode object, long found")

This custom __oct__ return a long object in Python level. So it didn't check the return type(it should return str or unicode). How to check the custom function return type? Modify the longOct in runtime/long.cpp? Please give a hint.

Kevin Modzelewski
@kmod
Jul 23 2015 19:08
@toshok ok cool, merged :)
if we've gotten in the fixes for the old gmp maybe we don't have to worry about getting the new one
@daetalus interesting, looks like cpython calls nb_oct, but we don't have that yet, so we only support longs there for now and we just call longOct
the easiest fix for now would be to change result = longOct((BooxedLong*)val)
to something like
if (val->cls == long_cls)
  result = longOct((BoxedLong*)val);
else
  result = callattr(val, "__oct__")
oh actually
we might support nb_oct
Sun
@Daetalus
Jul 23 2015 19:13
This code should added to a new nb_oct?
Kevin Modzelewski
@kmod
Jul 23 2015 19:13
oh sorry, I was talking about in str.cpp::_PyString_FormatLong where the assertion gets fired
we have some helpful comments there saying how we changed the cpython version
since I assume we added that function at a time we didn't have nb_oct/nb_hex, so we only handled it for the long-non-subclass case
you could try undoing the "Pyston changes" to that function and seeing if it works
Sun
@Daetalus
Jul 23 2015 19:15
I noticed that.
It could work. Already did that.
But I don't know how to check cutom function return type.
Kevin Modzelewski
@kmod
Jul 23 2015 19:18
sorry, not quite sure I know what you mean
I think the error message in this case (oct returned non-string) comes from the line
buf = PyString_AsString(result)
which checks the type of result
so we might not have to add any extra type checking
Sun
@Daetalus
Jul 23 2015 19:20
It could also check the return type of a cutom function in Python level?
Kevin Modzelewski
@kmod
Jul 23 2015 19:27
sorry, not quite sure I understand what you're proposing
are you saying that the custom __oct__ could know that it's supposed to return a str and do the type checking itself?
Sun
@Daetalus
Jul 23 2015 19:28
Sorry...
custom __oct__ return a long in Python.
Did the longOct in cpp would know what type does the __oct__ return?
Kevin Modzelewski
@kmod
Jul 23 2015 19:32
well, I think longOct always returns a string
longOct is just our implementation of long.__oct__, and doesn't check to see if a subclass overrode __oct__
Sun
@Daetalus
Jul 23 2015 19:32

Another questions:

# CPython
>>> max(1,2,3, key=lambda x:x, 2)
  File "<stdin>", line 1
SyntaxError: non-keyword arg after keyword arg
# Pyston
>> max(1,2,3, key=lambda x:x, 2)
Traceback (most recent call last):
  File "pystontmp_Bx2uH7/in.py", line 1, in :
    max(1,2,3, key=lambda x:x, 2)
SyntaxError: Expected keyword argument

libpypa should handle it, min and max doesn't check it, is that correct?

Travis Hance
@tjhance
Jul 23 2015 19:33
yeah it’s a syntax error so libpypa should handle it
looks like the error message is slightly wrong but I guess that’s a libpypa issue
Sun
@Daetalus
Jul 23 2015 19:36
Ok. I need to go to bed. It almost 4am in my local time. I will address longOct isssue and improve min_max tomorrow.
Kevin Modzelewski
@kmod
Jul 23 2015 19:36
this whole time zone thing is funny/tricky :)
Sun
@Daetalus
Jul 23 2015 19:38
Thanks @kmod and @tjhance . For the patiently answers!
Marius Wachtler
@undingen
Jul 23 2015 20:51
I will look into threading_local.py it's now 3times in a row failing for #736
Marius Wachtler
@undingen
Jul 23 2015 21:06
thread sanitizer gives a ton of output for this test.. hope there's something interesting in it
Chris Toshok
@toshok
Jul 23 2015 21:07
ohhh, tsan...
Marius Wachtler
@undingen
Jul 23 2015 21:28
ok I enabled pthread error checking and it returns: EDEADLK 35 / Resource deadlock would occur /
:-D
Chris Toshok
@toshok
Jul 23 2015 21:29
nice
Marius Wachtler
@undingen
Jul 23 2015 21:29
maybe wee need recursive mutex
and tsan says we lock a mutex in one thread and release it in another...
Marius Wachtler
@undingen
Jul 23 2015 21:37
yes we lock and unlock in different threads..
Chris Toshok
@toshok
Jul 23 2015 21:43
@dagar have you done much with cpack before?
Daniel Agar
@dagar
Jul 23 2015 21:44
just the basics?
is it package time?
Chris Toshok
@toshok
Jul 23 2015 21:45
i was hoping it would be a 15 minute hack and then “hey guys, we have a .deb!"
boy was I wrong :)
looks like the CPACK_* settings get clobbered from both llvm and lz4’s CMakeLists.txt
Daniel Agar
@dagar
Jul 23 2015 21:45
really?
oh
ha
right
also need to figure out where to put from_cpython?
Chris Toshok
@toshok
Jul 23 2015 21:46
eyah
I consider that an easy problem since it’s (mostly) fixable in code of some sort :)
deb’s aren’t hard to build, though. might be easier for me to flesh out a debian/ directory and build it manually
Daniel Agar
@dagar
Jul 23 2015 21:56
did you get make install to work?
Chris Toshok
@toshok
Jul 23 2015 21:56
nope
just barely got things to the point where CPack broke everything :)
Daniel Agar
@dagar
Jul 23 2015 21:59
I think step 1 is getting install working
Chris Toshok
@toshok
Jul 23 2015 22:00
nod
Chris Toshok
@toshok
Jul 23 2015 22:28
install unfortunately has some wrinkles - rpaths, for instance
Chris Toshok
@toshok
Jul 23 2015 23:07
is there a cmake variable corresponding to the CMAKE_BINARY_DIR version of the directory where the CMakeLists.txt file is?
i.e. instead of pyston/lib_pyston, I want pyston/build/Release/lib_pyston
ended up just doing ${CMAKE_BINARY_DIR}/lib_pyston, but was hoping for something cleaner
result of DESTDIR=/mnt/toshok/pyston-destdir ninja install gist.github.com/toshok/cd0efbf07fdc68410f21
Kevin Modzelewski
@kmod
Jul 23 2015 23:11
CMAKE_CURRENT_BINARY_DIR?
Chris Toshok
@toshok
Jul 23 2015 23:12
nice, didn’t know about the CURRENT variables. that works
Kevin Modzelewski
@kmod
Jul 23 2015 23:13
hey rudi I ran into the assert that chris mentioned
Rudi Chen
@rudi-c
Jul 23 2015 23:13
Travis or locally?
Kevin Modzelewski
@kmod
Jul 23 2015 23:13
+ pip install decorator==3.4.2
You are using pip version 6.0.8, however version 7.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting decorator==3.4.2
  Using cached decorator-3.4.2.tar.gz
Installing collected packages: decorator
  Running setup.py install for decorator
    warning: build_py: byte-compiling is disabled, skipping.
    warning: install_lib: byte-compiling is disabled, skipping.
Successfully installed decorator-3.4.2
+ pip install oauth2client==1.4.11 httplib2==0.9.1 pyasn1==0.1.7 pyasn1-modules==0.0.5 rsa==3.1.4
You are using pip version 6.0.8, however version 7.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting oauth2client==1.4.11
  Using cached oauth2client-1.4.11.zip
Collecting httplib2==0.9.1
  Using cached httplib2-0.9.1.zip
Collecting pyasn1==0.1.7
  Using cached pyasn1-0.1.7.tar.gz
Collecting pyasn1-modules==0.0.5
  Using cached pyasn1-modules-0.0.5.tar.gz
Collecting rsa==3.1.4
  Using cached rsa-3.1.4.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six>=1.6.1 in ./test_env/site-packages (from oauth2client==1.4.11)
Installing collected packages: rsa, pyasn1-modules, pyasn1, httplib2, oauth2client
  Running setup.py install for rsa
    warning: build_py: byte-compiling is disabled, skipping.
    warning: install_lib: byte-compiling is disabled, skipping.
    Installing pyrsa-sign script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-priv2pub script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-encrypt-bigfile script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-verify script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-keygen script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-decrypt-bigfile script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-encrypt script to /tmp/tmp0MVpWD/test_env/bin
    Installing pyrsa-decrypt script to /tmp/tmp0MVpWD/test_env/bin
pyston_dbg: ../../src/gc/heap.cpp:131: bool pyston::gc::hasOrderedFinalizer(pyston::BoxedClass *): Assertion `!cls->tp_del' failed.
Aborted (core dumped)
Traceback (most recent call last):
  File "/mnt/kmod/pyston/test/integration/virtualenv_test.py", line 42, in <module>:
    subprocess.check_call(["sh", "-c", sh_script], stdout=sys.stderr)
  File "/mnt/kmod/pyston/build/Debug/from_cpython/Lib/subprocess.py", line 540, in check_call:
    raise CalledProcessError(retcode, cmd)
locally
Rudi Chen
@rudi-c
Jul 23 2015 23:17
I haven't been able to repro myself. Can we add this one-liner to help debugging?
dropbox/pyston#754
Chris Toshok
@toshok
Jul 23 2015 23:18
sweet, after apt-get installing jemalloc packages, pyston runs from the destdir:
$ PYTHONPATH=/mnt/toshok/pyston-destdir/usr/local/lib/pyston0.3:/mnt/toshok/pyston-destdir/usr/local/lib/pyston0.3/lib_pyston /mnt/toshok/pyston-destdir/usr/local/bin/pyston 
Pyston v0.3 (rev 17207489b82901d9c61fa4bd42ddd9c52cb19c5c), targeting Python 2.7.6
>>
Kevin Modzelewski
@kmod
Jul 23 2015 23:19
hmm I ran the virtualenv test again and didn't reproduce the crash
Rudi Chen
@rudi-c
Jul 23 2015 23:20
I'm guessing it has to do with the caching. Though I did try to clear my cache.
Chris Toshok
@toshok
Jul 23 2015 23:59
weird.
$ /mnt/toshok/pyston-install/bin/pyston
ImportError: No module named _sysconfigdata
$ PYTHONPATH=/mnt/toshok/pyston-install/lib/pyston0.3/lib_pyston /mnt/toshok/pyston-install/bin/pyston gives me the repl, though, and if I look at sys.path: