These are chat archives for dropbox/pyston

3rd
Aug 2015
Travis Hance
@tjhance
Aug 03 2015 00:31

can somebody explain this stacktrace?

#2  0x0000000000625c34 in llvm::DenseMapInfo<pyston::InternedString>::isEqual (lhs=..., rhs=...) at ../../src/core/stringpool.h:131
#3  0x00000000006b7fa1 in pyston::(anonymous namespace)::ASTInterpreter::executeInner (interpreter=..., start_block=0x2aaab64ba120, start_at=0x2aaab64df500) at ../../src/codegen/ast_interpreter.cpp:386

I can’t figure out how on earth isEqual is getting called from executeInner

@undingen ?
Kevin Modzelewski
@kmod
Aug 03 2015 01:49
are the stack frames 0 and 1 in the baseline jit?
Travis Hance
@tjhance
Aug 03 2015 02:05
#0 0x00002aaab7dddecb in ?? ()

#1 0x00000042707f9690 in ?? ()
I would guess so?
wait but how can isEqual possibly call into the baseline jit?
anyway it looks like lhs and rhs have different pool attributes, so the asserts in operator== would fail
although.. there’s no assert message
it’s pretty weird
Marius Wachtler
@undingen
Aug 03 2015 07:23
Sadly it looks like gdb can't show the correct stacktrace for the bjit generated frames :-(.
I will try with lldb maybe this one works better
Marius Wachtler
@undingen
Aug 03 2015 08:24
yes lldb shows the correct backtrace
But sadly the cmd is really different: http://lldb.llvm.org/lldb-gdb.html
Do you change the SP inside you new patchpoint code? Because this would destroy the unwind info...
Marius Wachtler
@undingen
Aug 03 2015 08:30
but still this 0x00002aaab7dddecb address looks strange... 0x00000042707f9690 should be a generator but the other address looks odd
Sun
@Daetalus
Aug 03 2015 18:23

Hi, could some give a hint about how to debug ast_interpreter? Or how to fix the problem in below?

It seems ast_interpreter could not handle complex statment correctly. Such as 1.5+1.5j, 1+0j, etc. It always store 0 to node->n_double. Then use 0 to call createPureImaginary. Which always get 0.0j.

I check the libpypa, it parse complex statment correctly.

Rudi Chen
@rudi-c
Aug 03 2015 18:28
Do you mean n_float?
Sun
@Daetalus
Aug 03 2015 18:29
Ah, yes, n_float.
I want to improve the complex. So maybe you could support NumPy easier.
Rudi Chen
@rudi-c
Aug 03 2015 18:31
I'm not sure if this is your question, but as far as the parser and interpreter is concerned, there is no such thing as a number with both a real and a complex part.
The parser will read the real part as an individual number, then an imaginary part as an individual (complex) number, then add the two together.
Sun
@Daetalus
Aug 03 2015 18:32
Yes, I checked it. Here is the libpypa output:
[Module]
    - body: [

        [Expr]
            - value: 
            [BinOp]
                - left: 
                [Num]
                    - n: 1.5
                - op: 
                [Add]
                - right: 
                [Num]
                    - n: 1.5j
Rudi Chen
@rudi-c
Aug 03 2015 18:32
Yeah, there's two separate Num(s) that are added together with a BinOp.
Sun
@Daetalus
Aug 03 2015 18:32
But ast_interpreter read the imaginary part as Zero.
Rudi Chen
@rudi-c
Aug 03 2015 18:33
Oh it does?
Wops I thought I only saw 1+0j and forgot about 1.5+1.5j.
Sun
@Daetalus
Aug 03 2015 18:34
And it does not call visit_binop. Instead, it call other visit_xx
Rudi Chen
@rudi-c
Aug 03 2015 18:36
When I run test/tests/complex.py on master I get the expected output though.
Sun
@Daetalus
Aug 03 2015 18:36
Here is the backtrace of 1.5+1.5j:
(gdb) bt
#0  createPureImaginary (i=0) at ../../src/runtime/complex.cpp:30
#1  0x000000000090709a in pyston::BoxedModule::getPureImaginaryConstant (this=0x1270156d08, d=0) at ../../src/runtime/types.cpp:528
#2  0x00000000006c18bb in pyston::(anonymous namespace)::ASTInterpreter::visit_num (this=0x7fffffffd438, node=0x7ffff38d01f0)
    at ../../src/codegen/ast_interpreter.cpp:1398
#3  0x00000000006bf2fe in pyston::(anonymous namespace)::ASTInterpreter::visit_expr (this=0x7fffffffd438, node=0x7ffff38d01f0)
    at ../../src/codegen/ast_interpreter.cpp:1265
#4  0x00000000006bf8a0 in pyston::(anonymous namespace)::ASTInterpreter::visit_call (this=0x7fffffffd438, node=0x7ffff38cd5c0)
    at ../../src/codegen/ast_interpreter.cpp:1328
#5  0x00000000006bf216 in pyston::(anonymous namespace)::ASTInterpreter::visit_expr (this=0x7fffffffd438, node=0x7ffff38cd5c0)
    at ../../src/codegen/ast_interpreter.cpp:1249
#6  0x00000000006bd9ac in pyston::(anonymous namespace)::ASTInterpreter::visit_assign (this=0x7fffffffd438, node=0x7ffff38cfd40)
    at ../../src/codegen/ast_interpreter.cpp:1200
#7  0x00000000006bb868 in pyston::(anonymous namespace)::ASTInterpreter::visit_stmt (this=0x7fffffffd438, node=0x7ffff38cfd40)
    at ../../src/codegen/ast_interpreter.cpp:905
#8  0x00000000006b45c0 in pyston::(anonymous namespace)::ASTInterpreter::executeInner (interpreter=..., start_block=0x7ffff38b52b0, 
    start_at=0x7ffff38cfb80) at ../../src/codegen/ast_interpreter.cpp:402
I believe we should not rely on test/tests/complex.py. Instead, improve the complex, then run test/cpython/test_complex.
Rudi Chen
@rudi-c
Aug 03 2015 18:39
I agree with you. What I mean is that on master, I don't seem to always get 0j.
Sun
@Daetalus
Aug 03 2015 18:40
Oh... let me see...
My branch is alreay updated. The submodule is equal as the master branch...
Rudi Chen
@rudi-c
Aug 03 2015 18:45
gdb --args ./pyston_dbg test/tests/complex.py
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./pyston_dbg...done.
(gdb) break complex.cpp:30
Breakpoint 1 at 0x877c34: file ../../src/runtime/complex.cpp, line 30.
(gdb) run
Starting program: /home/vagrant/pyston/pyston_dbg test/tests/complex.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, createPureImaginary (i=0.5) at ../../src/runtime/complex.cpp:30
30        return new BoxedComplex(0.0, i);
(gdb) bt 10
#0  createPureImaginary (i=0.5) at ../../src/runtime/complex.cpp:30
#1  0x0000000000909cea in pyston::BoxedModule::getPureImaginaryConstant (this=0x1270152d08, d=0.5) at ../../src/runtime/types.cpp:528
#2  0x00000000006c46e0 in pyston::(anonymous namespace)::ASTInterpreter::visit_num (this=0x7fffffffd6f8, node=0x7ffff47dd0d0) at ../../src/codegen/ast_interpreter.cpp:1393
#3  0x00000000006c213e in pyston::(anonymous namespace)::ASTInterpreter::visit_expr (this=0x7fffffffd6f8, node=0x7ffff47dd0d0) at ../../src/codegen/ast_interpreter.cpp:1265
#4  0x00000000006c2441 in pyston::(anonymous namespace)::ASTInterpreter::visit_binop (this=0x7fffffffd6f8, node=0x7ffff47dd940) at ../../src/codegen/ast_interpreter.cpp:541
#5  0x00000000006c2039 in pyston::(anonymous namespace)::ASTInterpreter::visit_expr (this=0x7fffffffd6f8, node=0x7ffff47dd940) at ../../src/codegen/ast_interpreter.cpp:1247
#6  0x00000000006c081c in pyston::(anonymous namespace)::ASTInterpreter::visit_assign (this=0x7fffffffd6f8, node=0x7ffff47daf00) at ../../src/codegen/ast_interpreter.cpp:1200
#7  0x00000000006be6e0 in pyston::(anonymous namespace)::ASTInterpreter::visit_stmt (this=0x7fffffffd6f8, node=0x7ffff47daf00) at ../../src/codegen/ast_interpreter.cpp:905
#8  0x00000000006b74a0 in pyston::(anonymous namespace)::ASTInterpreter::executeInner (interpreter=..., start_block=0x7ffff40b2390, start_at=0x7ffff47daf00)
    at ../../src/codegen/ast_interpreter.cpp:402
#9  0x00000000006b7075 in executeInnerFromASM (interpreter=..., start_block=0x0, start_at=0x0) at ../../src/codegen/ast_interpreter.cpp:1659
(More stack frames follow...)
(gdb) quit
Sun
@Daetalus
Aug 03 2015 18:47
Don't use test/tests/complex.py... It puts complex before add sign. Which cause it become complex add float.
Marius Wachtler
@undingen
Aug 03 2015 18:47
can you try running it with -n e.g. ./pyston_dbg -n <yourtest>
Rudi Chen
@rudi-c
Aug 03 2015 18:47
Oh I see
Marius Wachtler
@undingen
Aug 03 2015 18:48
this disables the interpreter and uses the llvm tier. Is they problem than really gone?
Rudi Chen
@rudi-c
Aug 03 2015 18:48
Ok yeah nvm it happens on master too
Sun
@Daetalus
Aug 03 2015 18:48
@undingen -n option also return 0.0j
Marius Wachtler
@undingen
Aug 03 2015 18:48
k
Sun
@Daetalus
Aug 03 2015 18:49
BTW, I saw k many times, does it mean know?
Rudi Chen
@rudi-c
Aug 03 2015 18:49
It's a shorthand for ok.
Sun
@Daetalus
Aug 03 2015 18:49
k, thx
Rudi Chen
@rudi-c
Aug 03 2015 18:49
:+1:
Marius Wachtler
@undingen
Aug 03 2015 18:50
saves 50% :-D
Marius Wachtler
@undingen
Aug 03 2015 18:55
looks liek the problem is inside pypa::string_to_double(c.imag, ptr->n_float);
the c.imag contains "+1.5j" but it will get converted to 0.0
Sun
@Daetalus
Aug 03 2015 18:56
Ah... I didn't check pypa carefully, just check the ast... Thx!
Marius Wachtler
@undingen
Aug 03 2015 18:57
in pypy-parser.cpp is the code which converts between the pypa ast and ours
Sun
@Daetalus
Aug 03 2015 18:58
I see. Write it to my notebook..
Marius Wachtler
@undingen
Aug 03 2015 19:00
Ẁith sscanf(c.imag.c_str(), "%lf", &ptr->n_float); I'm getting 1.5j. but this is still wrong
oh and it's hard to debug because we cache the ast nodes to disk. so you have to touch the .py file or remove the pyc to force a pypa invokation
Kevin Modzelewski
@kmod
Aug 03 2015 19:03
sorry guys, forgot to send this out early like we talked about : https://notes.dropbox.com/Pyston-status-notes-7272015-XfhqNchrPg1EUql7cZgNO
Sun
@Daetalus
Aug 03 2015 19:06
@kmow Would you mind to let me access it, please?
Kevin Modzelewski
@kmod
Aug 03 2015 19:06
just did :)
Sun
@Daetalus
Aug 03 2015 19:06
Thanks1
Kevin Modzelewski
@kmod
Aug 03 2015 19:22
@daetalus want to add a section to the doc? just gave you write access :)
Sun
@Daetalus
Aug 03 2015 19:25
Am I qualified? Thanks!
Marius Wachtler
@undingen
Aug 03 2015 19:28

@Daetalus I think something like this should fix the parsing issue:

ResultPtr read(pypa::AstComplex& c) {
        AST_Num* imag = new AST_Num();
        location(imag, c);
        imag->num_type = AST_Num::COMPLEX;
        sscanf(c.imag.c_str(), "%lf", &imag->n_float);
        if (!c.real)
            return imag;

        AST_BinOp* binop = new AST_BinOp();
        location(binop, c);
        binop->op_type = AST_TYPE::Add;
        binop->right = readItem(c.real, interned_strings);
        binop->left = imag;
        return binop;
    }

does this give you the correct result? (you will have to change the python source file / touch it)

Sun
@Daetalus
Aug 03 2015 19:34
@undingen Thank you very much! I check it now.
Marius Wachtler
@undingen
Aug 03 2015 19:34
I just updated it, before the edit it crashed if one didn't specify a real number :-P
Sun
@Daetalus
Aug 03 2015 19:35
Thank you very much!
Marius Wachtler
@undingen
Aug 03 2015 19:37
thanks for working on the compatibility :-)
Sun
@Daetalus
Aug 03 2015 19:37
You are welcome.
I addressing #777, already done. Tested and passed in console, and I want to add a test file. When I run make check_future_builtins. It could not import the new created future_builtins module. What else should I need to add?
Rudi Chen
@rudi-c
Aug 03 2015 20:15
What problem are you running into?
Sun
@Daetalus
Aug 03 2015 20:18
python ./tools/tester.py -R pyston_dbg -j1 -a=-S -k /home/sun/pyston2/test/tests future_builtins -K
Last line of stderr: AttributeError: 'module' object has no attribute 'map'
/home/sun/pyston2/test/tests/future_builtins.py:
Traceback (most recent call last):
  File "./tools/tester.py", line 444, in worker_thread
    results[job[0]] = run_test(*job)
  File "./tools/tester.py", line 164, in run_test
    return determine_test_result(fn, opts, code, out, stderr, elapsed)
  File "./tools/tester.py", line 332, in determine_test_result
    raise Exception(msg)
Exception: Exited with code 1; add '# should_error' if this is expected
The test file like:
from future_builtins import  map, zip, filter
Rudi Chen
@rudi-c
Aug 03 2015 20:21
This looks familiar but I don't remember what the problem was. Have you tried a clean build?
Sun
@Daetalus
Aug 03 2015 20:25
Nope, I try it now
Sun
@Daetalus
Aug 03 2015 20:37
Not work...
Rudi Chen
@rudi-c
Aug 03 2015 20:39
Did you add things to from_cpython/setup.py?
Sun
@Daetalus
Aug 03 2015 20:41
yes, otherwise the import would not work for command line.
Rudi Chen
@rudi-c
Aug 03 2015 20:43
Does the function containing this line get run? m = Py_InitModule3("future_builtins", module_functions, module_doc);
Can you find all the files that you would expect in build/Debug/?
Sun
@Daetalus
Aug 03 2015 20:45
Could find it in build/Debug, maybe that is the problem
Thanks!
Rudi Chen
@rudi-c
Aug 03 2015 20:48
Btw if you can think of specific docs that would helpful, could you let me know? I've done some work on that recently but it's far from complete.
Maybe add it to the status docs.
Sun
@Daetalus
Aug 03 2015 20:49
You mean what I am mentioned in Notes?
Rudi Chen
@rudi-c
Aug 03 2015 20:49
Yeah
Sun
@Daetalus
Aug 03 2015 20:54
Ok, I will add it to the docs. Thanks!
Sun
@Daetalus
Aug 03 2015 21:07
from future_builtin import hex still not work. I have to sleep. If someone know how to solve it, please leave message. Thanks!
Marius Wachtler
@undingen
Aug 03 2015 21:42
yes, this sounds better, I looked last week a little bit in generating the normal JIT interface debug info but this sounded annoying because one has to generate real ELF objects...
wound be nice if the would provide a interface for just registering the function name...
Kevin Modzelewski
@kmod
Aug 03 2015 21:44
I'm still surprised that gdb can't use rbp-based unwinding
Marius Wachtler
@undingen
Aug 03 2015 21:45
surprised me too
I will try out the interface you linked, looks like this shouldn't be to much work (for getting just the function name)
Rudi Chen
@rudi-c
Aug 03 2015 23:16
This message was deleted