Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • Jul 14 22:41
    Travis dropbox/pyston (master) errored (4687)
  • Dec 27 2019 23:49
  • Jan 17 2019 05:30
    xxllp commented #1426
  • Dec 13 2018 23:14
    deronnax closed #1421
  • Jul 14 2018 08:28
    corporatepiyush commented #1426
  • May 23 2018 21:37
    toshok closed #528
  • May 23 2018 21:37
    toshok closed #558
  • May 23 2018 21:34
    toshok closed #609
  • May 23 2018 21:34
    toshok closed #756
  • Feb 21 2018 05:25
    osman-masood commented #1426
  • Feb 03 2018 02:34
    skywind3000 commented #1426
  • Feb 01 2018 22:06
    danluu commented #1426
  • Jan 27 2018 19:57
    lesshaste commented #1426
  • Jan 12 2018 13:11
    skywind3000 opened #1426
  • Nov 12 2017 19:59
    xoviat commented #1421
  • Aug 25 2017 15:58
    Travis dropbox/pyston (py3) passed (4685)
  • Aug 25 2017 15:31
    kmod commented #1425
  • Aug 25 2017 15:31
    kmod closed #1425
  • Aug 24 2017 15:02
    kmod commented #1422
  • Aug 01 2017 17:06
    denji closed #378
Stefan O'Neil
I believe in CPython, Python-ast.h and Python-ast.c are both generated by Parser/asdl_c.py whenever Parser/Python.asdl is changed. Do you happen to know if Pyston works this way as well?
Stefan O'Neil
Also, you linked me the src/codegen directory, but I am not seeing a grammar specification file in here. Does that mean Pyston not use a grammar specification file similar to CPython's Grammar/Grammar, but rather generates graminit.c and graminit.h some other way?
Again, I really appreciate all the help
Will reply you later...
Stefan O'Neil
Sure thing, thanks again
Kevin Modzelewski
I believe we copied Python-ast.h and Python-ast.c in from the CPython repository
and probably graminit.h/c as well
We never experimented with changing the grammar, but maybe one way to do that is to make the change in a CPython checkout, regenerate the various files, and re-copy them into Pyston
you probably need to regenerate src/codegen/cpython_ast.cpp (it includes the script to do so at the top of the file)
as I said, we haven't tried this, so I don't expect it to be super straightforward. let us know if you run into issues!
Stefan O'Neil
Oh I see, that’s good to know, thanks.
Stefan O'Neil
Yup, just confirmed, the graminit files are indeed identical with CPython. My apologies for the confusion.
Stefan O'Neil
Taking your recommendation for regenerating the graminit files, plus modifying a number of other files, I have successfully extended Pyston with a keyword, except for one problem
if I try to use the keyword from within a function, Pyston crashes.
in every other context, it seems to work just fine
the behavior of the keyword itself shouldn't be breaking anything, since it is just a trivial keyword for now
therefore i'm thinking maybe i need to modify Pyston's symbol table
Stefan O'Neil
in CPython the symbol table is mostly managed by Python/symtable.c, which isn't in Pyston
would src/codegen/irgen.cpp be the right place to look for this?
Kevin Modzelewski
We have three different ways that we execute Python code; irgen.cpp is the third, LLVM compilation
I'd pass -I to the pyston process, which forces it to stay in the AST interpreter
and get that working first
-I will force it to stay in codegen/ast_interpreter.cpp and make things much simpler
if that doesn't help, it might be some of the static analysis we do; if you post the crash then I could make a more informed guess
Stefan O'Neil
Sure thing:
$ ./pyston_dbg -I
Pyston v0.7.0 (rev f7e8fcf7d57c624da5ac8a8be7d0d55c260ff32e), targeting Python 2.7.8
>> escal x

reached an escal statement

>> def foo():
...  escal y
...  return 4
../../src/core/ast.h:1151: virtual bool pyston::ASTVisitor::visit_escal(pyston::AST_Escal *): Assertion `0' failed: 
Someone called abort!
Aborted (core dumped)
I also have gdb output, in case it is more informative:
(gdb) run happytest.py
Starting program: /home/escal3/debug_pyston/pyston/pyston_dbg happytest.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

reached an escal statement

../../src/core/ast.h:1151: virtual bool pyston::ASTVisitor::visit_escal(pyston::AST_Escal *): Assertion `0' failed: 
Someone called abort!

Program received signal SIGABRT, Aborted.
0x00007ffff5185c37 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56    ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
#0  0x00007ffff5185c37 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff5189028 in __GI_abort () at abort.c:89
#2  0x00000000007d0245 in abort () at ../../src/codegen/unwinding.cpp:1158
#3  0x00000000006167b7 in pyston::ASTVisitor::visit_escal (this=0x7fffffffaa88, 
    node=0x21f22d8) at ../../src/core/ast.h:1151
#4  0x00000000007dd1d1 in pyston::AST_Escal::accept (this=0x21f22d8, 
    v=0x7fffffffaa88) at ../../src/core/ast.cpp:557
#5  0x00000000014afdbc in pyston::NameCollectorVisitor::visit_functiondef (
    this=0x7fffffffaa88, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:683
#6  0x00000000014e4ca5 in pyston::AST_FunctionDef::accept (this=0x21f2190, 
    v=0x7fffffffaa88) at ../../src/core/ast.cpp:521
#7  0x00000000014afcdb in pyston::NameCollectorVisitor::collect (node=0x21f2190, 
    map=0x7fffffffab18, scoping=0x7fffffffc3b0)
    at ../../src/analysis/scoping_analysis.cpp:795
#8  0x0000000000612f9e in pyston::ScopingAnalysis::analyzeSubtree (
    this=0x7fffffffc3b0, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:977
#9  0x00000000014b0e85 in pyston::ScopingAnalysis::getScopeInfoForNode (
    this=0x7fffffffc3b0, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:986
#10 0x00000000014f3142 in pyston::ModuleCFGProcessor::runRecursively (
    this=0x7fffffffc3b0, body=..., name=0x7ffff49af2e0, lineno=3, args=0x21f21f0, 
    orig_node=0x21f2190) at ../../src/core/cfg.cpp:3538
#11 0x00000000014f4271 in pyston::CFGVisitor::visit_functiondef (
    this=0x7fffffffbba8, node=0x21f2190) at ../../src/core/cfg.cpp:1966
#12 0x00000000014e4ca5 in pyston::AST_FunctionDef::accept (this=0x21f2190, 
    v=0x7fffffffbba8) at ../../src/core/ast.cpp:521
#13 0x00000000007f217f in pyston::computeCFG (body=..., 
    ast_type=pyston::AST_TYPE::Module, lineno=1, args=0x0, filename=0x7ffff49adf48, 
    source=0x251de40, param_names=..., scoping=0x23ed9c0, cfgizer=0x7fffffffc3b0)
    at ../../src/core/cfg.cpp:3380
#14 0x00000000014f36ec in pyston::ModuleCFGProcessor::runRecursively (
    this=0x7fffffffc3b0, body=..., name=0x7ffff7e8f2b0, lineno=1, args=0x0, 
    orig_node=0x21f20a0) at ../../src/core/cfg.cpp:3581
#15 0x00000000014f3d3f in pyston::computeAllCFGs (ast=0x21f20a0, 
    globals_from_module=true, future_flags=16, fn=0x7ffff49adf48, bm=0x23686e0)
    at ../../src/core/cfg.cpp:3597
#16 0x0000000000752df2 in pyston::compileAndRunModule (m=0x21f20a0, bm=0x23686e0)
    at ../../src/codegen/irgen/hooks.cpp:234
#17 0x00000000005fa84b in pyston::main (argc=2, argv=0x7fffffffdfc8)
    at ../../src/jit.cpp:536
#18 0x00000000005f95e2 in main (argc=2, argv=0x7fffffffdfc8)
    at ../../src/jit.cpp:591
Kevin Modzelewski
Is that your new keyword?
Sounds like it just doesn't know what to do with it
Stefan O'Neil
my apologies for the late reply, i've been away
you were right about it being the scoping analysis
there were some places in src/analysis/scoping_analysis.cppwhere I didn't realize I needed to make additions
hence why the crash only showed up in function definitions
Stefan O'Neil
oh and yes, "escal" is just the name I am using for the keyword right now
Kevin Modzelewski
Ah, yeah there are some differences between how we handle functions and modules, which is the source of some annoying quirks like this
Hi, @stefanoneil you are working on a private repo?
Ali Ahsan
what are the instructions of running pyston after entering the docker?
Ali Ahsan
wow this project is effectively dead
Kevin Modzelewski
yes unfortunately
When in the docker container, python will end up running Pyston
And you can pip install things as well
Ali Ahsan
@kmod can pyston be used to develop some sort of a control flow graph, given a python program?
Kevin Modzelewski
Only inside of a single function
salvator lopez
what about guido van rossum ? .. is he working on dropbox when u switched to Golang and Dont wanna Develop Pyston anyway ?
hi, is Pyston still under development?
our lab needs a LLVM based Python compiler, so far I only found Pyston and Numba
Kevin Modzelewski
Nope, not under development
Python isn't really compile-able, so I think what you're asking for isn't a fit for Python
Probably the closest you can get is Cython->clang