Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
  • Jul 14 2020 22:41
    Travis dropbox/pyston (master) errored (4687)
  • Dec 27 2019 23:49
  • Jan 17 2019 05:30
    xxllp commented #1426
  • Dec 13 2018 23:14
    deronnax closed #1421
  • Jul 14 2018 08:28
    corporatepiyush commented #1426
  • May 23 2018 21:37
    toshok closed #528
  • May 23 2018 21:37
    toshok closed #558
  • May 23 2018 21:34
    toshok closed #609
  • May 23 2018 21:34
    toshok closed #756
  • Feb 21 2018 05:25
    osman-masood commented #1426
  • Feb 03 2018 02:34
    skywind3000 commented #1426
  • Feb 01 2018 22:06
    danluu commented #1426
  • Jan 27 2018 19:57
    lesshaste commented #1426
  • Jan 12 2018 13:11
    skywind3000 opened #1426
  • Nov 12 2017 19:59
    xoviat commented #1421
  • Aug 25 2017 15:58
    Travis dropbox/pyston (py3) passed (4685)
  • Aug 25 2017 15:31
    kmod commented #1425
  • Aug 25 2017 15:31
    kmod closed #1425
  • Aug 24 2017 15:02
    kmod commented #1422
  • Aug 01 2017 17:06
    denji closed #378
Kevin Modzelewski
Our command line flags mostly follow CPython's; you can find them here:
I think you can pass the -j flag and Pyston will spit out a .o file for each function it compiles, which you can then disassemble with objdump
There was an old reason that it xor'd with a constant, which should no longer be necessary
ARM code would be awesome :) but I'm sure is a lot of work
and since things aren't being actively developed it might be hard to get help
We have two places we emit code: one is through the LLVM jit, which should have ARM support. the other is our jit (src/asm_writing), which is probably much harder to convert
Kevin Modzelewski
You'll probably want to liberally deactivate features in order to get started
Chris Toshok
hey all, this thing still on? :)
got pyston running on OSX (baseline jit only at the moment) https://github.com/toshok/pyston/tree/osx-port
really, really rough at the moment.
Kevin Modzelewski
oh whoa, cool!
Chris Toshok
oh and absolutely nothing unwinding-related works :)
but that shouldn't be too hard to get working
Stefan O'Neil
Thanks for the helpful pointers @kmod !
Stefan O'Neil
I have another quick question for you or anyone who is decently familiar with the Pyston source code. Something I am working on in my current project is extending Pyston grammar with an additional keyword. In CPython, Python/graminit.c and Include/graminit.h are auto-generated from Grammar/Grammar when you run make.
In tracking down all the files I need to modify to implement an additional keyword in Pyston, I can’t seem to find the grammar file from which graminit.c and graminit.h are generated. The only grammar file I can find is from_cpython/Lib/lib2to3/Grammar.txt, but modifying this does not seem to trigger a regeneration of the graminit files.
Where should I be looking for the grammar file that Pyston generates these from?
Here: https://github.com/dropbox/pyston/tree/master/src/codegen. But first you may need copy new generated CPython Python-ast.h file. And then extend the Pyston interpreter and jit.
Stefan O'Neil
Excellent, thanks a ton for that
I believe in CPython, Python-ast.h and Python-ast.c are both generated by Parser/asdl_c.py whenever Parser/Python.asdl is changed. Do you happen to know if Pyston works this way as well?
Stefan O'Neil
Also, you linked me the src/codegen directory, but I am not seeing a grammar specification file in here. Does that mean Pyston not use a grammar specification file similar to CPython's Grammar/Grammar, but rather generates graminit.c and graminit.h some other way?
Again, I really appreciate all the help
Will reply you later...
Stefan O'Neil
Sure thing, thanks again
Kevin Modzelewski
I believe we copied Python-ast.h and Python-ast.c in from the CPython repository
and probably graminit.h/c as well
We never experimented with changing the grammar, but maybe one way to do that is to make the change in a CPython checkout, regenerate the various files, and re-copy them into Pyston
you probably need to regenerate src/codegen/cpython_ast.cpp (it includes the script to do so at the top of the file)
as I said, we haven't tried this, so I don't expect it to be super straightforward. let us know if you run into issues!
Stefan O'Neil
Oh I see, that’s good to know, thanks.
Stefan O'Neil
Yup, just confirmed, the graminit files are indeed identical with CPython. My apologies for the confusion.
Stefan O'Neil
Taking your recommendation for regenerating the graminit files, plus modifying a number of other files, I have successfully extended Pyston with a keyword, except for one problem
if I try to use the keyword from within a function, Pyston crashes.
in every other context, it seems to work just fine
the behavior of the keyword itself shouldn't be breaking anything, since it is just a trivial keyword for now
therefore i'm thinking maybe i need to modify Pyston's symbol table
Stefan O'Neil
in CPython the symbol table is mostly managed by Python/symtable.c, which isn't in Pyston
would src/codegen/irgen.cpp be the right place to look for this?
Kevin Modzelewski
We have three different ways that we execute Python code; irgen.cpp is the third, LLVM compilation
I'd pass -I to the pyston process, which forces it to stay in the AST interpreter
and get that working first
-I will force it to stay in codegen/ast_interpreter.cpp and make things much simpler
if that doesn't help, it might be some of the static analysis we do; if you post the crash then I could make a more informed guess
Stefan O'Neil
Sure thing:
$ ./pyston_dbg -I
Pyston v0.7.0 (rev f7e8fcf7d57c624da5ac8a8be7d0d55c260ff32e), targeting Python 2.7.8
>> escal x

reached an escal statement

>> def foo():
...  escal y
...  return 4
../../src/core/ast.h:1151: virtual bool pyston::ASTVisitor::visit_escal(pyston::AST_Escal *): Assertion `0' failed: 
Someone called abort!
Aborted (core dumped)
I also have gdb output, in case it is more informative:
(gdb) run happytest.py
Starting program: /home/escal3/debug_pyston/pyston/pyston_dbg happytest.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

reached an escal statement

../../src/core/ast.h:1151: virtual bool pyston::ASTVisitor::visit_escal(pyston::AST_Escal *): Assertion `0' failed: 
Someone called abort!

Program received signal SIGABRT, Aborted.
0x00007ffff5185c37 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56    ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
#0  0x00007ffff5185c37 in __GI_raise (sig=sig@entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff5189028 in __GI_abort () at abort.c:89
#2  0x00000000007d0245 in abort () at ../../src/codegen/unwinding.cpp:1158
#3  0x00000000006167b7 in pyston::ASTVisitor::visit_escal (this=0x7fffffffaa88, 
    node=0x21f22d8) at ../../src/core/ast.h:1151
#4  0x00000000007dd1d1 in pyston::AST_Escal::accept (this=0x21f22d8, 
    v=0x7fffffffaa88) at ../../src/core/ast.cpp:557
#5  0x00000000014afdbc in pyston::NameCollectorVisitor::visit_functiondef (
    this=0x7fffffffaa88, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:683
#6  0x00000000014e4ca5 in pyston::AST_FunctionDef::accept (this=0x21f2190, 
    v=0x7fffffffaa88) at ../../src/core/ast.cpp:521
#7  0x00000000014afcdb in pyston::NameCollectorVisitor::collect (node=0x21f2190, 
    map=0x7fffffffab18, scoping=0x7fffffffc3b0)
    at ../../src/analysis/scoping_analysis.cpp:795
#8  0x0000000000612f9e in pyston::ScopingAnalysis::analyzeSubtree (
    this=0x7fffffffc3b0, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:977
#9  0x00000000014b0e85 in pyston::ScopingAnalysis::getScopeInfoForNode (
    this=0x7fffffffc3b0, node=0x21f2190)
    at ../../src/analysis/scoping_analysis.cpp:986
#10 0x00000000014f3142 in pyston::ModuleCFGProcessor::runRecursively (
    this=0x7fffffffc3b0, body=..., name=0x7ffff49af2e0, lineno=3, args=0x21f21f0, 
    orig_node=0x21f2190) at ../../src/core/cfg.cpp:3538
#11 0x00000000014f4271 in pyston::CFGVisitor::visit_functiondef (
    this=0x7fffffffbba8, node=0x21f2190) at ../../src/core/cfg.cpp:1966
#12 0x00000000014e4ca5 in pyston::AST_FunctionDef::accept (this=0x21f2190, 
    v=0x7fffffffbba8) at ../../src/core/ast.cpp:521
#13 0x00000000007f217f in pyston::computeCFG (body=..., 
    ast_type=pyston::AST_TYPE::Module, lineno=1, args=0x0, filename=0x7ffff49adf48, 
    source=0x251de40, param_names=..., scoping=0x23ed9c0, cfgizer=0x7fffffffc3b0)
    at ../../src/core/cfg.cpp:3380
#14 0x00000000014f36ec in pyston::ModuleCFGProcessor::runRecursively (
    this=0x7fffffffc3b0, body=..., name=0x7ffff7e8f2b0, lineno=1, args=0x0, 
    orig_node=0x21f20a0) at ../../src/core/cfg.cpp:3581
#15 0x00000000014f3d3f in pyston::computeAllCFGs (ast=0x21f20a0, 
    globals_from_module=true, future_flags=16, fn=0x7ffff49adf48, bm=0x23686e0)
    at ../../src/core/cfg.cpp:3597
#16 0x0000000000752df2 in pyston::compileAndRunModule (m=0x21f20a0, bm=0x23686e0)
    at ../../src/codegen/irgen/hooks.cpp:234
#17 0x00000000005fa84b in pyston::main (argc=2, argv=0x7fffffffdfc8)
    at ../../src/jit.cpp:536
#18 0x00000000005f95e2 in main (argc=2, argv=0x7fffffffdfc8)
    at ../../src/jit.cpp:591
Kevin Modzelewski
Is that your new keyword?