by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 27 2019 23:49
  • Jan 17 2019 05:30
    xxllp commented #1426
  • Dec 13 2018 23:14
    deronnax closed #1421
  • Jul 14 2018 08:28
    corporatepiyush commented #1426
  • May 23 2018 21:37
    toshok closed #528
  • May 23 2018 21:37
    toshok closed #558
  • May 23 2018 21:34
    toshok closed #609
  • May 23 2018 21:34
    toshok closed #756
  • Feb 21 2018 05:25
    osman-masood commented #1426
  • Feb 03 2018 02:34
    skywind3000 commented #1426
  • Feb 01 2018 22:06
    danluu commented #1426
  • Jan 27 2018 19:57
    lesshaste commented #1426
  • Jan 12 2018 13:11
    skywind3000 opened #1426
  • Nov 12 2017 19:59
    xoviat commented #1421
  • Aug 25 2017 15:58
    Travis dropbox/pyston (py3) passed (4685)
  • Aug 25 2017 15:31
    kmod commented #1425
  • Aug 25 2017 15:31
    kmod closed #1425
  • Aug 24 2017 15:02
    kmod commented #1422
  • Aug 01 2017 17:06
    denji closed #378
  • Jul 28 2017 00:31
    Travis dropbox/pyston#1425 passed (4684)
Stefan O'Neil
@stefanoneil
Finally, how can I check the virtual memory address for a given variable/object? Another aspect of the project I am working on requires that I am able to map memory regions from a Pyston process into kernel memory, and then externally access or modify values internal to that Pyston process (probably using mmap() from within a kernel module). Thus, being able to find exactly where in memory an object is stored would be very useful. I find that id() often returns negative values - is this still a memory address, perhaps using twos complement, or is id() not a reliable way to check addresses in Pyston?
My apologies if any of these questions are obvious or answered elsewhere already - I did search fairly extensively before asking, but I could easily have overlooked something. I am pretty new both to the world of compilers and to exploring large code bases. All help is deeply appreciated.
Kevin Modzelewski
@kmod
Our command line flags mostly follow CPython's; you can find them here:
https://github.com/dropbox/pyston/blob/master/src/jit.cpp#L184
I think you can pass the -j flag and Pyston will spit out a .o file for each function it compiles, which you can then disassemble with objdump
There was an old reason that it xor'd with a constant, which should no longer be necessary
ARM code would be awesome :) but I'm sure is a lot of work
and since things aren't being actively developed it might be hard to get help
We have two places we emit code: one is through the LLVM jit, which should have ARM support. the other is our jit (src/asm_writing), which is probably much harder to convert
Kevin Modzelewski
@kmod
You'll probably want to liberally deactivate features in order to get started
Chris Toshok
@toshok
hey all, this thing still on? :)
got pyston running on OSX (baseline jit only at the moment) https://github.com/toshok/pyston/tree/osx-port
really, really rough at the moment.
Kevin Modzelewski
@kmod
oh whoa, cool!
Chris Toshok
@toshok
oh and absolutely nothing unwinding-related works :)
but that shouldn't be too hard to get working
Sun
@Daetalus
great!
Stefan O'Neil
@stefanoneil
Thanks for the helpful pointers @kmod !
Stefan O'Neil
@stefanoneil
I have another quick question for you or anyone who is decently familiar with the Pyston source code. Something I am working on in my current project is extending Pyston grammar with an additional keyword. In CPython, Python/graminit.c and Include/graminit.h are auto-generated from Grammar/Grammar when you run make.
In tracking down all the files I need to modify to implement an additional keyword in Pyston, I can’t seem to find the grammar file from which graminit.c and graminit.h are generated. The only grammar file I can find is from_cpython/Lib/lib2to3/Grammar.txt, but modifying this does not seem to trigger a regeneration of the graminit files.
Where should I be looking for the grammar file that Pyston generates these from?
Sun
@Daetalus
Here: https://github.com/dropbox/pyston/tree/master/src/codegen. But first you may need copy new generated CPython Python-ast.h file. And then extend the Pyston interpreter and jit.
Stefan O'Neil
@stefanoneil
Excellent, thanks a ton for that
I believe in CPython, Python-ast.h and Python-ast.c are both generated by Parser/asdl_c.py whenever Parser/Python.asdl is changed. Do you happen to know if Pyston works this way as well?
Stefan O'Neil
@stefanoneil
Also, you linked me the src/codegen directory, but I am not seeing a grammar specification file in here. Does that mean Pyston not use a grammar specification file similar to CPython's Grammar/Grammar, but rather generates graminit.c and graminit.h some other way?
Again, I really appreciate all the help
Sun
@Daetalus
Will reply you later...
Stefan O'Neil
@stefanoneil
Sure thing, thanks again
Kevin Modzelewski
@kmod
I believe we copied Python-ast.h and Python-ast.c in from the CPython repository
and probably graminit.h/c as well
We never experimented with changing the grammar, but maybe one way to do that is to make the change in a CPython checkout, regenerate the various files, and re-copy them into Pyston
you probably need to regenerate src/codegen/cpython_ast.cpp (it includes the script to do so at the top of the file)
as I said, we haven't tried this, so I don't expect it to be super straightforward. let us know if you run into issues!
Stefan O'Neil
@stefanoneil
Oh I see, that’s good to know, thanks.
Stefan O'Neil
@stefanoneil
Yup, just confirmed, the graminit files are indeed identical with CPython. My apologies for the confusion.
Stefan O'Neil
@stefanoneil
Taking your recommendation for regenerating the graminit files, plus modifying a number of other files, I have successfully extended Pyston with a keyword, except for one problem
if I try to use the keyword from within a function, Pyston crashes.
in every other context, it seems to work just fine
the behavior of the keyword itself shouldn't be breaking anything, since it is just a trivial keyword for now
therefore i'm thinking maybe i need to modify Pyston's symbol table
Stefan O'Neil
@stefanoneil
in CPython the symbol table is mostly managed by Python/symtable.c, which isn't in Pyston
would src/codegen/irgen.cpp be the right place to look for this?
Kevin Modzelewski
@kmod
We have three different ways that we execute Python code; irgen.cpp is the third, LLVM compilation
I'd pass -I to the pyston process, which forces it to stay in the AST interpreter
and get that working first
-I will force it to stay in codegen/ast_interpreter.cpp and make things much simpler
if that doesn't help, it might be some of the static analysis we do; if you post the crash then I could make a more informed guess
Stefan O'Neil
@stefanoneil
Sure thing:
$ ./pyston_dbg -I
Pyston v0.7.0 (rev f7e8fcf7d57c624da5ac8a8be7d0d55c260ff32e), targeting Python 2.7.8
>> 
>> 
>> escal x


reached an escal statement

>> 
>> 
>> def foo():
...  escal y
...  return 4
... 
../../src/core/ast.h:1151: virtual bool pyston::ASTVisitor::visit_escal(pyston::AST_Escal *): Assertion `0' failed: 
Someone called abort!
Aborted (core dumped)
I also have gdb output, in case it is more informative: