by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 27 2019 23:49
  • Jan 17 2019 05:30
    xxllp commented #1426
  • Dec 13 2018 23:14
    deronnax closed #1421
  • Jul 14 2018 08:28
    corporatepiyush commented #1426
  • May 23 2018 21:37
    toshok closed #528
  • May 23 2018 21:37
    toshok closed #558
  • May 23 2018 21:34
    toshok closed #609
  • May 23 2018 21:34
    toshok closed #756
  • Feb 21 2018 05:25
    osman-masood commented #1426
  • Feb 03 2018 02:34
    skywind3000 commented #1426
  • Feb 01 2018 22:06
    danluu commented #1426
  • Jan 27 2018 19:57
    lesshaste commented #1426
  • Jan 12 2018 13:11
    skywind3000 opened #1426
  • Nov 12 2017 19:59
    xoviat commented #1421
  • Aug 25 2017 15:58
    Travis dropbox/pyston (py3) passed (4685)
  • Aug 25 2017 15:31
    kmod commented #1425
  • Aug 25 2017 15:31
    kmod closed #1425
  • Aug 24 2017 15:02
    kmod commented #1422
  • Aug 01 2017 17:06
    denji closed #378
  • Jul 28 2017 00:31
    Travis dropbox/pyston#1425 passed (4684)
Ethan Smith
@ethanhs
Ah, yes of course that makes a lot of sense. I didn't consider that. I suppose you could be clever and do things like cache frames, but that would still be slower. Thanks!
mhsjlw
@mhsjlw
@kmod my question about compiling to a binary was just because I wanted to run it on my raspberry pi and not need an entire jit
and as far as libunwind goes, i think it would be really good to figure out a way to fix it, because as far as i can see, if we don't need libunwind i'm pretty sure i'd be able to get this to work on a mac
this is kind of stupid, but if you know a way to compile to a binary we could go beyond raspberry pi and maybe even run python on ios?
anyway, that's aside the point
i know languages like crystal and rust use the llvm and they are able to compile to native binaries, but from what i can see from pyston it's not exactly simple
Ethan Smith
@ethanhs
@mhsjlw the issue is that one must write to either JIT or statically compile. A JIT works fundamentally differently.
mhsjlw
@mhsjlw
of course, yeah
Stefan O'Neil
@stefanoneil
Hi everyone! I have some questions about working with Pyston source code.
Is there any documentation about the command line arguments that Pyston accepts? I cannot find such documentation anywhere, and neither “-h” nor “--help" seem to be supported. However it would help me tremendously to know all the different options I can use with Pyston.
How can I inspect the assembly code that Pyston generates? Part of the project that I am working on requires building support into Pyston for generating ARM code, so being able to check the machine code that Pyston is generating will be pretty essential.
Finally, how can I check the virtual memory address for a given variable/object? Another aspect of the project I am working on requires that I am able to map memory regions from a Pyston process into kernel memory, and then externally access or modify values internal to that Pyston process (probably using mmap() from within a kernel module). Thus, being able to find exactly where in memory an object is stored would be very useful. I find that id() often returns negative values - is this still a memory address, perhaps using twos complement, or is id() not a reliable way to check addresses in Pyston?
My apologies if any of these questions are obvious or answered elsewhere already - I did search fairly extensively before asking, but I could easily have overlooked something. I am pretty new both to the world of compilers and to exploring large code bases. All help is deeply appreciated.
Kevin Modzelewski
@kmod
Our command line flags mostly follow CPython's; you can find them here:
https://github.com/dropbox/pyston/blob/master/src/jit.cpp#L184
I think you can pass the -j flag and Pyston will spit out a .o file for each function it compiles, which you can then disassemble with objdump
There was an old reason that it xor'd with a constant, which should no longer be necessary
ARM code would be awesome :) but I'm sure is a lot of work
and since things aren't being actively developed it might be hard to get help
We have two places we emit code: one is through the LLVM jit, which should have ARM support. the other is our jit (src/asm_writing), which is probably much harder to convert
Kevin Modzelewski
@kmod
You'll probably want to liberally deactivate features in order to get started
Chris Toshok
@toshok
hey all, this thing still on? :)
got pyston running on OSX (baseline jit only at the moment) https://github.com/toshok/pyston/tree/osx-port
really, really rough at the moment.
Kevin Modzelewski
@kmod
oh whoa, cool!
Chris Toshok
@toshok
oh and absolutely nothing unwinding-related works :)
but that shouldn't be too hard to get working
Sun
@Daetalus
great!
Stefan O'Neil
@stefanoneil
Thanks for the helpful pointers @kmod !
Stefan O'Neil
@stefanoneil
I have another quick question for you or anyone who is decently familiar with the Pyston source code. Something I am working on in my current project is extending Pyston grammar with an additional keyword. In CPython, Python/graminit.c and Include/graminit.h are auto-generated from Grammar/Grammar when you run make.
In tracking down all the files I need to modify to implement an additional keyword in Pyston, I can’t seem to find the grammar file from which graminit.c and graminit.h are generated. The only grammar file I can find is from_cpython/Lib/lib2to3/Grammar.txt, but modifying this does not seem to trigger a regeneration of the graminit files.
Where should I be looking for the grammar file that Pyston generates these from?
Sun
@Daetalus
Here: https://github.com/dropbox/pyston/tree/master/src/codegen. But first you may need copy new generated CPython Python-ast.h file. And then extend the Pyston interpreter and jit.
Stefan O'Neil
@stefanoneil
Excellent, thanks a ton for that
I believe in CPython, Python-ast.h and Python-ast.c are both generated by Parser/asdl_c.py whenever Parser/Python.asdl is changed. Do you happen to know if Pyston works this way as well?
Stefan O'Neil
@stefanoneil
Also, you linked me the src/codegen directory, but I am not seeing a grammar specification file in here. Does that mean Pyston not use a grammar specification file similar to CPython's Grammar/Grammar, but rather generates graminit.c and graminit.h some other way?
Again, I really appreciate all the help
Sun
@Daetalus
Will reply you later...
Stefan O'Neil
@stefanoneil
Sure thing, thanks again
Kevin Modzelewski
@kmod
I believe we copied Python-ast.h and Python-ast.c in from the CPython repository
and probably graminit.h/c as well
We never experimented with changing the grammar, but maybe one way to do that is to make the change in a CPython checkout, regenerate the various files, and re-copy them into Pyston
you probably need to regenerate src/codegen/cpython_ast.cpp (it includes the script to do so at the top of the file)
as I said, we haven't tried this, so I don't expect it to be super straightforward. let us know if you run into issues!
Stefan O'Neil
@stefanoneil
Oh I see, that’s good to know, thanks.
Stefan O'Neil
@stefanoneil
Yup, just confirmed, the graminit files are indeed identical with CPython. My apologies for the confusion.
Stefan O'Neil
@stefanoneil
Taking your recommendation for regenerating the graminit files, plus modifying a number of other files, I have successfully extended Pyston with a keyword, except for one problem
if I try to use the keyword from within a function, Pyston crashes.
in every other context, it seems to work just fine
the behavior of the keyword itself shouldn't be breaking anything, since it is just a trivial keyword for now