by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 27 2019 23:49
  • Jan 17 2019 05:30
    xxllp commented #1426
  • Dec 13 2018 23:14
    deronnax closed #1421
  • Jul 14 2018 08:28
    corporatepiyush commented #1426
  • May 23 2018 21:37
    toshok closed #528
  • May 23 2018 21:37
    toshok closed #558
  • May 23 2018 21:34
    toshok closed #609
  • May 23 2018 21:34
    toshok closed #756
  • Feb 21 2018 05:25
    osman-masood commented #1426
  • Feb 03 2018 02:34
    skywind3000 commented #1426
  • Feb 01 2018 22:06
    danluu commented #1426
  • Jan 27 2018 19:57
    lesshaste commented #1426
  • Jan 12 2018 13:11
    skywind3000 opened #1426
  • Nov 12 2017 19:59
    xoviat commented #1421
  • Aug 25 2017 15:58
    Travis dropbox/pyston (py3) passed (4685)
  • Aug 25 2017 15:31
    kmod commented #1425
  • Aug 25 2017 15:31
    kmod closed #1425
  • Aug 24 2017 15:02
    kmod commented #1422
  • Aug 01 2017 17:06
    denji closed #378
  • Jul 28 2017 00:31
    Travis dropbox/pyston#1425 passed (4684)
Kevin Modzelewski
@kmod
Yep you could! There are some tricky cases (integer addition overflows into longs), but you could get integer addition to be fast
other cases get much more complicated and you start needing more help from the rest of the runtime
ex fetching attributes
Ethan Smith
@ethanhs
Ah, yes of course that makes a lot of sense. I didn't consider that. I suppose you could be clever and do things like cache frames, but that would still be slower. Thanks!
mhsjlw
@mhsjlw
@kmod my question about compiling to a binary was just because I wanted to run it on my raspberry pi and not need an entire jit
and as far as libunwind goes, i think it would be really good to figure out a way to fix it, because as far as i can see, if we don't need libunwind i'm pretty sure i'd be able to get this to work on a mac
this is kind of stupid, but if you know a way to compile to a binary we could go beyond raspberry pi and maybe even run python on ios?
anyway, that's aside the point
i know languages like crystal and rust use the llvm and they are able to compile to native binaries, but from what i can see from pyston it's not exactly simple
Ethan Smith
@ethanhs
@mhsjlw the issue is that one must write to either JIT or statically compile. A JIT works fundamentally differently.
mhsjlw
@mhsjlw
of course, yeah
Stefan O'Neil
@stefanoneil
Hi everyone! I have some questions about working with Pyston source code.
Is there any documentation about the command line arguments that Pyston accepts? I cannot find such documentation anywhere, and neither “-h” nor “--help" seem to be supported. However it would help me tremendously to know all the different options I can use with Pyston.
How can I inspect the assembly code that Pyston generates? Part of the project that I am working on requires building support into Pyston for generating ARM code, so being able to check the machine code that Pyston is generating will be pretty essential.
Finally, how can I check the virtual memory address for a given variable/object? Another aspect of the project I am working on requires that I am able to map memory regions from a Pyston process into kernel memory, and then externally access or modify values internal to that Pyston process (probably using mmap() from within a kernel module). Thus, being able to find exactly where in memory an object is stored would be very useful. I find that id() often returns negative values - is this still a memory address, perhaps using twos complement, or is id() not a reliable way to check addresses in Pyston?
My apologies if any of these questions are obvious or answered elsewhere already - I did search fairly extensively before asking, but I could easily have overlooked something. I am pretty new both to the world of compilers and to exploring large code bases. All help is deeply appreciated.
Kevin Modzelewski
@kmod
Our command line flags mostly follow CPython's; you can find them here:
https://github.com/dropbox/pyston/blob/master/src/jit.cpp#L184
I think you can pass the -j flag and Pyston will spit out a .o file for each function it compiles, which you can then disassemble with objdump
There was an old reason that it xor'd with a constant, which should no longer be necessary
ARM code would be awesome :) but I'm sure is a lot of work
and since things aren't being actively developed it might be hard to get help
We have two places we emit code: one is through the LLVM jit, which should have ARM support. the other is our jit (src/asm_writing), which is probably much harder to convert
Kevin Modzelewski
@kmod
You'll probably want to liberally deactivate features in order to get started
Chris Toshok
@toshok
hey all, this thing still on? :)
got pyston running on OSX (baseline jit only at the moment) https://github.com/toshok/pyston/tree/osx-port
really, really rough at the moment.
Kevin Modzelewski
@kmod
oh whoa, cool!
Chris Toshok
@toshok
oh and absolutely nothing unwinding-related works :)
but that shouldn't be too hard to get working
Sun
@Daetalus
great!
Stefan O'Neil
@stefanoneil
Thanks for the helpful pointers @kmod !
Stefan O'Neil
@stefanoneil
I have another quick question for you or anyone who is decently familiar with the Pyston source code. Something I am working on in my current project is extending Pyston grammar with an additional keyword. In CPython, Python/graminit.c and Include/graminit.h are auto-generated from Grammar/Grammar when you run make.
In tracking down all the files I need to modify to implement an additional keyword in Pyston, I can’t seem to find the grammar file from which graminit.c and graminit.h are generated. The only grammar file I can find is from_cpython/Lib/lib2to3/Grammar.txt, but modifying this does not seem to trigger a regeneration of the graminit files.
Where should I be looking for the grammar file that Pyston generates these from?
Sun
@Daetalus
Here: https://github.com/dropbox/pyston/tree/master/src/codegen. But first you may need copy new generated CPython Python-ast.h file. And then extend the Pyston interpreter and jit.
Stefan O'Neil
@stefanoneil
Excellent, thanks a ton for that
I believe in CPython, Python-ast.h and Python-ast.c are both generated by Parser/asdl_c.py whenever Parser/Python.asdl is changed. Do you happen to know if Pyston works this way as well?
Stefan O'Neil
@stefanoneil
Also, you linked me the src/codegen directory, but I am not seeing a grammar specification file in here. Does that mean Pyston not use a grammar specification file similar to CPython's Grammar/Grammar, but rather generates graminit.c and graminit.h some other way?
Again, I really appreciate all the help
Sun
@Daetalus
Will reply you later...
Stefan O'Neil
@stefanoneil
Sure thing, thanks again
Kevin Modzelewski
@kmod
I believe we copied Python-ast.h and Python-ast.c in from the CPython repository
and probably graminit.h/c as well
We never experimented with changing the grammar, but maybe one way to do that is to make the change in a CPython checkout, regenerate the various files, and re-copy them into Pyston
you probably need to regenerate src/codegen/cpython_ast.cpp (it includes the script to do so at the top of the file)
as I said, we haven't tried this, so I don't expect it to be super straightforward. let us know if you run into issues!
Stefan O'Neil
@stefanoneil
Oh I see, that’s good to know, thanks.
Stefan O'Neil
@stefanoneil
Yup, just confirmed, the graminit files are indeed identical with CPython. My apologies for the confusion.
Stefan O'Neil
@stefanoneil
Taking your recommendation for regenerating the graminit files, plus modifying a number of other files, I have successfully extended Pyston with a keyword, except for one problem