These are chat archives for dropbox/pyston

27th
Jul 2015
Travis Hance
@tjhance
Jul 27 2015 05:44 UTC
so what exactly is “calibration” in the measure_perf script for?
it appears to be a benchmark like any other
what’s special about it?
Travis Hance
@tjhance
Jul 27 2015 06:25 UTC

@tjhance I thought of a potential project -- it'd be nice if we could have some ability to guard in the middle of a patchpoint (ie after mutations)
ex for the callattr-of-descriptor case, but also for binops and calling constructors

why does this apply to binops and constructors?

Kevin Modzelewski
@kmod
Jul 27 2015 08:14 UTC
calibration is that the tester runs a fixed workload
so that you can get a measure of the overall runtime variability
ex if your machine is slower from doing other things as well
binops and constructors both have the form "do this first operation, and depending on the result do a second one"
for binops it's "first try lh.op(rhs), and if it's NotImplemented try rhs.rop(lhs)"
for constructors its "call cls.new, and if it's a subtype of cls call obj.init"
Sun
@Daetalus
Jul 27 2015 08:19 UTC
Hi, intBin(aka bin builtin function) not implemented. I am tring to implement it. intOct and intHex use snprintf. For intBin, could I convert decimal to binary manually? Or does there has a better way?
Travis Hance
@tjhance
Jul 27 2015 08:22 UTC
ah
Kevin Modzelewski
@kmod
Jul 27 2015 08:23 UTC
currently we go to some pretty decent lengths to try to find sets of cases where we can determine ahead of time how the second check will work out
but it's getting harder to keep finding those
@Daetalus the generic answer is to look at how CPython does it and copy it :)
Sun
@Daetalus
Jul 27 2015 08:24 UTC
Ok, thanks @kmod
Kevin Modzelewski
@kmod
Jul 27 2015 08:25 UTC
luckily our code is pretty similar to theirs (due to a lot of it being shared) so hopefully it's not too hard to get up and running with reading their codebase
it looks like in this case we should copy _PyInt_Format and hook up the bin/oct/hex to use that
Sun
@Daetalus
Jul 27 2015 08:27 UTC
Got it. In the meantime, I am reading "The Garbage Collection Handbook", hope I could make contribution to GC soon.
Kevin Modzelewski
@kmod
Jul 27 2015 08:34 UTC
nice :)
did you see rudi-c's doc?
sounds like things might start getting exciting for our gc :)
Sun
@Daetalus
Jul 27 2015 08:35 UTC
Yes.
But I could only see the last week summary.
Sun
@Daetalus
Jul 27 2015 08:41 UTC
Hi, it seems Python 2.7.6 use _PyLong_Format for both long and int. So I do the same way. Is that Ok?
Kevin Modzelewski
@kmod
Jul 27 2015 08:41 UTC
oh, not sure if he shared it, but rudi's thinking about extending our GC and he put together a good doc on it
Sun
@Daetalus
Jul 27 2015 08:42 UTC
Thank you very much! I will read it carefully!
Kevin Modzelewski
@kmod
Jul 27 2015 08:43 UTC
hmm in the version I have checked out, it looks like PyNumber_ToBase has a call to _PyInt_Format
maybe we're looking at different versions or different functions?
Sun
@Daetalus
Jul 27 2015 08:46 UTC
I am reading the Python 2.7.6 source code. Wait a moment please.
Confirmed, PyNumber_ToBase only called _PyLong_Format. And _PyInt_Format existed in Python 2.7.3. Didn't check other version.
Kevin Modzelewski
@kmod
Jul 27 2015 08:48 UTC
hmm interesting
how are you looking at the source code / how did you get it?
it looks like python 3 only has _PyLong_Format
Sun
@Daetalus
Jul 27 2015 08:48 UTC
Download the Python 2.7 source code from internet...
Kevin Modzelewski
@kmod
Jul 27 2015 08:49 UTC
was it cloning from the official repo?
Sun
@Daetalus
Jul 27 2015 08:49 UTC
python2-7-6278231825a1 I will try to search the code again.
Kevin Modzelewski
@kmod
Jul 27 2015 08:50 UTC
just asking since I think it's confusing that both python3 and python2 share a repository and it's not easy to tell which branch you're on
Sun
@Daetalus
Jul 27 2015 08:50 UTC
Couldn't remember. I will comfirm it.
Sun
@Daetalus
Jul 27 2015 09:07 UTC
I am wrong. I just got a correct Python 2.7 source code.
Sun
@Daetalus
Jul 27 2015 09:12 UTC
After some glance, if copy the _PyInt_Format to Pyston with some modification. It will be similar as _PyLong_Format. Is that fine?
Sun
@Daetalus
Jul 27 2015 09:18 UTC
Ignore the last message. I will try to implement _PyInt_Format.
Kevin Modzelewski
@kmod
Jul 27 2015 09:28 UTC
is it possible to just copy what they have directly?
Dong-hee Na
@corona10
Jul 27 2015 09:39 UTC
This message was deleted
Marius Wachtler
@undingen
Jul 27 2015 09:45 UTC
I don't know flask does it use the capi? I normally use our patched virtualenv (lives inside test/integration/virtualenv/) and install the library inside it with pip. If the source code needs patches I checkout the github source directly.
Dong-hee Na
@corona10
Jul 27 2015 09:45 UTC
omg.. i deleted it.
whenever i tried to install flask _ast.__version was not implemented.. so i implementing it
Sun
@Daetalus
Jul 27 2015 09:46 UTC
@kmod No need to copy it. It already exist in Pyston...
Dong-hee Na
@corona10
Jul 27 2015 09:47 UTC
and then pyston setup.py install than pyston stoped at importing _ctypes when i try to install setuptools
Marius Wachtler
@undingen
Jul 27 2015 09:48 UTC
oh yeah I think we miss the ast module, hope you just need the version... :-D
ctypes isn't yet integrated :-(.
Dong-hee Na
@corona10
Jul 27 2015 09:49 UTC
so i tried to integrating it so i need some way to integrating library i want to know big picture to do it
i means what source code can get i some tips..
Marius Wachtler
@undingen
Jul 27 2015 09:54 UTC
The source should be already inside pyston from_cpython directory. Next you have to integrate it into the build system
Dong-hee Na
@corona10
Jul 27 2015 09:56 UTC
ah thanks i will try it.
Marius Wachtler
@undingen
Jul 27 2015 09:56 UTC
if it should get directly linked into the pyston you have to add all source files to from_cpython/CMakeLists.txt file(GLOB_RECURSE STDMODULE_SRCS Modules
but ctypes is a separate shared library inside cpython so we should do the same: you have to adopt from_cpython/setup.py
use the setup.py from cpython as template.
We mark all changes we make inside source imported from cpython with: // Pyston change
Kevin Modzelewski
@kmod
Jul 27 2015 09:59 UTC
@rudi-c has been looking into ctypes and it sounds like there's some difficulty there
Marius Wachtler
@undingen
Jul 27 2015 10:00 UTC
:-(
Kevin Modzelewski
@kmod
Jul 27 2015 10:00 UTC
hopefully he can have it working soon :) but sounds like it's not just an issue of enabling it
I'd wait until he has an update on it
also, if import ast is failing for you, you can try editing src/runtime/builtin_modules/ast.cpp
Dong-hee Na
@corona10
Jul 27 2015 10:01 UTC
oh i did it about ast
can i pull request with list recursive printing?
Marius Wachtler
@undingen
Jul 27 2015 10:02 UTC
Oh I didn't know we have an AST module implementation... :-)
Kevin Modzelewski
@kmod
Jul 27 2015 10:02 UTC
err we don't really
if you define __version__ it's enough to get the import to work :P
though none of the features are implemented
but there are some things that try to import it first and then only in some rare code path actually make use of it
is this #764?
Dong-hee Na
@corona10
Jul 27 2015 10:05 UTC
#764 is just pull request about list recursive printing
if i pull request about _ast it may be contains list recursive printing it will be okay?
Kevin Modzelewski
@kmod
Jul 27 2015 10:07 UTC
well, I just merged #764 :) so if it's that specifically it might be a moot point
Sun
@Daetalus
Jul 27 2015 10:08 UTC
Please review #759 and #763 if you have time.
Dong-hee Na
@corona10
Jul 27 2015 10:22 UTC
thanks i pull #765 .
Dong-hee Na
@corona10
Jul 27 2015 12:16 UTC
funny thing happen with #765 commit
before i implemented ast._version
pyston pass integeration test
This message was deleted
This message was deleted
This message was deleted
This message was deleted
Dong-hee Na
@corona10
Jul 27 2015 12:23 UTC
This message was deleted
oh i got what is wrong.. haha
Marius Wachtler
@undingen
Jul 27 2015 12:28 UTC
ok, it could be that now that version is there that it tries to access more of the ast modules code which isn't yet implemented. and before it may just have risen an importerror etc...
and debugging what's wrong going wrong inside the virtualenv script is quite annoying because of all the subprocesses :-(
Dong-hee Na
@corona10
Jul 27 2015 13:48 UTC
haha i read comment now that
    // Uncommenting this makes `import ast` work, which may or may not be desired.
    // For now it seems like making the import fail is better than having the module not work properly.
    // ast_module->giveAttr("__version__", boxInt(82160));
if it is true that we don't have to maintain ??

from _ast import __version__
This message was deleted
Rudi Chen
@rudi-c
Jul 27 2015 17:12 UTC
@Daetalus Just gave you access to the GC docs.
Sun
@Daetalus
Jul 27 2015 17:13 UTC
@rudi-c Thank you very much!
Rudi Chen
@rudi-c
Jul 27 2015 17:17 UTC
Just wondering, do you get an email notification?
Sun
@Daetalus
Jul 27 2015 17:17 UTC
Yes, I got the notification.
Rudi Chen
@rudi-c
Jul 27 2015 17:18 UTC
kk cool
Chris Toshok
@toshok
Jul 27 2015 19:24 UTC
hm, ugh. virtualenv very clearly does not want to run out of python’s build directory