These are chat archives for dropbox/pyston

27th
Jul 2015
Travis Hance
@tjhance
Jul 27 2015 05:44
so what exactly is “calibration” in the measure_perf script for?
it appears to be a benchmark like any other
what’s special about it?
Travis Hance
@tjhance
Jul 27 2015 06:25

@tjhance I thought of a potential project -- it'd be nice if we could have some ability to guard in the middle of a patchpoint (ie after mutations)
ex for the callattr-of-descriptor case, but also for binops and calling constructors

why does this apply to binops and constructors?

Kevin Modzelewski
@kmod
Jul 27 2015 08:14
calibration is that the tester runs a fixed workload
so that you can get a measure of the overall runtime variability
ex if your machine is slower from doing other things as well
binops and constructors both have the form "do this first operation, and depending on the result do a second one"
for binops it's "first try lh.op(rhs), and if it's NotImplemented try rhs.rop(lhs)"
for constructors its "call cls.new, and if it's a subtype of cls call obj.init"
Sun
@Daetalus
Jul 27 2015 08:19
Hi, intBin(aka bin builtin function) not implemented. I am tring to implement it. intOct and intHex use snprintf. For intBin, could I convert decimal to binary manually? Or does there has a better way?
Travis Hance
@tjhance
Jul 27 2015 08:22
ah
Kevin Modzelewski
@kmod
Jul 27 2015 08:23
currently we go to some pretty decent lengths to try to find sets of cases where we can determine ahead of time how the second check will work out
but it's getting harder to keep finding those
@Daetalus the generic answer is to look at how CPython does it and copy it :)
Sun
@Daetalus
Jul 27 2015 08:24
Ok, thanks @kmod
Kevin Modzelewski
@kmod
Jul 27 2015 08:25
luckily our code is pretty similar to theirs (due to a lot of it being shared) so hopefully it's not too hard to get up and running with reading their codebase
it looks like in this case we should copy _PyInt_Format and hook up the bin/oct/hex to use that
Sun
@Daetalus
Jul 27 2015 08:27
Got it. In the meantime, I am reading "The Garbage Collection Handbook", hope I could make contribution to GC soon.
Kevin Modzelewski
@kmod
Jul 27 2015 08:34
nice :)
did you see rudi-c's doc?
sounds like things might start getting exciting for our gc :)
Sun
@Daetalus
Jul 27 2015 08:35
Yes.
But I could only see the last week summary.
Sun
@Daetalus
Jul 27 2015 08:41
Hi, it seems Python 2.7.6 use _PyLong_Format for both long and int. So I do the same way. Is that Ok?
Kevin Modzelewski
@kmod
Jul 27 2015 08:41
oh, not sure if he shared it, but rudi's thinking about extending our GC and he put together a good doc on it
Sun
@Daetalus
Jul 27 2015 08:42
Thank you very much! I will read it carefully!
Kevin Modzelewski
@kmod
Jul 27 2015 08:43
hmm in the version I have checked out, it looks like PyNumber_ToBase has a call to _PyInt_Format
maybe we're looking at different versions or different functions?
Sun
@Daetalus
Jul 27 2015 08:46
I am reading the Python 2.7.6 source code. Wait a moment please.
Confirmed, PyNumber_ToBase only called _PyLong_Format. And _PyInt_Format existed in Python 2.7.3. Didn't check other version.
Kevin Modzelewski
@kmod
Jul 27 2015 08:48
hmm interesting
how are you looking at the source code / how did you get it?
it looks like python 3 only has _PyLong_Format
Sun
@Daetalus
Jul 27 2015 08:48
Download the Python 2.7 source code from internet...
Kevin Modzelewski
@kmod
Jul 27 2015 08:49
was it cloning from the official repo?
Sun
@Daetalus
Jul 27 2015 08:49
python2-7-6278231825a1 I will try to search the code again.
Kevin Modzelewski
@kmod
Jul 27 2015 08:50
just asking since I think it's confusing that both python3 and python2 share a repository and it's not easy to tell which branch you're on
Sun
@Daetalus
Jul 27 2015 08:50
Couldn't remember. I will comfirm it.
Sun
@Daetalus
Jul 27 2015 09:07
I am wrong. I just got a correct Python 2.7 source code.
Sun
@Daetalus
Jul 27 2015 09:12
After some glance, if copy the _PyInt_Format to Pyston with some modification. It will be similar as _PyLong_Format. Is that fine?
Sun
@Daetalus
Jul 27 2015 09:18
Ignore the last message. I will try to implement _PyInt_Format.
Kevin Modzelewski
@kmod
Jul 27 2015 09:28
is it possible to just copy what they have directly?
Dong-hee Na
@corona10
Jul 27 2015 09:39
This message was deleted
Marius Wachtler
@undingen
Jul 27 2015 09:45
I don't know flask does it use the capi? I normally use our patched virtualenv (lives inside test/integration/virtualenv/) and install the library inside it with pip. If the source code needs patches I checkout the github source directly.
Dong-hee Na
@corona10
Jul 27 2015 09:45
omg.. i deleted it.
whenever i tried to install flask _ast.__version was not implemented.. so i implementing it
Sun
@Daetalus
Jul 27 2015 09:46
@kmod No need to copy it. It already exist in Pyston...
Dong-hee Na
@corona10
Jul 27 2015 09:47
and then pyston setup.py install than pyston stoped at importing _ctypes when i try to install setuptools
Marius Wachtler
@undingen
Jul 27 2015 09:48
oh yeah I think we miss the ast module, hope you just need the version... :-D
ctypes isn't yet integrated :-(.
Dong-hee Na
@corona10
Jul 27 2015 09:49
so i tried to integrating it so i need some way to integrating library i want to know big picture to do it
i means what source code can get i some tips..
Marius Wachtler
@undingen
Jul 27 2015 09:54
The source should be already inside pyston from_cpython directory. Next you have to integrate it into the build system
Dong-hee Na
@corona10
Jul 27 2015 09:56
ah thanks i will try it.
Marius Wachtler
@undingen
Jul 27 2015 09:56
if it should get directly linked into the pyston you have to add all source files to from_cpython/CMakeLists.txt file(GLOB_RECURSE STDMODULE_SRCS Modules
but ctypes is a separate shared library inside cpython so we should do the same: you have to adopt from_cpython/setup.py
use the setup.py from cpython as template.
We mark all changes we make inside source imported from cpython with: // Pyston change
Kevin Modzelewski
@kmod
Jul 27 2015 09:59
@rudi-c has been looking into ctypes and it sounds like there's some difficulty there
Marius Wachtler
@undingen
Jul 27 2015 10:00
:-(
Kevin Modzelewski
@kmod
Jul 27 2015 10:00
hopefully he can have it working soon :) but sounds like it's not just an issue of enabling it
I'd wait until he has an update on it
also, if import ast is failing for you, you can try editing src/runtime/builtin_modules/ast.cpp
Dong-hee Na
@corona10
Jul 27 2015 10:01
oh i did it about ast
can i pull request with list recursive printing?
Marius Wachtler
@undingen
Jul 27 2015 10:02
Oh I didn't know we have an AST module implementation... :-)
Kevin Modzelewski
@kmod
Jul 27 2015 10:02
err we don't really
if you define __version__ it's enough to get the import to work :P
though none of the features are implemented
but there are some things that try to import it first and then only in some rare code path actually make use of it
is this #764?
Dong-hee Na
@corona10
Jul 27 2015 10:05
#764 is just pull request about list recursive printing
if i pull request about _ast it may be contains list recursive printing it will be okay?
Kevin Modzelewski
@kmod
Jul 27 2015 10:07
well, I just merged #764 :) so if it's that specifically it might be a moot point
Sun
@Daetalus
Jul 27 2015 10:08
Please review #759 and #763 if you have time.
Dong-hee Na
@corona10
Jul 27 2015 10:22
thanks i pull #765 .
Dong-hee Na
@corona10
Jul 27 2015 12:16
funny thing happen with #765 commit
before i implemented ast._version
pyston pass integeration test
This message was deleted
This message was deleted
This message was deleted
This message was deleted
Dong-hee Na
@corona10
Jul 27 2015 12:23
This message was deleted
oh i got what is wrong.. haha
Marius Wachtler
@undingen
Jul 27 2015 12:28
ok, it could be that now that version is there that it tries to access more of the ast modules code which isn't yet implemented. and before it may just have risen an importerror etc...
and debugging what's wrong going wrong inside the virtualenv script is quite annoying because of all the subprocesses :-(
Dong-hee Na
@corona10
Jul 27 2015 13:48
haha i read comment now that
    // Uncommenting this makes `import ast` work, which may or may not be desired.
    // For now it seems like making the import fail is better than having the module not work properly.
    // ast_module->giveAttr("__version__", boxInt(82160));
if it is true that we don't have to maintain ??

from _ast import __version__
This message was deleted
Rudi Chen
@rudi-c
Jul 27 2015 17:12
@Daetalus Just gave you access to the GC docs.
Sun
@Daetalus
Jul 27 2015 17:13
@rudi-c Thank you very much!
Rudi Chen
@rudi-c
Jul 27 2015 17:17
Just wondering, do you get an email notification?
Sun
@Daetalus
Jul 27 2015 17:17
Yes, I got the notification.
Rudi Chen
@rudi-c
Jul 27 2015 17:18
kk cool
Chris Toshok
@toshok
Jul 27 2015 19:24
hm, ugh. virtualenv very clearly does not want to run out of python’s build directory