These are chat archives for dropbox/pyston

19th
Mar 2015
Chris Toshok
@toshok
Mar 19 2015 03:21
hm, pyston ../path/to/file.py isn’t happy when we lack permissions to write to ../path/to/
Kevin Modzelewski
@kmod
Mar 19 2015 03:33
drat I thought I had fixed that
Chris Toshok
@toshok
Mar 19 2015 03:34
works with the pypa parser
Kevin Modzelewski
@kmod
Mar 19 2015 03:38
ok, I had missed one of two places we could try to write out the pyc file
try rebasing off master?
Travis Hance
@tjhance
Mar 19 2015 03:39
We write pyc files?
Kevin Modzelewski
@kmod
Mar 19 2015 03:40
yeah, to avoid doing the slow shelling-out-to-cpython
it's just the output of parse_ast.py, since we had already come up with a hacky wire protocol for that
Travis Hance
@tjhance
Mar 19 2015 04:23
ohh
Travis Hance
@tjhance
Mar 19 2015 05:30
time for me to learn how OSR works ^.^
Chris Toshok
@toshok
Mar 19 2015 05:44
@kmod: works
Travis Hance
@tjhance
Mar 19 2015 05:45
it seems like the most reasonable thing to do is just pass a FrameInfo* to the OSR function as an argument
just like we pass the variables as arguments
hmm
that might not actually be necessary
the OSR function can just know where the FrameInfo on the stack lies
since it should be at a fixed place, relatively speaking
although that involves some assumptions
for example, that that particular OSR function will only be used for that one particular compilation of the function its in
hm
Travis Hance
@tjhance
Mar 19 2015 05:50
I guess I’ll stick with the easier thing for now
and optimize later
although it’s not actually clear that this is easier
Travis Hance
@tjhance
Mar 19 2015 05:55
since it means that the OSR function has a FrameInfo* variable instead of just a FrameInfo alloca
hm actually, that doesn’t cause much difficulty, because the IR is still the same (i.e., I can use a pointer and the result of an alloca interchangeably… I think)
checks llvm docs
Chris Toshok
@toshok
Mar 19 2015 05:58
yeah, allocas give you a pointer
Travis Hance
@tjhance
Mar 19 2015 05:58
okay yeah, the result of an alloca is a pointer type
so it will be less efficient code, but nearly the same IR
Chris Toshok
@toshok
Mar 19 2015 06:37
in module scope globals() and locals() should be the same, right?
Travis Hance
@tjhance
Mar 19 2015 06:38
I think so
right now that is probably not true
Chris Toshok
@toshok
Mar 19 2015 06:38
nod. this test fails on pyston:
major = 0
minor = 4
micro = 0
print '%(major)d.%(minor)d.%(micro)d' % (locals())
but s/locals/globals and it works
Travis Hance
@tjhance
Mar 19 2015 06:40
yeah hm
locals() is probably empty I assume?
Chris Toshok
@toshok
Mar 19 2015 06:41
yeah
Travis Hance
@tjhance
Mar 19 2015 06:41
yeah this is kinda tough
hmm
right so, in CPython, every variable access in the global scope is a NAME bytecode
which means that it’s actually modifying the locals dictionary, which happens to be the same as the module dictionary
which is how that’s supposed to work
in our case I guess we should just implement locals() differently for module scopes by having it return the module’s attrwrapper thingy
since we access globals by modifying the module attributes directly
Travis Hance
@tjhance
Mar 19 2015 06:51
In fact, locals() is globals() should be True in case that wasn’t clear
Kevin Modzelewski
@kmod
Mar 19 2015 08:13
oh interesting
could we just keep doing what we currently do but have locals() return the globals object in these cases?
ie, if they're the same, the user can't tell we set the name in the globals() instead of the locals()
I guess it's not clear how much that would buy us
@tjhance maybe the easiest thing for now is to just disable OSR/reoptimization if there is a boxedlocals?
or do we support boxedlocals in jitted frames?
Travis Hance
@tjhance
Mar 19 2015 16:53
I don’t see any reason to disable it for OSRs, and I don’t think it’s a good idea. FrameInfo also has the ExcInfo, which we need to preserve, and besides, a boxedLocals can pop up at any time (e.g. if the code calls locals())
switching topics back: so it turns out the __dict__ of a module is a readonly attribute
this is basically a christmas miracle
Travis Hance
@tjhance
Mar 19 2015 16:59
because, well, I don’t know for sure what would happen otherwise, but it would probably involve the globals() and locals() becoming different objects
and then that plan of just having locals() return the module dict wouldn’t work
but, it looks like we won’t have to worry about it
Travis Hance
@tjhance
Mar 19 2015 17:05
@kmod actually you might be right that we can disable OSR for the NAME frames
Marius Wachtler
@undingen
Mar 19 2015 17:05
are yo usure about module().dict being read only? I think I encountered it somewhere i nvurtualenv
Travis Hance
@tjhance
Mar 19 2015 17:06
huh really? I just tried it
Marius Wachtler
@undingen
Mar 19 2015 17:06
I'm trying to find it again
Travis Hance
@tjhance
Mar 19 2015 17:06
Traceback (most recent call last):
File "gl.py", line 11, in <module>
currentmodule._dict = {}
TypeError: readonly attribute
Marius Wachtler
@undingen
Mar 19 2015 17:07
ok it's doing a dict.update() and not replacing it
Travis Hance
@tjhance
Mar 19 2015 17:07
phew
Marius Wachtler
@undingen
Mar 19 2015 17:07

```# Module Factory Factory (no, this isn't Java, I know)

Here to stop this being duplicated all over the place.

def moduleFactoryFactory(factory):
moduleCache = {}

def moduleFactory(baseModule, *args, **kwargs):
    if isinstance(ModuleType.__name__, type("")):
        name = "_%s_factory" % baseModule.__name__
    else:
        name = b"_%s_factory" % baseModule.__name__

    if name in moduleCache:
        return moduleCache[name]
    else:
        mod = ModuleType(name)
        objs = factory(baseModule, *args, **kwargs)
        mod.__dict__.update(objs)
        moduleCache[name] = mod
        return mod

return moduleFactory```
Travis Hance
@tjhance
Mar 19 2015 17:08
@kmod almost all access of the FrameInfo are via introspection, right? The only exception is in a NAME scope where I access the boxedLocals directly?
In that case it would make sense to not pass in the FrameInfo to the OSR function and just disable OSR for the NAME scopes, at least temporarily
(well, you can say that almost anything makes sense “temporarily” but you know what I mean)
Kevin Modzelewski
@kmod
Mar 19 2015 19:27
well we should probably be passing the FrameInfo to make sys.exc_info propagate (this looks like it's currently a bug that we don't do that)
but I think for the boxedlocals you're right?
Marius Wachtler
@undingen
Mar 19 2015 19:37
I'm happy that I did not assign me to the exec task :-P
Chris Toshok
@toshok
Mar 19 2015 21:16
more locals/globals happiness: numpy has code that gets the caller (using traceback’s tbframe + frame’s fback), then does this: `caller_file = eval('__file', frame.f_globals, frame.f_locals)`
caller_file = eval('__file__', frame.f_globals, frame.f_locals)
alas, formatting