These are chat archives for dropbox/pyston

24th
Apr 2015
Chris Toshok
@toshok
Apr 24 2015 00:34
is this rewrite abort because we are calling get, which can invalidate the guards we have on this rewrite?
Kevin Modzelewski
@kmod
Apr 24 2015 00:41
I think it's because we can't do guards after a mutation
Chris Toshok
@toshok
Apr 24 2015 00:41
right
Kevin Modzelewski
@kmod
Apr 24 2015 00:41
and a generic __get__ could cause mutations, and then we'd potentially have to do more guards after
(it could also abort once we run into those guards)
I was looking into why virtualenv is slow, and it looks like a big contributor is that regex's are slow since they hit this issue
what's the type of the descr object?
I'm thinking of adding rewriting for method_cls aka BoxedMethodDescriptors
Chris Toshok
@toshok
Apr 24 2015 00:43
one thing about callable-iterator is that it’s tp_iter is PyObject_SelfIter. detecting that (and just replacing the obj.__iter__ call with obj would fix this particular case
Kevin Modzelewski
@kmod
Apr 24 2015 00:43
can we call tp_iter?
rather than looking for __iter__?
at least if rewrite_args is null
Chris Toshok
@toshok
Apr 24 2015 00:45
yeah, I expect so. let me see if I can have it bail out earlier
Travis Hance
@tjhance
Apr 24 2015 01:31
yes, we often bail when we call __get__
you might be running into this? dropbox/pyston#189
or something similar
(ooooh, shiny gitter feature)
Chris Toshok
@toshok
Apr 24 2015 01:36
Yup, that looks to be exactly it
andrewchambers
@andrewchambers
Apr 24 2015 02:04
Are you guys planning on complete compatibility, or are you willing to compromise obscure features for speed?
I guess your focus is dropbox server code anyway.
Chris Toshok
@toshok
Apr 24 2015 02:06
Given the size of the code base we want to support, chances are there are few unused obscure features :)
andrewchambers
@andrewchambers
Apr 24 2015 02:07
Haha, just have some flags -risky -crazy -areyoumad
Though It is a bit of a cheat for speed, it may allow you to selectively dominate pypy
But full compat still needed anyway. So keep going. watching this project is really fun.
andrewchambers
@andrewchambers
Apr 24 2015 02:12
You could also adopt the type annotations stuff which is going around.
or just custom annotations
which are noops outside of pystone
pyston*
@inline @purefunc etc etc
andrewchambers
@andrewchambers
Apr 24 2015 02:17
The fact that you are a method jit I think gives you a cleaner boundary to apply these sorts of things.
Marius Wachtler
@undingen
Apr 24 2015 10:11
has anyone noticed the platform.py:libc_ver function? It looks like it searches inside the pyston binary with a regular expression for the glibc version....
and this is terrible slow for pyston compared to cpython. virtualenv uses this function...
Marius Wachtler
@undingen
Apr 24 2015 10:24
the pyston executable is also 60MB vs. 4MB... :-D
when I strip the executable the virtualenv integration test get's 2.5secs faster..
Marius Wachtler
@undingen
Apr 24 2015 10:29
?!?
Marius Wachtler
@undingen
Apr 24 2015 11:13
we should create a custom linker script which puts the glibc string at the first few bytes of the executable... :-P
Marius Wachtler
@undingen
Apr 24 2015 11:30
when I let cpython retrieve the libc version of the pyston executable it takes 2.9secs, pyston needs 3.6secs. So we are not much slower (which does not surprise me because the regex stuff is written in C). When I retrieve the libc version of the cpython executable it takes 0.2secs(cpython) vs 0.3 (pyston)
Chris Toshok
@toshok
Apr 24 2015 17:00
why is virtualenv even doing that? that’s bizarre :)
Marius Wachtler
@undingen
Apr 24 2015 17:05
ok I looked it up: pip uses it for the download user agent string
and it's gets multiple times executed because virtualenv installs pip and setuputils.
Kevin Modzelewski
@kmod
Apr 24 2015 18:47
lol that's crazy
btw @toshok did you upgrade jitdev to gcc 4.9? :P
Chris Toshok
@toshok
Apr 24 2015 18:48
i didn’t.. at least not knowingly
did gcc change?
ah, it has both
Kevin Modzelewski
@kmod
Apr 24 2015 18:50
hmm I thought 4.8 was the default on 14.04
and it looks like jitdev has some added ppas that might be related
Chris Toshok
@toshok
Apr 24 2015 18:51
ah, yeah likely due to me trying to build libgcc :/
is it interfering? is cmake picking it over 4.8?
Kevin Modzelewski
@kmod
Apr 24 2015 18:52
well it looks like the 4.9 libstdc++ is the default now
which causes those static_asserts
Chris Toshok
@toshok
Apr 24 2015 18:53
ugh
Kevin Modzelewski
@kmod
Apr 24 2015 18:53
but also some weird errors about no member named 'max_align_t' in the global namespace
Daniel Agar
@dagar
Apr 24 2015 18:56
I was going to try and remove the hard coded BoxedDict size
but, I'm not sure how to deal with the header dependencies
Kevin Modzelewski
@kmod
Apr 24 2015 19:05
chris are you able to build pyston on jitdev?
Chris Toshok
@toshok
Apr 24 2015 19:05
after rebasing no (switching to cmake) - apparently I hadn’t rebased since before that switch happened
without forcing make, that is
let me remove gcc-4.9. it’s certainly not needed by anything we need
Chris Toshok
@toshok
Apr 24 2015 19:17
hrm, uninstalling libstdc++-4.9 causes c++ compiles to fail. what tells clang++ which version directory of /usr/include/c++ to use?
crap, have to run for dr’s appt :/
Kevin Modzelewski
@kmod
Apr 24 2015 19:25
I believe that's determined/specified at configure time
Chris Toshok
@toshok
Apr 24 2015 19:35
I tried compiling a c++ file that just included one of the libstdc++ headers and it failed.
Kevin Modzelewski
@kmod
Apr 24 2015 19:53
@undingen that's crazy... can we just hardcode the libc version or something?
I think our makefile build adds debug symbols by default which is why it ends up being 60MB
the cmake build ends up around 20MB since it doesn't have debug symbols
pypy's binary is something like 40MB so we're in good company :P
Chris Toshok
@toshok
Apr 24 2015 21:23
k, jitdev cpython builds should be fine now
s/cpython/cmake
Kevin Modzelewski
@kmod
Apr 24 2015 21:24
awesome, thanks :)
Michael Arntzenius
@rntz
Apr 24 2015 22:11
hm, looks like libunwind doesn't pick up on the handler or lsda for jitted frames
I should... probably have noticed that before now.
Kevin Modzelewski
@kmod
Apr 24 2015 22:23
does it make sense to think about still using the libgcc unwinder?
ie is it something that's reusable at all?
Chris Toshok
@toshok
Apr 24 2015 22:41
on top of the regexp stuff, django does this while parsing variable nodes as well:
try:
   # is it a float?
   self.literal = float(var)

  # ok it’s a float
  ...
except ValueError:
   # ok it’s not a float
   ...
Michael Arntzenius
@rntz
Apr 24 2015 22:54
@kmod Possibly! It's also possible we're just not passing the right arguments to _U_dyn_register, or there's some way of getting the info out in a different way.
Kevin Modzelewski
@kmod
Apr 24 2015 23:01
yeah and we do that weird thing where we register it as a "remote table"
Travis Hance
@tjhance
Apr 24 2015 23:19
guys we need to support CPython bytecode so we can run this: http://pyos.github.io/dg/
Chris Toshok
@toshok
Apr 24 2015 23:44
sure would be nice if we could inline the float(), and if we’re going to throw inside a matching try: we just branch directly to the landing pad