These are chat archives for dropbox/pyston

12th
Jun 2015
Rudi Chen
@rudi-c
Jun 12 2015 03:53
Ok, rebased everything and ran my code with timers again. Now for django-template, regular mark phase takes about 0.83s, finalization ordering mark phase takes 1.2s, almost nothing on actual finalizer calls. 8.3 million objects get marked as normally reachable across all GC runs, vs 1.3 million unreachable objects with finalizers or reachable from an object with finalizers
Out of a total of 14.3s
Not too bad
Chris Toshok
@toshok
Jun 12 2015 04:23
that is so weird that visiting fewer objects takes longer
Rudi Chen
@rudi-c
Jun 12 2015 04:25
Everything I need to compile should already be on the EC2 right?
I get ```
CMake Error at /home/rudi/pyston_deps/llvm-trunk/cmake/modules/AddLLVM.cmake:466 (add_executable):
  add_executable cannot create target "clang-tblgen" because another target
  with the same name already exists.  The existing target is an executable
  created in source directory
  "/home/rudi/pyston_deps/llvm-trunk/tools/clang/utils/TableGen".  See
  documentation for policy CMP0002 for more details.
Call Stack (most recent call first):
  /home/rudi/pyston_deps/llvm-trunk/cmake/modules/AddLLVM.cmake:538 (add_llvm_executable)
  /home/rudi/pyston_deps/llvm-trunk/cmake/modules/TableGen.cmake:76 (add_llvm_utility)
  /home/rudi/pyston_deps/llvm-trunk/tools/c/utils/TableGen/CMakeLists.txt:3 (add_tablegen)
Chris Toshok
@toshok
Jun 12 2015 04:26
yeah the cmake build should jfw
weird. did you build in the llvm directory already?
Rudi Chen
@rudi-c
Jun 12 2015 04:28
I did make llvm_up
Chris Toshok
@toshok
Jun 12 2015 04:30
can you gist the whole log? that’s really weird
Chris Toshok
@toshok
Jun 12 2015 04:37
hm, i blew away my pyston-build-dbg, and everything in pyston_deps except llvm-trunk
and things seem to be building
Rudi Chen
@rudi-c
Jun 12 2015 04:47
Ok, I blew away everything in ~ and restarted from scratch. Seems to be working so far.
Chris Toshok
@toshok
Jun 12 2015 05:11
cool
Rudi Chen
@rudi-c
Jun 12 2015 17:33
pyston (calibration)                      :    1.0s baseline: 1.0 (-0.1%)
pyston django_template.py                 :    8.0s baseline: 7.5 (+6.6%)
pyston sqlalchemy_imperative.py           : failed (code 1)
pyston django_migrate.py                  :    2.0s baseline: 2.0 (+0.9%)
pyston virtualenv_bench.py                : failed (code 1)
pyston interp2.py                         :    4.7s baseline: 4.7 (+1.6%)
pyston raytrace.py                        :    6.6s baseline: 6.1 (+9.7%)
pyston nbody.py                           :    9.6s baseline: 8.3 (+15.4%)
pyston fannkuch.py                        :    7.6s baseline: 6.9 (+10.1%)
pyston chaos.py                           :   21.5s baseline: 19.8 (+8.6%)
pyston fasta.py                           :    5.2s baseline: 4.9 (+6.2%)
pyston pidigits.py                        :    7.8s baseline: 5.7 (+37.8%)
pyston richards.py                        :    2.5s baseline: 2.4 (+2.6%)
pyston deltablue.py                       :    1.6s baseline: 1.6 (+0.4%)
For finalizers - perf doesn't show any major hotspots like allocationFrom anymore.
Not sure what's happening in pidigits.py
Kevin Modzelewski
@kmod
Jun 12 2015 20:28
You should try out the new investigate.py tool I wrote :P
there's no documentation but I can walk you through it
Chris Toshok
@toshok
Jun 12 2015 21:40
ah, yeah we can’t call endUnwind before switching to the landing pad :/
if we assume that endUnwind is either going to deallocate the ExcInfo storage or at the very least clear it
because the first thing the landing pad needs is the exception
Kevin Modzelewski
@kmod
Jun 12 2015 21:42
what if we keep ExcInfo managed the same way?
Chris Toshok
@toshok
Jun 12 2015 21:42
that has the same issue - the thing we want to do is unroot/clear the exc/type/tb
so we can only do that once the handler has copied it
we could insert a call to endUnwind() into every landing pad we jit
Kevin Modzelewski
@kmod
Jun 12 2015 21:54
hmm I think it's probably ok for now to not clear them
Chris Toshok
@toshok
Jun 12 2015 21:54
i actually think i found a solution
the only thing we have to worry about is not marking them
we can null them out when we begin the unwind. ending the unwind just sets a flag such that we don’t mark them anymore.
a nice flag to have anyway so we can assert we aren’t beginning an unwind twice without an intervening end
Chris Toshok
@toshok
Jun 12 2015 22:05
huh, the protobuf test is crashing for me now too:
/home/travis/build/dropbox/pyston/src/codegen/unwinding.cpp:354: pyston::AST_stmt *pyston::PythonFrameIteratorImpl::getCurrentStatement(): Assertion `0' failed: no frame info found at offset 0x7830 / ip 0x7fa2dfd15830!
Kevin Modzelewski
@kmod
Jun 12 2015 22:06
do you have a backtrace?
oh, sorry that was for a different assert - got the same crash twice
the assert corresponding to that gist is:
unwinding.cpp:354: pyston::AST_stmt *pyston::PythonFrameIteratorImpl::getCurrentStatement(): Assertion `0' failed: no frame info found at offset 0x2f269 / ip 0x7fdc1f940269!
which corresponse to frame #16:
#15 0x00007fdc1f3bfedc in compile_e3_osr161_from_compile_e2_427_551 ()
#16 0x00007fdc1f94026a in compile_e2_427 ()
Kevin Modzelewski
@kmod
Jun 12 2015 22:11
hmm sounds like we should be skipping that frame
since we osr'd out of it
Chris Toshok
@toshok
Jun 12 2015 22:11
yeah
Kevin Modzelewski
@kmod
Jun 12 2015 22:11
(I don't think we add frame introspection info to osr exit points)
Chris Toshok
@toshok
Jun 12 2015 22:12
damn, i see why now. so much for refactoring that code to be nicer :/
Kevin Modzelewski
@kmod
Jun 12 2015 22:38
@rudi-c kmod/pyston@cdbcb74
that was just a quick hack but could serve as a starting point/reference
Rudi Chen
@rudi-c
Jun 12 2015 22:48
Hmm this is quite similar to what I already had w/ a few extra things.
Why was it just a quick hack?
Kevin Modzelewski
@kmod
Jun 12 2015 23:12
oh I didn't spend any time verifying it
Kevin Modzelewski
@kmod
Jun 12 2015 23:23
fyi everyone, we just merged #460 which disables the old makefile build
we are now 100% on cmake :)
(you will also have to do a full rebuild)
Chris Toshok
@toshok
Jun 12 2015 23:24
so we can nuke ../pyston-build-*, everything goes into pyston/build/*?
Kevin Modzelewski
@kmod
Jun 12 2015 23:28
yep!
I'm going to hold off on actually removing the old directories though
Chris Toshok
@toshok
Jun 12 2015 23:29
i jumped in with both feet
hope i didn’t just mess that up :)