These are chat archives for dropbox/pyston

29th
Sep 2015
Marius Wachtler
@undingen
Sep 29 2015 09:51
@corona10 I saw your PR about the ellipsis object :-) and thought because kmod is afk that I will give some feedback so that you don't have to wait. hope you don't mind.
We have currently 3 different tiers. interpreter, baseline jit and llvm tier. Your patch teaches the ast interpreter how to handle the ellipsis object :-) But the other tiers don't yet know about it. (that's why you get a error when running pyston using the -n / -O option which disables the interpreter and executes everything inside the llvm tier.
Marius Wachtler
@undingen
Sep 29 2015 09:56
by changing the ASTInterpreter::visit_ellipsis function to return Value(Ellipsis, jit ? jit->imm(Ellipsis) : NULL); the baseline jit should work too. but than the llvm tier is still missing.
Sun
@Daetalus
Sep 29 2015 09:59
Ah, greate information for me too! Thanks @undingen !
Marius Wachtler
@undingen
Sep 29 2015 09:59
To fix the llvm tier you have to adopt src/codegen/irgen/irgenerator.cpp it's best to look how we handle the none object getNone() to get a idea what has to be done
if any questions pop up just ask me I will be around
Sun
@Daetalus
Sep 29 2015 10:02
@undingen Would you mind to take a look #907 ? clang failed, but gcc passed. Or would you mind to tell me how to reproduce the problem on local machine?
Marius Wachtler
@undingen
Sep 29 2015 10:04
oh thats interesting that's very similar to the issue I see with #937 which I can't reproduce locally.
Dong-hee Na
@corona10
Sep 29 2015 10:13
oh my.. thanks @undingen
i found out why it can not passed pyston_force_llvm_tests ,so i guessed 'is this problem about jit-emmiting??' at that time you give me lots of tip for this problem really thank you
Marius Wachtler
@undingen
Sep 29 2015 10:18
no problem, just ask if you have questions
Dong-hee Na
@corona10
Sep 29 2015 11:59
@undingen thanks! i make it works on other tiers. can you review it?
Marius Wachtler
@undingen
Sep 29 2015 12:01
looks good to me, but kmod has more knowledge around the irgen area.
Marius Wachtler
@undingen
Sep 29 2015 12:19
I think you also have to change type_analysis.cpp
otherwise the type analysis will return UNKNOWN but irgen will return typeFromClass(type_cls)
for the Ellipsis type
Marius Wachtler
@undingen
Sep 29 2015 15:47
this clang-3.5 not found error is starting to get really annoying - for most time of today the travis-ci builds are failing again :-(
If this happens more often I would prefer to switch back to clang 3.4 because it looks like all travis ci instances have this package by default so we don't have to install it
Sun
@Daetalus
Sep 29 2015 15:50
Does there has compatibility issue between 3.4 and 3.5?
Marius Wachtler
@undingen
Sep 29 2015 15:55
I don't think so because afaik I had fairly recently used clang 3.4
and our build instructions too mention 3.4
Sun
@Daetalus
Sep 29 2015 16:25
The annoying ctype issue passed. in #937
Marius Wachtler
@undingen
Sep 29 2015 16:56
yes but the only thing I changed is that I set CFLAGS = -g -O0 :-(
Marius Wachtler
@undingen
Sep 29 2015 17:40
with CFLAGS = -g -O3 it fails
Marius Wachtler
@undingen
Sep 29 2015 18:07
@Daetalus @aisk as suspected the #942 build also encountered the ctypes crash
Sun
@Daetalus
Sep 29 2015 18:08
I see. Have any ideas about what caused it?
Marius Wachtler
@undingen
Sep 29 2015 18:11
no I tried today several stuff but could not track it down :-(, I also installed a ubuntu 12.04 VM with the same compiler major versions as the travis ci container has but could not trigger it.
the gdb backtrace says it crashes inside the p->obj = NULL line inside
PyCArgObject *
PyCArgObject_new(void)
{
    PyCArgObject *p;
    p = PyObject_New(PyCArgObject, &PyCArg_Type);
    if (p == NULL)
        return NULL;
    p->pffi_type = NULL;
    p->tag = '\0';
    p->obj = NULL;
    memset(&p->value, 0, sizeof(p->value));
    return p;
}
which would be really strange but the line number may not be correct because I had to compile the file with optimization enabled to trigger the crash
Kevin Modzelewski
@kmod
Sep 29 2015 21:17
I'm starting to really dislike our conservative GC :/
if we are able to switch away from it hopefully these non-reproducible bugs will go away
Sun
@Daetalus
Sep 29 2015 21:18
Want to use a new type GC?
Kevin Modzelewski
@kmod
Sep 29 2015 21:18
since we won't be so susceptible to stack layout and what not
it'd be nice to use precise stack scanning
though that is hard to get working with C extensions
another option is to switch back to refcounting
Marius Wachtler
@undingen
Sep 29 2015 21:23
I think there is also the problem that we will have to switch to a more sophisticated incremental GC at some point because of the large unpredictables pauses which could happen with our current gc
Kevin Modzelewski
@kmod
Sep 29 2015 21:23
marius do you have a link to one of the builds that ran into that unreproducible failure?
Marius Wachtler
@undingen
Sep 29 2015 21:24
for example: https://travis-ci.org/dropbox/pyston/jobs/82784060 which is the #942 change
this one uses the multiprocessing module but just by using your ctypes fix + test it can also be triggered on travis ci
I#m talking about this one: Daetalus/pyston@746335a
Marius Wachtler
@undingen
Sep 29 2015 21:30
oh I just saw that you merged my change so now one only needs to remove the skip-if line inside multiprocessing_ctypes_test.py test
Sun
@Daetalus
Sep 29 2015 21:37
Kevin, I think I need to skip this week's Pyston notes(Nothing for Pyston in last week). But I will be back in next week.