These are chat archives for ipython/ipython

13th
Feb 2015
Jason Grout
@jasongrout
Feb 13 2015 01:07
@jdfreder - thanks for your comments. I'll look at them tomorrow. Today I was concentrating on getting a minimum viable PR :).
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 02:05
@minrk I have a simple PR fixing the default value problem in the traitlets. (I am procrastinating on a lecture I must prepare for tomorrow night)
Min RK
@minrk
Feb 13 2015 02:30
@SylvainCorlay great.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:15
marked as 4.0?
Min RK
@minrk
Feb 13 2015 03:27
If it's a simple fix (and ready to go for the RC tomorrow), it should be fine for 3.0. If not, we can hold it for 3.1 or maybe 4.0, depending on the change.
@SylvainCorlay is #7770 the fix you are referring to? Since that's a relatively significant API change (removes instance_init entirely), I think we should hold it. Since it's a backward-incompatible change, it wouldn't be a candidate for 3.1.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:31
yes, it is the fix i was referring to
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:38
I have a bunch of traitlet related stuff:
1 - Default value fix #7770
2 - Parameterized trait dict #7726
3 - allow_none=False for all trait types but Instance and Type #7708
4 - early validation hook #7603
  1. is the only one to be a bug fix.
Min RK
@minrk
Feb 13 2015 03:39
If it were smaller, I'd definitely want it in 3.0. The default value that's unused has bothered me for a long time. But the refactor necessary doesn't seem appropriate hours before the release candidate.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:39
hours?
ok, did not know it was hours
Min RK
@minrk
Feb 13 2015 03:39
We will cut the release candidate tomorrow, less than 24 hours from now.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:40
Congrats! I should have woken up earlier on all that, but I was busy with non-opensource things.
Min RK
@minrk
Feb 13 2015 03:41
At which point we won't touch 3.0 any more other than docs and important bugfixes.
Thanks! Part of the reason we are pushing to get it out the door is so we can get back to accepting improvements like the ones you keep making.
The plan is RC tomorrow, docs polishing for the next two weeks, and (assuming all goes well) 3.0 final in two weeks.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:43
alright, hopes this gets in before the mess of the split of the repos,
Min RK
@minrk
Feb 13 2015 03:45
Yeah, we are going to push against new PRs while we are in the middle of splitting, but it would just be mean to apply that to PRs opened before we even shipped 3.0.
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 03:46
ok, thanks for the explanation. I've got yo prepare my course now that I am really really late.
ciao
Remi Rampin
@remram44
Feb 13 2015 14:48
How does the rich display protocol works? Can I delay the choice of the display method until IPython actually needs to display it?
the best I can come up with is create the correct DisplayObject subclass when _repr_html_ is first called and pass the result of its _repr_html_ from there on
Scott Sanderson
@ssanderson
Feb 13 2015 14:55
@remram afaik, the way it works is that, when it needs to display your object (either because you explicitly call a display method or because your object is the output of a cell) ipython will call all your object's repr_* methods and send the results back to the frontend, which then decides which to actually show
Remi Rampin
@remram44
Feb 13 2015 14:56
the thing is that the DisplayObject subclasses are very convenient but I can't really use them
it would make sense to have one more level of indirection, so that my object can build and return Image, SVG, ... and then IPython can call _repr_*_ on that
Scott Sanderson
@ssanderson
Feb 13 2015 14:58
Your object doesn't have to subclass DisplayObject
I'm pretty sure you can just implement whatever repr methods you want to be used
Remi Rampin
@remram44
Feb 13 2015 14:59
it's the only convenient way to go if you want to use all the nice logic that's in IPython.display
you have to subclass one of them, but that's very static
Scott Sanderson
@ssanderson
Feb 13 2015 15:01
Hrm. I'm surprised by that. Because, for example, pandas DataFrames have a nice repr_html, and they're definitely not subclassing DisplayObject
What is the extra functionality that IPython.display buys you over implementing a repr method directly?
Remi Rampin
@remram44
Feb 13 2015 15:08
that's because a panda DataFrame object is always rendered in the same way
I have a class that might get rendered as an image, text, or something else, and I don't know that when I create the object
stuff will happen to the object which will change how it is supposed to get rendered
Remi Rampin
@remram44
Feb 13 2015 15:14
my point is that people are unlikely to introduce new display methods for their objects, but very likely to have objects that represent to one of the already provided ones (Image, TextDisplay, JSON)
Scott Sanderson
@ssanderson
Feb 13 2015 15:15
ah
have you looked at _ipython_display_?
if your object has that method, then it overrides IPython's built-in logic for figuring out how to show your object
@minrk did some work to make that easier to use in ipython/ipython#6919 as well
that lets you register a function to be called on your object when IPython needs to display it
iirc
it was prompted by quantopian/qgrid#8 for github.com/quantopian/qgrid
Remi Rampin
@remram44
Feb 13 2015 15:18
is there doc for _ipython_display_ anywhere?
Scott Sanderson
@ssanderson
Feb 13 2015 15:18
(we haven't gotten around to actually using the machinery for it yet though; hopefully will when 3.0 stable is released
Remi Rampin
@remram44
Feb 13 2015 15:18
I don't know what it does
has some stuff on it
idk if there's a more canonical source
Remi Rampin
@remram44
Feb 13 2015 15:19
(unrelated: http://nbviewer.ipython.org/github/VisTrails/VisTrails/blob/new-api_/test/vtapitest.ipynb 404; is that because there is a slash in the branch name?)
Scott Sanderson
@ssanderson
Feb 13 2015 15:20
no idea
but that's a good guess
you might open an issue in nbviewer about that
Remi Rampin
@remram44
Feb 13 2015 15:21
will confirm and open an issue
Jason Grout
@jasongrout
Feb 13 2015 19:35
@minrk - I think there is a race condition in binary buffers in iopub messages
I'm definitely seeing one -- a binary message is sent before a json message, but the json message handler is called first
I think it has to do with the FileReader() call when deserializing binary messages -- it introduces asynchronicity in the process, which allows the json message to get parsed and through first
Min RK
@minrk
Feb 13 2015 19:37
Ah, the deserialization callback probably does it
yeah
grumble
Jason Grout
@jasongrout
Feb 13 2015 19:37
nasty race conditions
so we can introduce a promise that enforces an order
Min RK
@minrk
Feb 13 2015 19:38
gross
Jason Grout
@jasongrout
Feb 13 2015 19:38
(similar to the state change promise we keep in widgets)
any message gets queued up at the end of the promise .then() chain
Min RK
@minrk
Feb 13 2015 19:38
Really sucks that FileReader has no sync API
Jason Grout
@jasongrout
Feb 13 2015 19:39
yeah, well, we're all supposed to be async here, right?
Min RK
@minrk
Feb 13 2015 19:40
except we aren't - if we are using promises to serialize callbacks, it's not async at all, just reimplementing sync with async APIs
a sync API would just be better
Jason Grout
@jasongrout
Feb 13 2015 19:41
yeah
Fernando Perez
@fperez
Feb 13 2015 19:41
Hey @minrk, have you tested running the nb server (ipython notebook) under py2, but starting a py3 notebook?
I'm getting an error, and before I go debugging my own setup, I want to make sure that works for you guys...
Jason Grout
@jasongrout
Feb 13 2015 19:42
you can do it using webworkers, apparently: FileReaderSync
Fernando Perez
@fperez
Feb 13 2015 19:42
My py3 kernel is picking up py2 libraries. It may be a setup issue on my side, so I just want to make sure it works for you...
Min RK
@minrk
Feb 13 2015 19:42
Yes, but not in a few days. I'll check shortly.
Fernando Perez
@fperez
Feb 13 2015 19:42
ok, tx
Min RK
@minrk
Feb 13 2015 19:42
Do you have PYTHONPATH?
Fernando Perez
@fperez
Feb 13 2015 19:42
I'm running from master as of now.
I do, that may be it.
that's why I want to make sure it's ok on your side. If so, no worries.
But this close to the RC, I want to double check.
Min RK
@minrk
Feb 13 2015 19:43
yeah, PYTHONPATH should cause a problem for multiple Python versions in the same env
Fernando Perez
@fperez
Feb 13 2015 19:43
yup
Jason Grout
@jasongrout
Feb 13 2015 19:44
so you can have "synchronous I/O" only on an asynchronous thread
Fernando Perez
@fperez
Feb 13 2015 19:44
as long as it works for you, I'm good.
Jason Grout
@jasongrout
Feb 13 2015 19:46
@minrk - we want synchronicity for only one part of the program, but we want other parts to proceed.
it seems we are having this pattern a lot - it would be great if we could somehow abstract it out.
Min RK
@minrk
Feb 13 2015 19:47
yup, if only Javasript weren't garbage.
Jason Grout
@jasongrout
Feb 13 2015 19:47
well, we'd have the same problem in other languages
but it would be nice if javascript was better too :)
do you want to take a crack at this, or I can...
Min RK
@minrk
Feb 13 2015 19:49
True, but Javascript offers no real synchronization tools, and async as the only option combines to make these things as hard as possible
So a lot of it really is the language's fault
Jason Grout
@jasongrout
Feb 13 2015 19:49
what would you love to have? generators and yield?
Min RK
@minrk
Feb 13 2015 19:50
A coroutine style where we could express the entire message handler as something to complete before processing the next message.
Honestly, I don't think we need any yields inside the websocket message handler stack at all
We would be 100% fine if that were a single, blocking, callback chain.
Jason Grout
@jasongrout
Feb 13 2015 19:52
so back to the ditches here -- we can force all message handling to proceed linearly, or we can try to just enforce parts to be linear.
I don't think we want to make the main handling linear - then shell messages can't preempt other messages
I think it would probably be best to make each channel linear
what if all of the message handling was done in a web worker?
then it could be synchronous
Jason Grout
@jasongrout
Feb 13 2015 20:00
you can even do zero-copy transfers of objects between the main thread and the worker: http://www.w3.org/html/wg/drafts/html/master/infrastructure.html#transferable-objects
Min RK
@minrk
Feb 13 2015 20:01
I'll see if I can get a simple fix. I'll be cutting 3.0rc today, but we can work on this one for 3.1.
do you have a preference for processing messages in a web worker, vs. enforcing synchronicity at the channel level via promises?
Min RK
@minrk
Feb 13 2015 20:03
Since iopub messages were already fully synchronous callbacks before, if I can get the binary serialization to get back to that, I have no problem with a full iopub message blocking everything
That's what I want
Jason Grout
@jasongrout
Feb 13 2015 20:04
you want it to Just Work (TM)
Min RK
@minrk
Feb 13 2015 20:04
I'm not sure, yet. My main priority on it is simplicity.
Jason Grout
@jasongrout
Feb 13 2015 20:04
I think promises are simpler in this case
I'll see what I can do.
Min RK
@minrk
Feb 13 2015 20:04
I'm not remotely concerned about the performance issue for processing events while deserializing a binary message
Jason Grout
@jasongrout
Feb 13 2015 20:04
since you're busy with RC
Min RK
@minrk
Feb 13 2015 20:05
If there were an async: false for FileReader, that would be the right answer, no question.
it's simply awful that it isn't an option.
Great, thanks
Jason Grout
@jasongrout
Feb 13 2015 20:05
it is - but only in async workers :)
Matthias Bussonnier
@Carreau
Feb 13 2015 20:06
Guys, Going to lunch ?
Min RK
@minrk
Feb 13 2015 20:06
Thomas just set out, I'm skipping today
Matthias Bussonnier
@Carreau
Feb 13 2015 20:06
ok.
Jonathan Frederic
@jdfreder
Feb 13 2015 20:14
just read the bit about the race condition
bummer
:(
Jason Grout
@jasongrout
Feb 13 2015 20:31
@jdfreder - yeah, it was messing up my binary state sync messages
Min RK
@minrk
Feb 13 2015 20:47
@/all anything that should be in the 3.0rc that's not in master?
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 20:49
all our pull requests ;)
Min RK
@minrk
Feb 13 2015 20:50
ha
Sylvain Corlay
@SylvainCorlay
Feb 13 2015 20:50
nothing comes to my mind.
Jonathan Frederic
@jdfreder
Feb 13 2015 21:04
:shipit:
Fernando Perez
@fperez
Feb 13 2015 23:16
Hey, one small thing I'm seeing with the terminals in current master (it's not a new issue, I'd seen it before)...
Try opening an existing file with emacs in the terminal (using emacs -nw). At first, the file content isn't shown, only an empty window. And if you touch an arrow key, the window does refresh with the file contents, but a spurious character is inserted (C for a right arrow, for example). After that it behaves normally, but it's kind of confusing.
I see the problem with emacs, not with vi nor jed.
I can't replicate it on tmpnb b/c they don't have emacs installed.
Min RK
@minrk
Feb 13 2015 23:20
I see the full content here
Fernando Perez
@fperez
Feb 13 2015 23:20
Not a critical problem, and likely more in terminado/termjs than IPython. But nice to have it fixed as we start advertising...
Mmh. What OS, on your mac or on a linux vm?
Min RK
@minrk
Feb 13 2015 23:20
OS X
Yeah, it's almost certainly an issue in an upstream package
Fernando Perez
@fperez
Feb 13 2015 23:21
as long as it's only visible with emacs, it's not too bad.
would be nice to track down eventually. Just want to make sure you're convinced it's not us (ipython) dropping a character along the way at just the right time.
We don't filter control characters or anything like that?
I'm sure that at startup, emacs probably sends weird stuff on the wire to the terminal to set up the "window" (i.e. the full-screen terminal app).
So I wanted to make sure we're not dropping any of that info by accident.
Min RK
@minrk
Feb 13 2015 23:26
I would be very surprised for something like that to be an IPython issue.
There's very little IPython code in the terminals.
We don't handle keyboard events, or anything.
Min RK
@minrk
Feb 13 2015 23:37
@/all shipped rc1
Now we focus on docs!