These are chat archives for ipython/ipython

9th
Feb 2015
Jonathan Frederic
@jdfreder
Feb 09 2015 17:53
@minrk or @takluyver are we marking qtconsole issues as 3.0, or should I mark them as 3.1 or 4.0? We are still experimenting with a hands off approach to the qtconsole right?
Min RK
@minrk
Feb 09 2015 18:02
@jdfreder depends on the issue.
Jason Grout
@jasongrout
Feb 09 2015 19:45
@jdfreder - did you ever look at using @minrk's binary buffers to transfer data over comms or widget syncs?
Jonathan Frederic
@jdfreder
Feb 09 2015 19:45
No I haven't, have you?
It seems like it would work well for the image widget.
Jason Grout
@jasongrout
Feb 09 2015 19:46
I think I thought about it, but I'm looking at it now.
just trying to remember how it worked...
yes, it would be ideal for an image widget, for example.
Jonathan Frederic
@jdfreder
Feb 09 2015 19:46
I was looking at a diff with the changes just the other day...
I forget why though.
Let me see if there's an issue open for the image widget sync. If not, I'll open one for 4.0.
It seems ridiculous to b64 the contents over while we have binary support. lol
Jason Grout
@jasongrout
Feb 09 2015 19:47
interestingly, if you have to transfer a list of numbers, using compact json is maybe about 74% growth in size. It's much better to Base64 encode it (33% growth).
of course, much better still to transfer it using binary buffers
ipython/ipython#6110
Perhaps if a widget traitlet is declared (..., sync=True, binary=True), it could be transferred using the binary buffer
or something
Jonathan Frederic
@jdfreder
Feb 09 2015 19:51
ipython/ipython#7722
Jason Grout
@jasongrout
Feb 09 2015 19:53
if binary=True in the traitlet metadata, the to_json and from_json can convert to/from binary buffers instead of to/from json
Jonathan Frederic
@jdfreder
Feb 09 2015 19:55
That sounds like a nice way to do it.
Tom Augspurger
@TomAugspurger
Feb 09 2015 20:01
debug
bah, sorry. Wrong window
Thomas Kluyver
@takluyver
Feb 09 2015 20:02
I think I've typed ls in here before ;-)
Jonathan Frederic
@jdfreder
Feb 09 2015 20:04
I seem to like typing cd.. in here
I think I've done it a few times now
jasongrout @jasongrout thinks we should make a bot for these sorts of things
Jonathan Frederic
@jdfreder
Feb 09 2015 20:06
hmmm
was the little GitHub activity list removed from gitter or just this room?
Or is it something I've done? I can't seem to figure out how to get it to display again...
Thomas Kluyver
@takluyver
Feb 09 2015 20:07
I still see it ->
Jonathan Frederic
@jdfreder
Feb 09 2015 20:08
:/
jdfreder @jdfreder sighs
Thomas Kluyver
@takluyver
Feb 09 2015 20:08
maybe it's size dependent? Try enlarging the window.
Jonathan Frederic
@jdfreder
Feb 09 2015 20:08
I did, no luck
O
spoke too soon
it only shows at 95% of my screen width
Thomas Kluyver
@takluyver
Feb 09 2015 20:11
aha
yeah, it needs about 2/3 of the screen I usually have the browser fullscreen on
Jon Wilson
@jsw-fnal
Feb 09 2015 20:18
I actually had a colleague type his login password in a chat window once.
(speaking of typing "ls" etc)
Jason Grout
@jasongrout
Feb 09 2015 21:05
@jdfreder: which do you prefer?
(..., sync=True, binary=True)
(..., sync='binary')
Jason Grout
@jasongrout
Feb 09 2015 22:09
@jdfreder - this change is getting more and more comprehensive. It's leading to finally cleaning up the wart of the unpack_models stuff
basically, because we need to have some way of telling the javascript how to deserialize a binary blob
so you might as well tell the javascript how to deserialize any field.
so you might as well tell the javascript explicitly what fields might contain a model reference, which should be checked on serialization/deserialization
so you might as well have typed fields for the models, so that the javascript knows about the types too.
Min RK
@minrk
Feb 09 2015 22:11
@jasongrout and it's okay that IPython is only handing you dumb DataViews?
you can do the deserialize bit at the Widget implementation level?
Jason Grout
@jasongrout
Feb 09 2015 22:12
yes, these are all changes to Widget
Min RK
@minrk
Feb 09 2015 22:12
awesome
Jason Grout
@jasongrout
Feb 09 2015 22:12
I just have to pass some metadata telling it what keys the list of buffers correspond to
so in js, I just need to have smarter widget models that know how to interpret the ArrayBuffers.
Min RK
@minrk
Feb 09 2015 22:23
That's excellent. I'm always concerned that we've missed something important when we add a feature and people don't start using it immediately.
Jonathan Frederic
@jdfreder
Feb 09 2015 22:23
@jasongrout sorry I was afk. @minrk yeah, the deserialization and serialization pieces that he's talking about sit in the widget framework.
oh woops
looks like my gitter wasn't updating completely
hate it when that happens...
@jasongrout that all sound good to me. What's the overhead look like for something like that?
Are you using type names stored as their string representation?
Or do you have an enumeration that maps the different types?
I'm curious to see how you implement this.
Jason Grout
@jasongrout
Feb 09 2015 22:33
@minrk - I haven't finished yet, though...
it may be that lots of times, we immediately get the arraybuffer from the dataview and create a typed array buffer.
Min RK
@minrk
Feb 09 2015 22:34
makes sense
Jason Grout
@jasongrout
Feb 09 2015 22:40
@jdfreder - yeah, I'm curious how I'll implement it too
@jdfreder - I think initially, the most general way to do it is just to have serialization/deserialization functions sitting in the main widget model
that can be overridden if you subclass the model
every widget field is passed through those functions, and you are responsible for providing a js model that can make sense of your fields.
Jonathan Frederic
@jdfreder
Feb 09 2015 22:43
Is every value just going to be a key value pair? [type, value]
Jason Grout
@jasongrout
Feb 09 2015 22:43
no, not dynamically typed
rather, statically typed -- the types are carried in metadata, and are set at creation
and json types are automatically handled anyway
the only modification we have right now is to tell which fields possibly have model references
well, we have (a) generic json fields, (b) json fields that can have model references, and (c) binary fields
Jonathan Frederic
@jdfreder
Feb 09 2015 22:47

@jasongrout so you're thinking of something like this:

{
  metadata: {
    models: ['B', 'C.D']
  }
  data: {
    A: 'text',
    B: 'A GUID',
    C: {
      D: 'A GUID'
    }
  }
}

?

Jason Grout
@jasongrout
Feb 09 2015 22:48
{
  data: {
 models: ['B', 'C'],
binary: ['E']
state: { 
   A: 'text',
    B: 'modelid12389457129yrfg8',
    C: {
      D: 'modelid9f82h354f2374h43'
    }
  }
}
}
Jonathan Frederic
@jdfreder
Feb 09 2015 22:48
I was talking about this, statically typed all of the time:
{
    A: ['string', 'text'],
    B: ['model', 'A GUID'],
    C: {
      D: ['model', 'A GUID']
    }
  }
Jason Grout
@jasongrout
Feb 09 2015 22:49
that's dynamic typing -- the values are tagged with the types
actually, I was thinking more like: the comm_open message says that 'C' should be processed with the pack/unpack models serialization.
Jonathan Frederic
@jdfreder
Feb 09 2015 22:50
Ah I see what you mean now!
Jason Grout
@jasongrout
Feb 09 2015 22:50
I don't want to support C: {D: ['model', 'GUID']}
I'm not trying to reinvent all of typing theory here -- no nested types
Jonathan Frederic
@jdfreder
Feb 09 2015 22:51
:P
Jason Grout
@jasongrout
Feb 09 2015 22:52
I'm just trying to avoid having the pack models logic go through every single thing we ever send
instead, only use it on stuff the user has notified could contain models
so another option would be to declare that a certain widget field should have these serialization things applied: ['model', 'some_other_filter', 'etc.']
The js side gets that data once, and looks up and applies those functions.
to those fields
Jonathan Frederic
@jdfreder
Feb 09 2015 22:55
Yeah, I can see how that would be useful.
Not supporting nesting though... I guess like you say "premature optimization is the root of all evil".
Jason Grout
@jasongrout
Feb 09 2015 22:56
really, I guess we're just trying to avoid subclassing the widget model by specifying js serialization on the python side
maybe we should just be more free about subclassing the widget model. If you have a widget that has needs to do some special serialization on the javascript side, you should subclass the widget model and do it.
I remember when we implemented widgets, I realized that we were never doing anything with the model, so really everybody could basically have the same model
but now working around that is making things awkward
so the cleanest solution would have a separate model for each python class, that knows how to serialize/deserialize each field appropriately
separate js model class
Jonathan Frederic
@jdfreder
Feb 09 2015 23:00
Yes, where the dict and list types would be recursive.
Jason Grout
@jasongrout
Feb 09 2015 23:00
so basically you have to specify your serialization/deserialization twice then - on python and js.
okay, I need to run, but I'll think about this and be back tomorrow.
Jonathan Frederic
@jdfreder
Feb 09 2015 23:01
Ok take care