These are chat archives for ipython/ipython

10th
Feb 2015
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 03:26
as I am separating out several js nbextensions in different modules, i find myself requiring the same js libraries in nbextensions that are rather independent. Hence I would not want one of them to become dependent on the other one.
A solution would be to create python-less nbextensions only holding those common dependencies.
looking into bower
Nicholas Bollweg
@bollwyvl
Feb 10 2015 15:21
:plus1: bower
having a bower.json for an nbextension would make things far, far shinier
Nicholas Bollweg
@bollwyvl
Feb 10 2015 15:28
jupyter install-nbextension --bower=d3
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 15:30
yes something like this.
Cedric GESTES
@cgestes
Feb 10 2015 15:31
Hi guy, do you kwow if there is a way to inject javascript and css files from an extension into a notebook. I would like it to stay a file, no display(Javascript(readall)).
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 15:31
Still having some duplication for javascript deps at the moment.
Cedric GESTES
@cgestes
Feb 10 2015 15:31
Right now I use install-nbextension, then I deduce the address of the script, and inject it into head.
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 15:31
@ctaf42 what do you mean by "into a notebook" ?
Cedric GESTES
@cgestes
Feb 10 2015 15:33
@SylvainCorlay into the webpage.
What I would like: be able to advertise a static folder (from my python extension), and be able to load script (from python) in a simple way.
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 15:36
The widgets allow you to load a given require path for its views and models, so that the javascript is not loaded with the notebook
Cedric GESTES
@cgestes
Feb 10 2015 15:38
@SylvainCorlay ?
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 15:39
ipython/ipython#6494
Jason Grout
@jasongrout
Feb 10 2015 15:52
how do you join this room? I have a friend who wants to join, but when we logged into Gitter, couldn't find a way to add this room.
Armin Burgmeier
@aburgm
Feb 10 2015 15:54
Jason, it worked eventually; the room showed up after a while
Jason Grout
@jasongrout
Feb 10 2015 15:55
oh, hey Armin. Great
weird.
@takluyver - Armin and I were just discussing event loops and IPython.
I mentioned to him that you were interested in looking at asyncio the other day
(Armin == @aburgm )
Cedric GESTES
@cgestes
Feb 10 2015 15:58
@SylvainCorlay thanks. calling install_nbextension automatically. ok!
Armin Burgmeier
@aburgm
Feb 10 2015 15:59
More specifially, I was looking into integrating custom event loops with ipython, and found the %gui stuff
If there is a way to integrate asyncio with ipython, I could simply integrate mine with asyncio
Jason Grout
@jasongrout
Feb 10 2015 16:01
(@takluyver may not be on now - he's in west coast time, but I'm sure he'll eventually see these messages)
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:13
@jdfreder
When requiring js module for widget views and models :
_view_module = Unicode('nbextensions/entire_path_for_widget', sync=True)
I find it too bad that the entire path has to be specified.
Jason Grout
@jasongrout
Feb 10 2015 16:14
it would be nice if there was a _view_module_base = Unicode('nbextensions/entire/path/', sync=True)
that could be set once in a base class for the module
and then after that, _view_module could be a relative path that could change in each class
This message was deleted
This message was deleted
This message was deleted
(luckily, I can really take back and delete things...)
so _view_module_base=Unicode('nbextensions/mywidget/', sync=True)
_view_module = Unicode('mymodule', sync=True)
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:18
_view_module_base would be defined where?
Jason Grout
@jasongrout
Feb 10 2015 16:18
at the top level
and then inherited
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:18
ok, then all views and models will need to inherit from that guy
Jason Grout
@jasongrout
Feb 10 2015 16:18
or each top-level thing would have to declare it.
but that seems reasonable
in the actual python file, you could make it a private constant at the top, and just use it over and over again
so it really is only specified once
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:19
not sure it is not even more cumbersome that the current mechanism
Jason Grout
@jasongrout
Feb 10 2015 16:20
to me, it seems a little better.
but do you have an idea too?
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:20
no
this is the most reasonable, just that I don't like to have this top-level base class
Jason Grout
@jasongrout
Feb 10 2015 16:21
I guess with the current mechanism, you could still have that private module-level BASE_DIR
and then every class has _view_module = Unicode(BASE_DIR+'mymodule', sync=True)
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:22
yes
Jason Grout
@jasongrout
Feb 10 2015 16:22
actually, that doesn't seem so bad
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:22
I think the module-level thing is better
Jason Grout
@jasongrout
Feb 10 2015 16:22
certainly less complicated
+1 for not complicating things that go over the wire
Sylvain Corlay
@SylvainCorlay
Feb 10 2015 16:24
with the west and east coast time difference, we cover quite a big time range with activity on this gitter chat
we need some people in asia and europe now :)
Jason Grout
@jasongrout
Feb 10 2015 16:38
there were years where we had 24/7 chatting going on the sage chat channels
(but that was also because we stayed up many nights too)
Jonathan Frederic
@jdfreder
Feb 10 2015 17:24
@SylvainCorlay we could just add nbextensions to the requirejs path.
Thomas Kluyver
@takluyver
Feb 10 2015 17:24
@jasongrout @aburgm - cool, that's good to hear. Doing it through the event loop integration stuff is a good idea, I hadn't thought of that. Docs on that are here: http://ipython.org/ipython-doc/dev/config/eventloops.html#event-loops-in-the-kernel
I have an experimental asyncio based kernel
written last Friday
Jonathan Frederic
@jdfreder
Feb 10 2015 17:30
@/all meeting still at 9:30?
Or are we pushing it back b/c @Carreau moved to the states?
Matthias Bussonnier
@Carreau
Feb 10 2015 17:31
10 since I'm here
thomas send a mail to the ML
Jonathan Frederic
@jdfreder
Feb 10 2015 17:31
Ah, the thing I don't check often
Matthias Bussonnier
@Carreau
Feb 10 2015 17:32
:-)
I don't check gitter often
Armin Burgmeier
@aburgm
Feb 10 2015 17:37
@takluyver Thanks! The docs are a bit sparse especially when it comes to completely avoiding some sort of timer-based polling. But I can probably figure it out.
Doing it in the kernel would have the advantage that one could still plug a separate GUI event loop in, I guess
Damian Avila
@damianavila
Feb 10 2015 17:38
weird connection today (had problems with previous meetings)... probably I gonna miss this one... grrr.... I hope the connection gets better in the next minutes... @/all
Armin Burgmeier
@aburgm
Feb 10 2015 17:41
(Even though your experimental code wouldn't allow that yet, right?)
Matthias Bussonnier
@Carreau
Feb 10 2015 17:45
@takluyver are you in Barker ?
Min RK
@minrk
Feb 10 2015 17:46
@Carreau yup, we are here
@jdfreder 10
Jonathan Frederic
@jdfreder
Feb 10 2015 17:47
got it
Matthias Bussonnier
@Carreau
Feb 10 2015 17:47
Ok, I'll be ther in 10 then.
Jonathan Frederic
@jdfreder
Feb 10 2015 17:47
It would be nice to have a Jupyter team Google calendar.
Then I could just add it to my personal calendar.
Min RK
@minrk
Feb 10 2015 17:48
probably
Jonathan Frederic
@jdfreder
Feb 10 2015 17:49
"probably" = we have too many apps already?
Cedric GESTES
@cgestes
Feb 10 2015 17:49
@SylvainCorlay _view_module is working. Do you know if there is the same thing for css?
Min RK
@minrk
Feb 10 2015 17:50
@jdfreder we should probably make a calendar.
Jonathan Frederic
@jdfreder
Feb 10 2015 17:52
We do use a lot of communication tools already: mailing list, github, gitter, google hangout, hackpad, and trello. And that doesn't even count what programs we use for development.
Matthias Bussonnier
@Carreau
Feb 10 2015 17:57
I'll be with Fernando from BIDS.
Jonathan Frederic
@jdfreder
Feb 10 2015 18:01
ping
Min RK
@minrk
Feb 10 2015 18:01
starting up
Cedric GESTES
@cgestes
Feb 10 2015 18:09
@jdfreder can they get registered automatically with a python extension? calling install_nbextension is cumbersome. having to install both python and js extension is too. and if we call install_nbextension from the package, the page has to be reloaded. It would be nice to be able to map some directory on the webserver directly from the code.
Jonathan Frederic
@jdfreder
Feb 10 2015 18:11
@ctaf42 , @takluyver has a good example of how to write and deploy a JS+PY IPython plugin. https://github.com/takluyver/cite2c/
Jason Grout
@jasongrout
Feb 10 2015 18:14
serialization! yeah!
Jason Grout
@jasongrout
Feb 10 2015 18:42
%%sh

echo 'hi'
works in Sandstorm
@Carreau - by the way, please keep me in the loop with RT collaboration
Cedric GESTES
@cgestes
Feb 10 2015 18:47
@jdfreder I know this example, but there is still a need to install the nbextensions and the extensions, would be nice just to install both at the same time. (in fact python setup.py install and that's all). furthermore the python and js side of a widget are kinda linked
@jdfreder do you know how hard it is to implement popup. (I saw a patch with iFrame support but did not get the purpose of it)
Jason Grout
@jasongrout
Feb 10 2015 18:49
how does this framework discussion play in the thoughts of using phosphor?
Jonathan Frederic
@jdfreder
Feb 10 2015 18:50
@ctaf42 my flightwidget repo does it all with pip install . - https://github.com/jdfreder/ipython-flightwidgets/blob/master/setup.py
I don't "enable" my extension by default, because it's lazy loaded with require.js .
But if you look at @takluyver 's code you can see how to enable it by default.
@ctaf42 you could probably implement something that proxied the info to the window with a medium effort, but doing it correctly is a lot of work (like the iframe PR @KesterTong and I had opened a couple of months ago).
@ctaf42 if it's okay to have an in-page popup, you can use jquery to do it easily with the Box widget.
Jason Grout
@jasongrout
Feb 10 2015 18:54
mithril, etc.
Cedric GESTES
@cgestes
Feb 10 2015 18:57
@jdfreder In my case I just installed my python extension. (pip install -e) and I import it into a notebook. I want the code of the widget to be available at that time. (without having to call stuff to do symlink).
@jdfreder dock and popin I made an extension for that. (still very experimental). I would also like to be able to popup to be able to display data on another screen.
Cedric GESTES
@cgestes
Feb 10 2015 19:03
@jdfreder cite2c you are right... I always focused on install.py, I did not see the handler for static data. that's exactly what I'am looking for. Should that be a little more standard? It seems common to all people wanting to have an extension with python and js. (it goes very well along with __view_module).
Jonathan Frederic
@jdfreder
Feb 10 2015 19:04
@ctaf42 you're right, it should be more standard, but it's built on really new code, so it just hasn't had enough exposure.
Thomas Kluyver
@takluyver
Feb 10 2015 19:04
the expectation is that static files, e.g. the JS for _view_module will be served from the nbextensions directory
e.g. have a look at what I do for this turtle widget: https://github.com/takluyver/mobilechelonian
the only reason I need to add a static handler for cite2c is that it's serving something that is updated by another program, so I can't just copy it to nbextensions
Jonathan Frederic
@jdfreder
Feb 10 2015 19:07
@takluyver what do you think about adding the nbextension repository to the require.js path?
Thomas Kluyver
@takluyver
Feb 10 2015 19:08
you can already require e.g. "nbextensions/mobilechelonianjs/turtlewidget"
I don't think it's important to save typing nbextensions in that. It's not something you need to type very often, and explicit beats implicit.
Jonathan Frederic
@jdfreder
Feb 10 2015 19:10
my grandma uses blogger :)
seriously
Thomas Kluyver
@takluyver
Feb 10 2015 19:10
Adding nbextensions to path would mean nbextensions could do crazy overriding of our JS, which I don't think is a good id
idea
Jonathan Frederic
@jdfreder
Feb 10 2015 19:10
Is that true? If it was added to the end of the path?
I think IPython would take a priority, right?
Cyrille Rossant
@rossant
Feb 10 2015 19:11
what are the pros/cons of:
  1. serving static JS/CSS for widgets through _view_module and static handler
  2. use nbextensions
    ?
Jonathan Frederic
@jdfreder
Feb 10 2015 19:12
@rgbkrk I second your CSS statement.
Kyle Kelley
@rgbkrk
Feb 10 2015 19:15
I say you just put a ghost blog up Jon, I'll start posting content.
Cedric GESTES
@cgestes
Feb 10 2015 19:15
@takluyver which mean having to call install_nbextension in the code and to reload the page before it works the first time. I would like to define the handler directly so no need to install the symlink. (I just add a rules to the handler, when my extension activate).
Jonathan Frederic
@jdfreder
Feb 10 2015 19:19
@rossant I don't think I understand your question. The nbextension method can be used either way, via the widget registry or via lazy loading.
Cyrille Rossant
@rossant
Feb 10 2015 19:21
rephrasing: what would be the problems with not putting the JS in nbextension (keeping the JS code in the Python package)?
Thomas Kluyver
@takluyver
Feb 10 2015 19:22
@ctaf42 there's no need to reload the page before it works using IPython 3. Just make sure that install_nbextension is called before the thing is displayed. I do this in mobilechelonian
install_nbextension should be reasonably cheap when the file is already copied, so I just do it on every Turtle() instantiation
Jonathan Frederic
@jdfreder
Feb 10 2015 19:23
@rossant I don't think there's a problem with it at all. A lot of my extensions are still done that way. However, it does mean the user needs to load the extension at runtime (which pushes the JS to the notebook).
It also means that the JS gets persisted to the notebook, which can be annoying at times, and beneficial otherwise.
There are subtleties associated with that, for example, you can't load the JS on Python import. Loading the JS on Python import only loads JS the first time import is called for the kernel. If the user refreshes the page and re-runs the cell with the import call, the JS won't get pushed to the page again.
@takluyver I haven't looked, does the nbextension install method have a symlink option instead of copying the files?
@takluyver that would make it easy for me to support pip install -e for example in https://github.com/jdfreder/ipython-flightwidgets/blob/master/setup.py .
Cedric GESTES
@cgestes
Feb 10 2015 19:27
@takluyver but it looks like a hack if I never load the extension. (it's just used to serve widgets files)
@jdfreder yes it does. symlink=True, user=True (to avoid permission denied)
Jonathan Frederic
@jdfreder
Feb 10 2015 19:29
@ctaf42 thanks, I should implement that in my setup.py.
Thomas Kluyver
@takluyver
Feb 10 2015 19:31
@ctaf42 I don't think that's a hack. The extension code is loaded by displaying the widgets. That seems perfectly OK.
symlink is possible, but if you want to support Windows, best not to rely on it
or do symlink=(os.name=='posix')
symlinks are theoretically possible on Windows, but in practice it's rare to be able to create them
Min RK
@minrk
Feb 10 2015 19:33
I think symlinks aren't so bad post-XP, but I'm not sure.
Thomas Kluyver
@takluyver
Feb 10 2015 19:34
you need post-XP, Python 3, and a permission bit that's normally only given to administrators
the last one is the real killer
Jonathan Frederic
@jdfreder
Feb 10 2015 19:34
I'm just going to use symlink for develop installs, so I'm not too concerned with how well it works, as long as it works for me ;)
Thomas Kluyver
@takluyver
Feb 10 2015 19:35
basically, Windows supports symlinks, but only if you go and configure some obscure option to make it do so
Jonathan Frederic
@jdfreder
Feb 10 2015 19:35
That's the kind of support (lack thereof) you're guaranteed using my extensions. :D
Cedric GESTES
@cgestes
Feb 10 2015 19:35
@takluyver symlink are needed to develop. The current solution with extension is not flexible. nbextension are nice for what they are. But here we are not exactly speaking of a nbextension but more of a python extension with css/js. this case should be easy and straighforward. adding a line to the handler solves all issues. (allowing dev and production mode)
Jonathan Frederic
@jdfreder
Feb 10 2015 19:42
I should write a widget 2.x->3.x migration doc. I think I'm going to spend some time doing that today.
Jonathan Frederic
@jdfreder
Feb 10 2015 20:29
Oh no, unicorns on GitHub
:/
Thomas Kluyver
@takluyver
Feb 10 2015 20:30
:horse: :icecream:
you have to make your own unicorns around here
Jonathan Frederic
@jdfreder
Feb 10 2015 20:49
@SylvainCorlay @jasongrout @minrk @takluyver in the process of figuring out what has changed in the widgets for 3.0, I went through all the widget PRs I could find by hand (I searched for the keyword "widget"), verified that they were related to the widgets, were merged, and targeted for 3.0. Here's the list:
Widget related PRs:
Add placeholder attribute to text widgets ipython/ipython#5652
Add latex support in widget labels, ipython/ipython#5937
Allow widgets to display more than once within container widgets. ipython/ipython#5963
use require.js, ipython/ipython#5980
Range widgets ipython/ipython#6050
Interact on_demand option ipython/ipython#6051
Allow text input on slider widgets ipython/ipython#6106
support binary buffers in comm messages ipython/ipython#6110
Embrace the flexible box model in the widgets ipython/ipython#6125
Widget trait serialization ipython/ipython#6128
Make Container widgets take children as the first positional argument ipython/ipython#6153
once-displayed ipython/ipython#6168
Validate slider value, when limits change ipython/ipython#6171
Unregistering comms in Comm Manager ipython/ipython#6216
Add EventfulList and EventfulDict trait types. ipython/ipython#6228
Remove add/remove_class and set/get_css. ipython/ipython#6235
avoid unregistering widget model twice ipython/ipython#6250
Widget property lock should compare json states, not python states ipython/ipython#6332
Strip the IPYMODEL prefix from widget IDs before referencing them. ipython/ipython#6377
"event" is not defined error in Firefox ipython/ipython#6437
Javascript link ipython/ipython#6454
Bulk update of widget attributes ipython/ipython#6463
Creating a widget registry on the Python side. ipython/ipython#6493
Allow widget views to be loaded from require modules ipython/ipython#6494
Fix Issue #6530 ipython/ipython#6532
Make comm manager (mostly) independent of InteractiveShell ipython/ipython#6540
Add semantic classes to top-level containers for single widgets ipython/ipython#6609
Selection Widgets: forcing 'value' to be in 'values' ipython/ipython#6617
Allow widgets to be constructed from Javascript ipython/ipython#6664
Output widget ipython/ipython#6670
Minor change in widgets.less to fix alignment issue ipython/ipython#6681
Make Selection widgets respect values order. ipython/ipython#6747
Widget persistence API ipython/ipython#6789
Add promises to the widget framework. ipython/ipython#6818
SelectMultiple widget ipython/ipython#6890
Tooltip on toggle button ipython/ipython#6923
Allow empty text box while typing for numeric widgets ipython/ipython#6943
Ignore failure of widget MathJax typesetting ipython/ipython#6948
Refactor the do_diff and manual child view lists into a separate ViewList object ipython/ipython#6990
Add warning to widget namespace import. ipython/ipython#7107
lazy load widgets ipython/ipython#7120
Fix padding of widgets. ipython/ipython#7139
Persist widgets across page refresh ipython/ipython#7163
Make the widget experimental err
This message was deleted
This message was deleted
This message was deleted
Make the widget experimental error a real python warning ipython/ipython#7200
Make the widget error message shorter and more understandable. ipython/ipython#7201
Make the widget warning brief and easy to filter ipython/ipython#7204
Add visual cue for widgets with dead comms ipython/ipython#7227
Widget values as positional arguments ipython/ipython#7260
Remove the popup widget ipython/ipython#7341
document and validate link, dlink ipython/ipython#7468
Document interact 5637 ipython/ipython#7525
Update some broken examples of using widgets ipython/ipython#7547
Use Output widget with Interact ipython/ipython#7554
don't send empty execute_result messages ipython/ipython#7560
Validation on the python side ipython/ipython#7602
only show prompt overlay if there's a prompt ipython/ipython#7661
Allow predictate to be used for comparison in selection widgets ipython/ipython#7674
Fix widget view persistence. ipython/ipython#7680
Revert "Use Output widget with Interact" ipython/ipython#7703
there we go. :)
Jason Grout
@jasongrout
Feb 10 2015 20:52
of course, the big one is the switch to promises, and the related _view_module thing.
Jonathan Frederic
@jdfreder
Feb 10 2015 20:52
Yeah
There's a lot of little API changes too.
I think I'll take a look at the changes required to make one of the example custom widgets work and document that.
It should help me write a good step-by-step guide for easy migration.
Jared Forsyth
@jaredly
Feb 10 2015 20:59
what version angular is required for this lib?
Jonathan Frederic
@jdfreder
Feb 10 2015 20:59
?
None
Jared Forsyth
@jaredly
Feb 10 2015 21:03
hmm it's breaking
pretty much copied over a demo, and there's an error deep in select.js "transclude not defined". angular 1.2
oh sorry. gitter is broken
Armin Burgmeier
@aburgm
Feb 10 2015 21:10
@takluyver I'm trying out the aiokernel with "ipython console --kernel aiokernel", but it just gets stuck at startup without giving me a prompt. Any ideas?
I have modified it slightly to work with Python2 and trollius
Thomas Kluyver
@takluyver
Feb 10 2015 21:35
I've only tried it with the notebook so far
it is like one afternoon's work
Armin Burgmeier
@aburgm
Feb 10 2015 21:38
It's the same problem in the notebook for me
it shows a prompt, but does not execute anything
I understand it's very experimental, was just wondering if you have a spontaneous idea -- I'll investigate myself otherwise
Thomas Kluyver
@takluyver
Feb 10 2015 21:40
hang on, let me just grab the repo on this computer and see if it still works
it's working here for me
Thomas Kluyver
@takluyver
Feb 10 2015 21:46
scatter some debugging prints through stuff, but be careful with the redirection - I got into an infinite loop with incautious debug printing. print(..., file=sys.__stderr__) should print to the regular terminal.
the problem may be in Zantedeschia - Min warned me that the ZMQ fds were tricky to use correctly, but I'm using them anyway.
Armin Burgmeier
@aburgm
Feb 10 2015 21:47
The zantedeschia unit tests all worked fine
Thomas Kluyver
@takluyver
Feb 10 2015 21:52
that's a good start
it's possible that the kernel is exercising some condition that the tests don't cover, though
Armin Burgmeier
@aburgm
Feb 10 2015 22:14
seems like an exception is thrown somewhere when processing incoming messages and silently ignored
probably a python2/3 incompatibility that only pops up at runtime...
Thomas Kluyver
@takluyver
Feb 10 2015 22:18
possibly. I haven't dealt with trollius at all, I have no idea how stable it is
Armin Burgmeier
@aburgm
Feb 10 2015 22:59
works now, except that the output of the executed command is not printed
but looking at the aiokernel code I don't see where that would happen, so I guess it's just not implemented?
Thomas Kluyver
@takluyver
Feb 10 2015 23:07
it should happen
Kernel.enable_stream_capture()
(which is called as part of execute - you shouldn't need to do anything)
Armin Burgmeier
@aburgm
Feb 10 2015 23:09
that captures things that are printed with print(), but if I just type 1+2 in a notebook cell, is it supposed to capture that too?
Thomas Kluyver
@takluyver
Feb 10 2015 23:09
oh, right
no, that's not implemented at present
you have to print() explicitly
Armin Burgmeier
@aburgm
Feb 10 2015 23:09
okay, perfect
Thomas Kluyver
@takluyver
Feb 10 2015 23:09
what was the trouble earlier?
Armin Burgmeier
@aburgm
Feb 10 2015 23:10
Several exceptions in from_wire() that were silently ignored, for example the digestmod parameter of hmac.new is not allowed to be a string in python2
and some confusions between unicode and bytes objects
Thomas Kluyver
@takluyver
Feb 10 2015 23:12
ah, yes
that would happen ;-)
Jonathan Frederic
@jdfreder
Feb 10 2015 23:24
@minrk I've written a small migration doc for widgets and 3.0. I want to post in the wiki so it can be hacked on, but I don't see a good place for it. Should I make one? Or should I convert my markdown to RST and put it in the docs?
Min RK
@minrk
Feb 10 2015 23:24
Hm
One one hand, the wiki is better because it's easier to edit by others
On the other, for version migration, it makes a certain amount of sense for it to be associated with the IPython version
Jonathan Frederic
@jdfreder
Feb 10 2015 23:26
Hehe, or option 3, post it on my blog and no one will ever find out about it.
Thomas Kluyver
@takluyver
Feb 10 2015 23:26
...you have a blog? ;-)
Jonathan Frederic
@jdfreder
Feb 10 2015 23:26
lol ^ that
I'll post it in the wiki and I can remove it and run it through a converter for are docs right before 3.0 if that's what we want. Sounds good?
(in other words, wiki for now)
Thomas Kluyver
@takluyver
Feb 10 2015 23:30
I have been cataloguing what uses what from IPython.utils: https://ipython.hackpad.com/What-components-use-from-utils-ZUxYtQKUHJj
I think there's enough commonality (importstring, path, text, data, io...) that it's worth doing a separate misc utils package rather than trying to stuff them into config
Min RK
@minrk
Feb 10 2015 23:34
@takluyver that makes me sad
I fear we are going to end up with an ETS-style coordinated release of multiple repos if they tangle too much
Jonathan Frederic
@jdfreder
Feb 10 2015 23:35
@takluyver haha, did you build a module dependency graph for IPython? I bet there are a LOT of lines pointing to utils.
@minrk I'm not familiar with ETS-style releases. Do you have a link where I could read about them?
I looked on Goog but a bunch of random, unrelated stuff showed up.
Thomas Kluyver
@takluyver
Feb 10 2015 23:39
No, I looked through this manually. I have a utility somewhere to build a module import graph, though
ETS-style is not a thing
ETS = Enthought Traits Suite
Min RK
@minrk
Feb 10 2015 23:39
not a thing?
oh, yeah, it's not an official pattern, or anything
Thomas Kluyver
@takluyver
Feb 10 2015 23:41
I don't think it will be too bad. Most of the common stuff is fairly stable by this point, so it shouldn't need too much synchronisation, and we can probably disentangle it some more later
Min RK
@minrk
Feb 10 2015 23:41
The Enthought Tool Suite is really one big project that happens to live in multiple repos. Because of the level of coupling between the repos, they do coordinated releases of everything at the same time. It's a huge pain to install as a result.
Thomas Kluyver
@takluyver
Feb 10 2015 23:42
but I think we will go mad if we try to unpick all of those quirky little modules before doing the Big Split :tm:
Min RK
@minrk
Feb 10 2015 23:42
@takluyver true, and there should hopefully be roughly no new code that goes into utils, as all new utility bits will go into project-specific utils as needed.
Thomas Kluyver
@takluyver
Feb 10 2015 23:42
yep
or spun out into little independent packages
Min RK
@minrk
Feb 10 2015 23:42
Yeah, from your list, it makes sense
Thomas Kluyver
@takluyver
Feb 10 2015 23:42
e.g. an ANSI escape preprocessor could be a standalone thing
Min RK
@minrk
Feb 10 2015 23:43
yeah, we could make several standalone packages
Thomas Kluyver
@takluyver
Feb 10 2015 23:44
but I also don't want to sink too much time into writing separate READMEs, putting them on PyPI, redirecting bugs to different trackers, etc.
Min RK
@minrk
Feb 10 2015 23:44
sure
I think there's a good case to be made for a single genutils package
we can do a bit of cleanup for that
e.g. I think the process stuff can go with core, pickleutil with parallel, maybe? Probably pull zmqrelated, etc.
Thomas Kluyver
@takluyver
Feb 10 2015 23:47
there's certainly a few things that are only used in one part
process is used in several, though
although I think some of that was just for the check_pid function
which could be split out on its own
Min RK
@minrk
Feb 10 2015 23:48
yeah. nbconvert uses it to launch subprocesses
but most other things outside core don't, as far as I can tell
hm
tricky
Thomas Kluyver
@takluyver
Feb 10 2015 23:50
For a first pass, we might just stick a lot of modules into genutils, and then we can progressively refine it as we go.
Min RK
@minrk
Feb 10 2015 23:50
yeah
wtf is utils.decorators?
I feel like some of this I've never seen before...
Thomas Kluyver
@takluyver
Feb 10 2015 23:52
yeah, there are a few there that I haven't looked at for ages
Jonathan Frederic
@jdfreder
Feb 10 2015 23:52
At the company I interned at, I worked on a big suite of tools. We too had a utils package (library in Windows speak) which every project depended on.
Min RK
@minrk
Feb 10 2015 23:53
@jdfreder you internet a company?
Jonathan Frederic
@jdfreder
Feb 10 2015 23:53
hehehe
Thomas Kluyver
@takluyver
Feb 10 2015 23:53
that one contains @flag_calls, which I don't think is used anywhere, and @undoc, which is just a name
Jonathan Frederic
@jdfreder
Feb 10 2015 23:53
@minrk you are quick to respond!
So much for my ninja edit...
Thomas Kluyver
@takluyver
Feb 10 2015 23:53
ah, no, pylabtools uses flag_calls
"We need to detect at runtime whether show() is called by the user. For this, we wrap it into a decorator which adds a 'called' flag."
Min RK
@minrk
Feb 10 2015 23:54
and undoc is used by sphinx autodoc?
Thomas Kluyver
@takluyver
Feb 10 2015 23:55
yep
which just looks for a decorator called undoc
so it doesn't need to do anything