These are chat archives for ipython/ipython

15th
Jun 2015
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 02:24
PR opened.
Matthias Bussonnier
@Carreau
Jun 15 2015 05:07
@fperez ?
Fernando Perez
@fperez
Jun 15 2015 05:09
hey
@Carreau?
Matthias Bussonnier
@Carreau
Jun 15 2015 05:10
Yep ?
Fernando Perez
@fperez
Jun 15 2015 05:10
thoughts?
I can just reinstall the whole thing...
Matthias Bussonnier
@Carreau
Jun 15 2015 05:10
Are component availlable in the conda install repo ?
Sure you can try ...
it might be 2 install conflicting...
Fernando Perez
@fperez
Jun 15 2015 05:11
it's odd...
which component is it?
what I don't get is that I had run it before...
Matthias Bussonnier
@Carreau
Jun 15 2015 05:11
python setup.py submodule
look in html/notebook/static/components
Fernando Perez
@fperez
Jun 15 2015 05:12
no static/
at all
Matthias Bussonnier
@Carreau
Jun 15 2015 05:12
??
that's not normal.
Fernando Perez
@fperez
Jun 15 2015 05:12
that was a conda install of 3.1...
weird.
argh, I really hate this shit.
we shouldn't be debugging installation problems in this time...
Matthias Bussonnier
@Carreau
Jun 15 2015 05:16
./IPython/html/static/components/jquery
./IPython/html/static/components/jquery/jquery-migrate.js
./IPython/html/static/components/jquery/jquery-migrate.min.js
./IPython/html/static/components/jquery/jquery.js
./IPython/html/static/components/jquery/jquery.min.js
./IPython/html/static/components/jquery/jquery.min.map
./IPython/html/static/components/jquery-ui
./IPython/html/static/components/jquery-ui/themes/smoothness/jquery-ui.css
./IPython/html/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
./IPython/html/static/components/jquery-ui/themes/smoothness/jquery.ui.theme.css
./IPython/html/static/components/jquery-ui/ui/i18n/jquery-ui-i18n.js
./IPython/html/static/components/jquery-ui/ui/jquery-ui.js
./IPython/html/static/components/jquery-ui/ui/minified/i18n/jquery-ui-i18n.min.js
./IPython/html/static/components/jquery-ui/ui/minified/jquery-ui.min.js
this are the files I have.
Fernando Perez
@fperez
Jun 15 2015 05:17
I know, no idea why it's empty...
I'm going to try and create a new conda env...
Matthias Bussonnier
@Carreau
Jun 15 2015 05:17
worse case you pip-install in your conda env.
Conda might do smth special....
Fernando Perez
@fperez
Jun 15 2015 05:20
Fresh reinstall, same problem again!
Matthias Bussonnier
@Carreau
Jun 15 2015 05:21
:-(
$ mdfind jquery.min.js ?
or even find . -name
Fernando Perez
@fperez
Jun 15 2015 05:24
wow, I try to install from source from the 3.1 tree, and I get this weird error now:
  File "/Users/fperez/ipython/ipython/setupbase.py", line 767, in run
    command.run(self)
  File "/Users/fperez/ipython/ipython/setupbase.py", line 231, in run
    check_package_data(self.package_data)
  File "/Users/fperez/ipython/ipython/setupbase.py", line 221, in check_package_data
    assert os.path.exists(path), "Missing package data: %s" % path
AssertionError: Missing package data: IPython/html/static/components/backbone/backbone-min.js
Should I try with pip?
Matthias Bussonnier
@Carreau
Jun 15 2015 05:27
Yeah I woudl pip.
I actually don't conda at all...
Fernando Perez
@fperez
Jun 15 2015 05:27
No luck, same error with that backbone message...
that missing backbone-min.js problem.
Wow, I am basically in a situation where I can't run anything.
nothing works.
Matthias Bussonnier
@Carreau
Jun 15 2015 05:29
Just clone the component repo manually for this time, we can debug that wednesday.
or tuesday..
Could it be writing permission error ?
Fernando Perez
@fperez
Jun 15 2015 05:32
ah, no, it was that I hadn't run git submodule update when I switched branches in the source tree to 3.1...
argh, always git submodules, always ;)
Matthias Bussonnier
@Carreau
Jun 15 2015 05:32
It's gone in 4 :-)
Fernando Perez
@fperez
Jun 15 2015 05:34
ok, got it working... there's evidently a problem with conda somewhere...
very odd.
thanks for the help!
go get some sleep :)
Matthias Bussonnier
@Carreau
Jun 15 2015 05:34
Conda might not work well with local dev install...
Fernando Perez
@fperez
Jun 15 2015 05:34
much appreciated.
well, I wasn't trying to do a dev install1
!
just a normal conda isntall
but it didn't work.
Matthias Bussonnier
@Carreau
Jun 15 2015 05:34
night then !
Fernando Perez
@fperez
Jun 15 2015 05:35
I resorted to a dev install b/c the normal conda one didn't work!
I'm truly baffled.
Brian E. Granger
@ellisonbg
Jun 15 2015 16:11
You have hit the fperez conditionals again :)
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 16:30
@takluyver is this also fixing the issue with cross validation?
nevermind. Should not have any effect. I was wondering since constructing IntSlider does not work.
Thomas Kluyver
@takluyver
Jun 15 2015 16:33
Yeah, I don't know of that issue; I was aiming for a change that has no effect. Did my last PR break something?
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 16:33
yeah, i commented on it after it was merged
it breaks the widgets
you cannot instantiate a slider
because of the cross validation.
cel4
@cel4
Jun 15 2015 16:40
Very general question: Are there any plans to make notebooks storing more information, e.g the full history of what changed at what point? I am starting to think that json notebooks as they are now are something I would like to have once my work is done. When doing my data analysis it would be much more helpful to have some kind of database format that keeps track of everything not just the final state.
Bas Nijholt
@basnijholt
Jun 15 2015 16:41
That's exactly what git will do for you.
cel4
@cel4
Jun 15 2015 16:44
I have tried to make notebooks + git working for me, but for me it's really a pain.
It feels a little like using a git repo to manage binary files. In principle possible, but more annoying than useful. At least that's my impression.
Bas Nijholt
@basnijholt
Jun 15 2015 17:19
If you delete the output with some filter it's much nicer to work with.
cel4
@cel4
Jun 15 2015 17:46
Hmh, true - But that would make things even more complicated. I do not really want to strip the plots from my notebooks. Most of the time I just want to look at what I did some months ago or copy some plots into a presentation. Rerunning is sometimes not even possible anymore, because of some renamed files, etc.
I mean that's a second thing that's really tricky: once you are using external resources, such as csv files you always have to make sure that your notebook and the files are in sync. Once you move the files, everything will break horribly. I think it would be a great idea to bind data to a notebook - I thought about that a couple of months ago, but I couldn't come up with a nice solution.
Matthias Bussonnier
@Carreau
Jun 15 2015 17:50
No, binding data to notebook is bad.
Notebook and kernel can be in different machines.
You can use 1 project = 1 git repo to keep things in sync,
but data in notebook break a lot of abstractions.
You could create format which bind togeter notebook+data like .tar.gz but a notebook with embeded data is more annoying than good.
Also don't we have a clear filter in nbconvert ?
$ ipython nbconvert --to notebook --ClearOutputPreprocessor.enable=True mynotebook.ipynb if it's buggy, please send PRs :-)
cel4
@cel4
Jun 15 2015 17:56
what would happen if the notebook would contain data as well, but the kernel is on some remote machine? Let's say it also contains a pickled version of a data frame. Would it be a problem?
just hypothetical - not really planning a revolution here :smile:
Min RK
@minrk
Jun 15 2015 18:00
The data would be inaccessible to the kernel, and thus pretty useless.
cel4
@cel4
Jun 15 2015 18:05
So I have the notebook on my computer, but send every execution request to the remote kernel, right? If I now would like to read a csv file from my hdd and do some basic plotting, would that actually work? Or would the file have to be transferred to the remote machine first?
Anton Akhmerov
@akhmerov
Jun 15 2015 18:13
@Carreau good point about ClearOutputPreprocessor. However git filters are expected to read from stdin and write into stdout. Is it possible to achieve the same with ipython nbconvert?
Min RK
@minrk
Jun 15 2015 18:15
nbconvert can write to stdout with --stdout
I don't think it can read from stdin.
Anton Akhmerov
@akhmerov
Jun 15 2015 18:16
Unfortunate.
Min RK
@minrk
Jun 15 2015 18:16
You can write a temp file in the filter, though.
Anton Akhmerov
@akhmerov
Jun 15 2015 18:21
It still means maintaining a custom script, which makes it comparable to writing a short Python script that would call nbconvert.
...at least with my level of shell-fu.
Min RK
@minrk
Jun 15 2015 18:23
three lines instead of 30, but yeah.
Anton Akhmerov
@akhmerov
Jun 15 2015 18:33
Do ipengines respect profile/startup scripts?
Min RK
@minrk
Jun 15 2015 18:34
They are supposed to, but I heard a recent report that they may not be doing so. I haven't had a chance to look into it.
Matthias Bussonnier
@Carreau
Jun 15 2015 18:48
You can probalby hack some using syntax like <( somethign) for stdin. no ?
Anton Akhmerov
@akhmerov
Jun 15 2015 18:50
I could only imagine min's solution, given that a filename is expected by nbconvert.
Bas Nijholt
@basnijholt
Jun 15 2015 19:36
Is there a starmap equivalent for IPython.parallel?
Min RK
@minrk
Jun 15 2015 19:59
There isn't, but it would be easy to write without support from IPython parallel.
Since it's just zipping args for map.
Brian E. Granger
@ellisonbg
Jun 15 2015 20:06
@/all how do we add folks to our jupyter hackpad org?
I am working with the students and want to use hackpad with them
Min RK
@minrk
Jun 15 2015 20:07
There's an invite box on the right
Bas Nijholt
@basnijholt
Jun 15 2015 20:20
Do you know any numbers of when v.gather v.scatter is faster than lview.load_balanced_view()?
Min RK
@minrk
Jun 15 2015 20:21
I don't
Bas Nijholt
@basnijholt
Jun 15 2015 20:21
I notice that when the execution time is ~100 ms gather, scatter is faster.
Min RK
@minrk
Jun 15 2015 20:21
It depends a lot on the network and whether data is part of the task or just instructions
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:25
hello
Matthias Bussonnier
@Carreau
Jun 15 2015 21:25
hi.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:26
at the request of Carlos, I am looking at instances where we use the 'profile_default' directory in spyder
ipython_qtconsole_config.py is loaded from that directory
but this is a jupyter thing now
load_pyconfig_files is in traitlets now, which is fine, since there is a shim for that in IPython.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:31
what would be the counterpart to ipython_qtconsole_config.py ?
or is there none?
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:43
Besides, Spyder tries to clean up connection files of kernels that it created. I wonder whether it should do this at all.
Matthias Bussonnier
@Carreau
Jun 15 2015 21:46
.ipython/profile_default/ -> .jupyter/
ipython_*_config -> jupyter_*_config
You can also .py -> .json amd manipulate programatically if you like.
Min RK
@minrk
Jun 15 2015 21:50
It would probably be nice for there to be a method to load default config from the Application classes.
This may already be possible.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:51
osp.join(get_ipython_dir(), 'profile_default') -> jupyter_runtime_dir
Min RK
@minrk
Jun 15 2015 21:51
jupyter_config_dir
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:51
ok
since load_pyconfig_files fails silently could load both all the time
Min RK
@minrk
Jun 15 2015 21:52
But I would first look into loading config with the the existing objects, rather than duplicating the loading code.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:52
sorry, I dit not understand what you mean
Min RK
@minrk
Jun 15 2015 21:55
Something like:
app = QtConsoleApp()
app.load_config()
cfg = app.config
Instead of entering paths, etc. yourself.
That way you will get system-wide config files, .py+.json files, and the changing paths, all properly handled.
I'm not sure exactly what that code would look like, though.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:56
well, in any case, we are still supporting old versions.
Min RK
@minrk
Jun 15 2015 21:57
Yes, and this would make that easier.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 21:57
so here: https://github.com/spyder-ide/spyder/blob/master/spyderlib/widgets/ipython.py#L680
I would just load from jupyter_config_dir if I can import it.
Hamzeh Alsalhi
@hamsal
Jun 15 2015 21:58
Hello, is there a post anywhere that talks about the terminal introduced to ipython notebooks?
or the small pane that pops up to display docstrings
Min RK
@minrk
Jun 15 2015 22:00
@SylvainCorlay if possible, I would instantiate a QtConsoleApp object. Or a dummy sublcass, load config using the App's own config loading mechanism, and return the Config object.
In that case, you would never write the path to a config file, or instantiate any Loaders directly, and any changes in the behavior of QtConsoleApp would be properly handled in Spyder.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 22:01
does [app].load_config exist in old versions of IPython?
1.*
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 22:07
ok, I will add such a function, and keep the old code for IPython < 4.0
Min RK
@minrk
Jun 15 2015 22:10
yes. It's Application.load_config_file, all the way back to 0.x.
It may require some code to handle changes in what needs to have happened before load_config_file is called.
For instance, in IPython, I think init_profile_dir may need to be called, but that method is gone in Jupyter.
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 22:12
we were only using the default profile in spyder
does that make the call to init_profile_dir unnecessary?
Sylvain Corlay
@SylvainCorlay
Jun 15 2015 22:18
nevermind, will look into all this closely
Min RK
@minrk
Jun 15 2015 22:37
I think not, since the ProfileDir object still needs to exist.