These are chat archives for ipython/ipython

16th
Dec 2014
Jonathan Frederic
@jdfreder
Dec 16 2014 01:47
Yay, looks like Travis hit the 5 hour mark with #7227
Let's see how long it will take!
Thomas Kluyver
@takluyver
Dec 16 2014 01:58
I think it's speeding up a bit now
perhaps because the US working day is now over
apart from Hawai'i, I guess
...and how much work gets done there, right ;-)
Scott Sanderson
@ssanderson
Dec 16 2014 02:33
@minrk I'm getting an error in test_get_contents_binary_file where IPython's output appends a newline to the base64 of the input data and my implementation doesn't
not sure which one is correct
interestingly, b64decode of the both results returns the same value, whether the newline is present or not
Min RK
@minrk
Dec 16 2014 02:35
I think the newline is inconsequential, so it's probably best to compare the b64-decoded result, rather than the encoded value.
Scott Sanderson
@ssanderson
Dec 16 2014 02:36
I think there's some asymmetry between how the FileManager tests are using b64encode/b64decode versus encodestring/decodestring
but yeah, I think the tests would pass for both of us if the comparison was on the decoded values
it just gives me the mild heebie-jeebies to be returning something different from the default
Jason Grout
@jasongrout
Dec 16 2014 20:10
@jdfreder? I had some problems with the widget persistence
basically, what happened is that one document's widgets showed up automatically in another document.
It's possible that the two documents had the same filename. How are we associating widget data stores with notebooks? Is it by filename?
or are we, say, storing a widget state with the hash associated with the notebook?
Brian E. Granger
@ellisonbg
Dec 16 2014 20:16
Hmmm, that sounds weird, I will talk to @jdfreder about this today
Jason Grout
@jasongrout
Dec 16 2014 20:21
there may have been some cache issues or something else at play; I'm not sure. I was a bit surprised to see some widgets from my previous notebook automatically constructed in my new notebook when I opened it.
Jonathan Frederic
@jdfreder
Dec 16 2014 20:45
hehe
that's funny
@jasongrout it's just localStorage
I'm not sure how the browser handles it
but it may be by URL
so if it was the same filename&path..
Jason Grout
@jasongrout
Dec 16 2014 21:16
perhaps it should also store some identifier of the notebook execution state, some sort of hash that would say that nothing changed since I last ran the notebook
Dan Bartow
@dbartow
Dec 16 2014 21:16
When I try to use nbconvert to save a notebook as a PDF, I get a 500 in the browser, or this at the commandline... any assistance is appreciated: File "/usr/local/lib/python2.7/dist-packages/IPython/nbconvert/filters/markdown.py", line 80, in MathBlockLexer
default_features = ['block_math', 'latex_environment'] + mistune.BlockLexer.default_features
AttributeError: type object 'BlockLexer' has no attribute 'default_features'
Jason Grout
@jasongrout
Dec 16 2014 21:16
or perhaps it should only restore the state on refresh with the currently active kernel. That would be more conservative.
Dan Bartow
@dbartow
Dec 16 2014 21:16
Followed by a large stack trace
Min RK
@minrk
Dec 16 2014 21:17
@dbartow try pip install --upgrade mistune
Dan Bartow
@dbartow
Dec 16 2014 21:18
@minrk It says requirement already up to date
"Requirement already up-to-date: mistune in /usr/local/lib/python2.7/dist-packages"
Min RK
@minrk
Dec 16 2014 21:20
hm python -c 'import mistune; print(mistune.__version__)'?
Dan Bartow
@dbartow
Dec 16 2014 21:20
0.5
Thomas Kluyver
@takluyver
Dec 16 2014 21:21
@dbartow how old is your IPython? If you haven't pulled in a week or more, you may need to pull now
Dan Bartow
@dbartow
Dec 16 2014 21:22
Quite a while since my last pull - a month or two.. pre Jupyter work... Good suggestion
Thomas Kluyver
@takluyver
Dec 16 2014 21:25
yep, if you've got mistune 0.5, you need IPython from the last week or so
there was a change in mistune, and we had to make a fix to match it
Dan Bartow
@dbartow
Dec 16 2014 21:27
@takluyver : Awesome man - thanks so much for the info!
Thomas Kluyver
@takluyver
Dec 16 2014 21:31
you're welcome
Anand Thakker
@anandthakker
Dec 16 2014 21:42
Hi all. Just started using ipython notebooks -- is there any way to make the .ipyn file less noisy in version control? (like, keeping the interactive session state in a different file or something?)
Min RK
@minrk
Dec 16 2014 21:43
You can use a git pre-commit hook or a git filter to remove the output you don't want to preserve
Scott Sanderson
@ssanderson
Dec 16 2014 21:44
@anandthakker there's also an ongoing discussion here ipython/ipython#7244 about how trust signatures are stored, which is often a source of noise when a single .ipynb file is shared between multiple users
Anand Thakker
@anandthakker
Dec 16 2014 21:45
@minrk @ssanderson cool thanks!
Scott Sanderson
@ssanderson
Dec 16 2014 21:47
@minrk is the signature stuff on the docket for the next dev meeting? I'd be interested in joining that discussion if so.
Anand Thakker
@anandthakker
Dec 16 2014 21:47
@minrk oh excellent -- i was hoping not to have to start learning the ipyn file format :)
Min RK
@minrk
Dec 16 2014 21:48
@ssanderson yes
Thomas Kluyver
@takluyver
Dec 16 2014 21:48
@ssanderson yes, it is. We'll try to get you in if there's space
Min RK
@minrk
Dec 16 2014 21:48
though I won't be there
Dale Jung
@dalejung
Dec 16 2014 23:17
Are there any plans for cells having ids?
Thomas Kluyver
@takluyver
Dec 16 2014 23:19
no, I don't think so
Min RK
@minrk
Dec 16 2014 23:21
It's been discussed, but cell IDs are a very delicate issue, as with anything that adds UUIDs or hashes to notebook content.
manually defined cell 'names'
have been discussed as an alternative, for the cases where the goal is things like HTML anchors.
Jason Grout
@jasongrout
Dec 16 2014 23:22
cell 'names' could be implemented with tags too
Min RK
@minrk
Dec 16 2014 23:25
except names are expected (not enforced) to be unique, whereas tags are expected to have many matches
Jason Grout
@jasongrout
Dec 16 2014 23:25
right, it's a bit of an abuse of tags
Min RK
@minrk
Dec 16 2014 23:25
so we probably wouldn't ever make HTML anchors based on tags
Jason Grout
@jasongrout
Dec 16 2014 23:25
definitely not by default
Min RK
@minrk
Dec 16 2014 23:26
or ever
Jason Grout
@jasongrout
Dec 16 2014 23:26
I mean, definitely not ever make default anchors based on tags
Min RK
@minrk
Dec 16 2014 23:27
right, if there's an anchor, it should always come from somewhere other than tags
Dale Jung
@dalejung
Dec 16 2014 23:27
hm, if I wanted to add a UUID, I guess I'd just plop it into the cell metadata?
Jason Grout
@jasongrout
Dec 16 2014 23:27
+1 to making anchors from something not tags. Whew, glad we got that off our chest...
Dale Jung
@dalejung
Dec 16 2014 23:27
on a first pass, i want my content manager to auto strip and store the output separate from the ipynb
i could just also store a mapping that's corresponds to order
but i think i want to try some funkier stuff with cells being first class. i.e. storing cells separately in DB backend.
i'll let you know how the rabbit hole goes
i think @Carreau had a PR for merging/diffing back in the day that included cell ids
Min RK
@minrk
Dec 16 2014 23:32
yes, that didn't get merged
You can add info to metadata
you will just need to be very careful to handle things like copies of cells that now have the same UUID
but different content
db storage is a good idea, and something I think people should be exploring, so I look forward to hearing back from you
we did have such things in mind when designing the notebook structure (as opposed to the JSON-specific file format)
but alternative notebook storage mechanisms should be really interesting, and give the ability to easily compose new notebooks with cell queries, etc.
Scott Sanderson
@ssanderson
Dec 16 2014 23:35
@dalejung if you haven't seen it already, you might be interested in https://github.com/quantopian/pgcontents
splitting cells apart sounds interesting
Min RK
@minrk
Dec 16 2014 23:36
one of the earlier notebook implementations in 2006-7 had notebooks entirely in a database, where the notebook itself was a skeleton of references to cells
I think it's highly appropriate to store cells independently, as well as outputs
so a notebook has a list of cell references and a cell has a list of output references
Dale Jung
@dalejung
Dec 16 2014 23:39
yeah, I've been thinking that a classical notebook is just a linear graph of cells. there are lots of times that I wish I could just branch real quick from Cell C
when cells D->G and H->K aren't necessarily right after each other
mentally
but i dunno how that interface would look like
Scott Sanderson
@ssanderson
Dec 16 2014 23:41
that's also tricky given that the html interface lets you reorder cells
Min RK
@minrk
Dec 16 2014 23:42
interfaces are the hard part
Dale Jung
@dalejung
Dec 16 2014 23:42
well i guess you'd only work on one chain at a time, A-B-C-D-E-F-G and A-B-C-H-I-J-K
it's more of an issue I have where I jump around a lot in a notebook, and it starts getting messy, cuz it's no longer linear
Min RK
@minrk
Dec 16 2014 23:43
but once you have it stored that way, you can build tools that use it - even if they aren't in the existing notebook UI, you could have a notebook query that goes from db -> linear notebook
Dale Jung
@dalejung
Dec 16 2014 23:46
yar, the way I'm thinking is that the nb model isn't the backend model, but the viewmodel. There to power the linear notebook, but the backend model is more hand wavy.
@ssanderson I'll take a look. My first pass is to split out the outputs. My default content manager is a bundle manager where each notebook is a directory, like an OSX package.
so sticking the outputs within that dir in like an _outputs subdir fits nicely
then i guess i'll just gitignore the *.ipynb/_outputs/