These are chat archives for ipython/ipython

8th
Dec 2014
epifanio
@epifanio
Dec 08 2014 05:34
hi @minrk , is the toc extansion working on ipython 2.x ?
epifanio
@epifanio
Dec 08 2014 06:11
on windows
Min RK
@minrk
Dec 08 2014 06:18
@epifanio No, toc.js only works on master.
epifanio
@epifanio
Dec 08 2014 06:20
@minrk thanks! i was working with a friend on windows here at univ . and i was wondering why the extension were not showing up (the js log says extension loaded) do you know which extension are working on 2.3 ?
Min RK
@minrk
Dec 08 2014 06:20
I don't, I only use my extensions on master, so that's the only place they are supported.
epifanio
@epifanio
Dec 08 2014 06:22
cool, thanks, i’ll suggest to switch to master then :)
Kyle Kelley
@rgbkrk
Dec 08 2014 17:40
When are we having our in-person dev meeting next?
Thomas Kluyver
@takluyver
Dec 08 2014 17:40
~February, I would guess
but I don't think we've confirmed dates
we should probably talk about that at the next not-in-person dev meeting
Kyle Kelley
@rgbkrk
Dec 08 2014 17:41
ha, k
Jonathan Frederic
@jdfreder
Dec 08 2014 18:20
@Carreau are you online?
Scott Sanderson
@ssanderson
Dec 08 2014 20:39
is there a specification somewhere for the expected return values from ContentsManager for the various model types (file vs. directory vs. notebook)?
I'm working on a database-backed ContentsManager subclass, but it seems like most of the logic for the specific model structures has been pushed down into FileContentsManager
Thomas Kluyver
@takluyver
Dec 08 2014 21:44
@ssanderson I don't know of any up to date documentation of it
there may be some bits we can pull out from the FileContentsManager and generalise a bit more, but probably not much - for a lot of the fields, it has to stat or open files.
Scott Sanderson
@ssanderson
Dec 08 2014 21:48
@takluyver understood that the implementation probably can't be easily lifted from FileContentsManager; the main thing that would be nice for subclass implementors would be some sort of validation on the expected keys and maybe the types of the values for each model
Min RK
@minrk
Dec 08 2014 21:51
@ssanderson model spec is here
Scott Sanderson
@ssanderson
Dec 08 2014 21:52
@minrk awesome!
and that's up to date?
Min RK
@minrk
Dec 08 2014 21:53
should be
Thomas Kluyver
@takluyver
Dec 08 2014 21:53
didn't we add mimetype?
Min RK
@minrk
Dec 08 2014 21:53
yes, I'll add that
Scott Sanderson
@ssanderson
Dec 08 2014 22:43
@minrk would it make sense for me to put together a PR for generic model validation in ContentsManager? I notice there's already a validate_notebook_model which is doing the heavy work for verify nbformat and all that, but it would be nice for subclasses to have some assurance that they're returning something with the right structure
Min RK
@minrk
Dec 08 2014 22:44
Hm
We only have schema for notebook documents, we don't have any schema for the REST APIs.
I would be hesitant to "schema all the things"
which would be the way to do that sort of thing.
Scott Sanderson
@ssanderson
Dec 08 2014 22:51
why the hesitation to schematize the top-level API?
it seems like there should be some way to at least ask the base class "does the dict I'm about to return have all the expected keys?"
Dale Jung
@dalejung
Dec 08 2014 22:53
is contents api locked down?
Min RK
@minrk
Dec 08 2014 22:54
locked down?
Dale Jung
@dalejung
Dec 08 2014 22:54
As in nothing on the TODO list in terms of API changes.
Min RK
@minrk
Dec 08 2014 22:54
@dalejung nothing known, at this point.
epifanio
@epifanio
Dec 08 2014 22:55
Hi,
do you know a way to store the resutls of each single node in a common list or dict ?
Min RK
@minrk
Dec 08 2014 22:56
@ssanderson because maintaining a schema is another source for bugs and stale documentation. We have tests that verify that it works correctly, which already validate the models. If we need more validation, we can add more tests.
@epifanio what's a node?
epifanio
@epifanio
Dec 08 2014 22:57
i’m trying to run a simulation and it works fine using ipython parallel, the function got executed N time with N different parameters .. but for now the only way i have to access the resutls is to use : results.display_outputs()
that shows me the print stastment of each function
the run is :smile: results = view.map(mcloop, list(a), ordered=False)
where mcloop take as input an element of the list ‘a'
Min RK
@minrk
Dec 08 2014 22:58
@epifanio and you want stdout of each node separately?
results.stdout will be a list
etc.
epifanio
@epifanio
Dec 08 2014 22:59
I need mcloop(a) to write the results in a dictionary
where str(a) will be the key
Min RK
@minrk
Dec 08 2014 23:00
What exactly do you mean by results? stdout, AsyncResult objects, return values?
epifanio
@epifanio
Dec 08 2014 23:01
return values
mcloop(a) return a dict , so i want “result” to be a dict of dicts with “a[i]” as key
a is the list i gave as input
Min RK
@minrk
Dec 08 2014 23:03
something like this:
async_result_dict = {}
# submit work
for item in a:
    async_result_dict[str(item)] = view.apply_async(mcloop, item)

# wait for results, and unpack into result_dict
result_dict = {}
for key, ar in async_result_dict.items():
    result_dict[key] = ar.get()
Scott Sanderson
@ssanderson
Dec 08 2014 23:03
@minrk I'd argue that having a well-defined schema is a form of documentation, and moreover it's the best kind of documentation because it actually has an effect on the behavior of the code.
Min RK
@minrk
Dec 08 2014 23:04
It is a form of documentation, and always a duplicate form of documentation, since schemas are never the only documentation.
Scott Sanderson
@ssanderson
Dec 08 2014 23:04
I buy that, from IPython's perspective, your test coverage is probably sufficient, but as a consumer of the API, my worry is that the only way for me to write tests is to compare my outputs to the output of FileContentsManager, which seems less than ideal.
Jonathan Frederic
@jdfreder
Dec 08 2014 23:06
@jasongrout I'm trying to find our discussion about all widgets having a padding != 0
Min RK
@minrk
Dec 08 2014 23:07
So if you have a formal schema, there are always at least four places the behavior is described:
  1. code, docstrings, etc.
  2. actual for-humans documentation
  3. tests verifying behavior
  4. schema that duplicates parts of 2 and 3
The more places you store the information, the more likely it is that some of them will be wrong.
Jonathan Frederic
@jdfreder
Dec 08 2014 23:09
@jasongrout @ellisonbg pointed out the widgets aren't aligning at all right now in master, I'm thinking of using padding, but instead of setting it in less, setting it as a default value for widget.padding.
Min RK
@minrk
Dec 08 2014 23:13
I'm not necessarily opposed to a schema for the REST API (and it would logicaly be for the entire REST API, not contents), it's mainly a question of whether the burden of maintaining a schema is worth it.
Scott Sanderson
@ssanderson
Dec 08 2014 23:16
I think the main benefit of having a schema is that it provides documentation that's necessarily never out of sync, so as an API consumer that's a huge win for me. It's also much better documentation than the code itself because it's consolidated in one place instead of being scattered across various http handlers and internal classes/files.
At the same time I recognize that having that crutch means there's less pressure for the IPython team to maintain other sources of documentation.
Min RK
@minrk
Dec 08 2014 23:17
The more places we have to update, the more out of sync they are guaranteed to become.
And it's actually quite easy for schemas to be out of date in inappropriately permissive ways
there was a schema for IPython v3 notebooks, and it was all kinds of wrong, inappropriately validating loads of notebooks.
It's absolutely not out of the question, it's just a question of who's going to do it, where it's going to go, etc.
I think it's too late for such a thing in IPython 3.0, but we can consider it in the future.
documentation burden is one of the hardest things for projects - it's extremely valuable, but extremely tedious, so keeping it manageable is important.
Because if the docs aren't maintainable they become the worst possible docs - inaccurate, which is worse than no docs at all.
And that's the state of a lot of IPython's existing docs.
Min RK
@minrk
Dec 08 2014 23:22
But free tests of an API are definitely a big win for having a schema.
Scott Sanderson
@ssanderson
Dec 08 2014 23:22
Agreed. We're in the process of revamping our docs on Zipline and it's a huge pain because they've rotted so much.
epifanio
@epifanio
Dec 08 2014 23:24
@minrk iin my attempt the key is correct .. but the dict content is for key 10 : '1.0': <AsyncResult: finished>
Scott Sanderson
@ssanderson
Dec 08 2014 23:25
So, given that a full-blown schema probably isn't in the cards in the near-term future, it would still be helpful for me as a consumer of the ContentsManager API to have some sort of validation codepath that's part of the base class and shared with the default implementation, so that I have some reasonable assurance that any bugs I have are at least also shared with the default implementation.
Min RK
@minrk
Dec 08 2014 23:27
Making our current tests reusable should be doable, I'll work on that.
epifanio
@epifanio
Dec 08 2014 23:29
oh seems i got it :)
Min RK
@minrk
Dec 08 2014 23:40
@ssanderson I can also make the REST API docs a bit more formal, so that it's more consolidated what keys are required and when.