Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Fred Mitchell
@fm75
Isn't there one in ipywidgets?
Michał Krassowski
@krassowski
but the ipywidgets one is kernel dependent, isn't it?
Fred Mitchell
@fm75
Yeah
blois
@blois
I keep wanting something like https://github.com/easylogic/codemirror-colorpicker. Presumably could trigger it with color('#aaa'), where color was a function that just returned a string, but was a hint to the editor to highlight the string argument as a color.
Michał Krassowski
@krassowski
wow, that's such a great code mirror extension! just needs a night theme to greatly improve UX!
Vidar Tonaas Fauske
@vidartf
so I guess it is a de-facto standard kind of ?
Steven Silvester
@blink1073
Yeah it was added later and not well documented it seems
Vidar Tonaas Fauske
@vidartf
would you consider it part of that standard that the partial file must be written after receiving chunk: 1 ?
Vidar Tonaas Fauske
@vidartf
or is that purely a detail in the default implementation
Jason Grout
@jasongrout
I would say (conservatively) an implementation detail?
Is there a specific capability you have in mind to make it part of the standard?
Vidar Tonaas Fauske
@vidartf
what I mean is that currently both lab and notebook will send the chunk argument along together with a partial file
any contents managers that do not understand chunk will simply end up overwriting the file with the contents of the final chunk
leading to broken/corrupt files
and no errors
so either I would want to document this as a required part of the contents api standard
or put some flag on the manager classes that the server can check if it receives chunks (and fail with a 4xx/5xx error)
*fail if unsupported
Jason Grout
@jasongrout
I agree about documenting "chunks" as a part of the standard. What I meant was is there a reason to insist that a partial file must be written after getting the first chunk as part of the standard?
Vidar Tonaas Fauske
@vidartf
I see that the LFM does it, and wondered there would ever be a case were some other system relied on it being returned with the dir listing while partial
just a sanity check I guess
since the hanbdler is expected to return a model
Jason Grout
@jasongrout
yeah, I could see that either way. I think a standard thing to do when having a partial file is to store it as a temporary file until the whole file is done, then rename it to the final name, so I would say writing the partial file is an implementation detail
but good point about the handler returning a model...
maybe that forces our hand if you have to return a model. Should we add something to the model indicating the file is partial?
Vidar Tonaas Fauske
@vidartf
I am happy to document the api as "if you choose to use the chunk api, you must regard all intermediate models as invalid until the final chunk is written" ?
Jason Grout
@jasongrout
or maybe the returned model should be basically null until the final chunk? I wonder how many frontends that would break

I am happy to document the api as "if you choose to use the chunk api, you must regard all intermediate models as invalid until the final chunk is written" ?

Sure, or a softer way is to say that the behavior is undefined, but I suppose we all know the tendency to write software assuming undefined behavior, so it's probably better to be clear in the documentation, then back it up in the implementation

in other words, if we document the models as invalid, we should change the implementation to make them invalid
otherwise the implementation does not match the documentation, and as we've seen in the past, with a single major implementation, it becomes the de facto standard, despite what the document says
Vidar Tonaas Fauske
@vidartf
well, it strictly depends on the file type whether a partial file would cause the model to be "invalid" or not (or rather, I guess the model might still be valid?)
the model would be valid for a partially written file, just that the file might possibly be correupt
Jason Grout
@jasongrout
in fact, past discussion about nbconvert has basically been that the de facto standard (i.e., implementation of the standard) overrode the standard when there was a conflict.

the model would be valid for a partially written file, just that the file might possibly be correupt

Yes, I think that's the correct interpretation for our current situation

If we are writing at chunk 1
Vidar Tonaas Fauske
@vidartf
but if the contents is cached, I guess the model to return is an empty one ?
Jason Grout
@jasongrout
So perhaps the model should contain a flag saying it is a partial file? That would be backwards compatible with our current implementation, but be more useful?
with the chunk parameter, do we know for sure when the file is done? Is there some flag in the upload saying the file is done? If a separate request is made to the server to list the files, will it know that a file is done?
Vidar Tonaas Fauske
@vidartf
the final chunk is numbered -1
Jason Grout
@jasongrout
ah, great.
However, we still have the problem of another asynchronous request to the server seeing the file on disk and returning the model
not knowing the file is a partial file
Again, perhaps the common pattern of writing to a temporary file then renaming at the end is a better pattern
Vidar Tonaas Fauske
@vidartf
I'm happy to leave the current notebook implementation as-is (with a note to self about the possible non-transactional nature of large uploads), and just document the current de-facto standard
at least as a first step
at least I have assured myself somewhat that neither lab nor classic actually use the intermediate models for anything
Jason Grout
@jasongrout
well, you still have the issue of the asynchronous directory list request returning a model that is not so useful. But thanks for checking the code of the upload part
+1 to documenting the implementation as a good first step