Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Curtis Rueden
    @ctrueden
    @gab1one I updated scijava-io-http to use scijava-common-2.76.0 and cut the 0.2.0 release. It was a little involved due to changes in the abstract DataHandleTest. When you have time, please take a look to make sure I didn't mess anything up.
    Gabriel Einsdorf
    @gab1one
    I looked at and only issue I see is that the remote reading test got lost, l added a PR to restore it
    Curtis Rueden
    @ctrueden
    @gab1one OK, here are the challenges I see to solve for a public domain image test repository:
    1) Actually getting images from people donated to CC0/UNLICENSE. This is not so hard, really, we just need to ask people. I have some images in quite a few formats I can donate as a starting point.
    2) Storage mechanism. Do we host them on plain HTTPS? I think this might be best. We could create https://samples.imagej.net. Right now we have only https://samples.fiji.sc. But "samples" is maybe not the same thing as "test images"
    3) For tests that need them, simple reusable code for A) downloading and caching them and opening locally; and B) opening them remotely via HTTP using scijava-io-http.
    Do you see any other issues?
    Gabriel Einsdorf
    @gab1one
    No that looks good, only thing I want to add B) won't work with all formats, that is why we will need A)
    Curtis Rueden
    @ctrueden
    Yeah, (3A) is needed, and I think better, in (nearly?) all cases.
    The only time I wouldn't want to download+cache is when we are explicitly testing the remote loading feature.
    Otherwise, it is just slower and wastes bandwidth.
    Gabriel Einsdorf
    @gab1one
    True
    Curtis Rueden
    @ctrueden
    OK, let's decide how ambitious we want to be. I like the idea of hosting all of these at samples.imagej.net.
    Not necessarily all in ImageJ/Fiji's Open Samples menu... just hosted from that domain.
    Because it is a public domain sample image collection.
    And I do not think it belongs at samples.scijava.org because ImageJ specifically is all about images.
    I guess we could do samples.scif.io.
    Gabriel Einsdorf
    @gab1one
    samples.scif.io looks cool
    Curtis Rueden
    @ctrueden
    Which do you like more? samples.imagej.net or samples.scif.io? The latter implies they are intended to go hand in hand with SCIFIO I/O, testing, etc. But SCIFIO is built on top of ImageJ.
    Hehe.
    I can set up samples.scif.io. Anyone else in the audience have an opinion?
    Curtis Rueden
    @ctrueden
    @gab1one Regarding (3), have you written that code yet? Do you need any guidance? It'll be analogous to how imagej-tensorflow downloads+caches models.
    I don't think it will be exactly like that, because SCIFIO tests won't need (or want) to update a status bar. But maybe they want to emit progress to the log or some such.
    Ideally this could be generalized as a core SciJava service but I dunno.
    Are you OK with all the samples being hosted zipped?
    Gabriel Einsdorf
    @gab1one
    There is no code for 3) yet, I will look into creating that
    Thanks for the example, looks like we already have most of the building blocks already, so this should be quite easy to build
    Curtis Rueden
    @ctrueden
    @gab1one Thanks, I will try to create samples.scif.io later this week and populate it with the LOCI sample data. My plan is to also edit the scif.io website to have a "Sample Data" section with thumbnails and links to the samples.scif.io files.
    Curtis Rueden
    @ctrueden
    @gab1one I updated the relevant issue to keep track of all this: https://github.com/scifio/scifio/issues/85#issuecomment-438668654
    Gabriel Einsdorf
    @gab1one
    :thumbsup:
    Fokatu
    @Fokatu
    I want to define a file type for my use. Is it possible to append an XML part to binary data?
    Or is there an exist file format similar to my needs?
    Curtis Rueden
    @ctrueden
    @gab1one @dietzc I continued hacking on the SCIFIO work on the plane. Still some problems to fix. OMEVisual uses the SCIFIO OME-TIFF support, which was removed in favor of Bio-Formats. bigdataviewer_fiji uses ImgOpener but the method sigs changed. I'll look into whether we can readd the old sigs for backwards compatibility. The scifio-cli component had not been updated; I pushed a new-scifio branch doing so (STILL UNTESTED).
    The branch is called new-scifio rather than scifio-1.0.0 because the latter looks like a release tag to Travis—we don't want Travis building these branches. (See scifio-bf-compat for example of why not.)
    The scifio-tutorials were also never updated... -_-
    Curtis Rueden
    @ctrueden
    @gab1one I am looking for test-dicom.zip and test-nrrd.zip. They are not on samples.scif.io yet. Do you have these files locally? Can you please share me a link so that I can upload them, so that the new SCIFIO tests can pass?
    Also, if you are available today, it would be great to do our scheduled video call to check in on current status. I am working on merging scifio/scifio#391 right now, but we gotta fix those failing tests.
    Curtis Rueden
    @ctrueden
    @gab1one Locally, I have cleaned up all the various SCIFIO repository location API update branches. I am about to start a Fiji melting pot. If all goes well I’ll merge and release everything.
    Jan Eglinger
    @imagejan
    Hi all, if someone could help me shortly on how to get series names using SCIFIO (forum topic), I'd be grateful.
    NicoKiaru
    @NicoKiaru
    Hi everybody, I'd like to dig into using Scifio in order to open VSI files. We are heavy users of this file format at our facility, and making this work with Scifio would be awesome.
    Is anybody interested and willing to join the effort ?
    NicoKiaru
    @NicoKiaru

    I believe there's a chance that it could work without too much effort. Here's a few things I've done and noticed:
    0 - I acquired a few test VSI images, widefield or fluorescent, multichannel or single channel, 2D or 3D
    1 - 2D Single channel images can already be open using SciFIO! But multi-channel images are causing troubles, I think because there's a confusion between channels and Z dimension
    2 - Pyramidal image can be put aside for the moment, but it would be nice to retrieve them for the sake a making a nice BigDataViewer Source

    Here's what I' trying to do at the moment:

    Curtis Rueden
    @ctrueden
    @NicoKiaru I thought Bio-Formats already supported VSI format? Can't you use SCIFIO with the Bio-Formats adapter layer?
    NicoKiaru
    @NicoKiaru

    Overall the format support with SCIFIO is currently not good enough for VSI, and we are heavy users of this file format in the facility where I work. We have to deal with huge 2D slide scanners dataset, stored in VSI file format. VSI contains several datasets and their associated pyramids. Among the current limitations:

    • I get this error (https://gist.github.com/NicoKiaru/3143871c62a42c2e35495e859f9dbda9) with many VSI files and SCIFIO. I dug into the code but was unable to grasp where I could solve the issue. With BioFormats importer however, I can open one of the many suggested images, so it's probably SciFIO related
    • Files take forever to open, in IJ1, but also when fetching an Img using SciFIO and giving it directly to BigDataViewer, (when loading the highest resolution image of the pyramid, which is required at some point anyway)
    • I couldn't find if it was possible to use the VSI stored pyramid to give it to BigDataViewer, via a Source<T>, using SciFIO

    What I have currently done:

    • Plenty of sample VSI datasets, some in 2D, in 3D, 1 or 2 channels with successive scans etc. I hope to be able to open all of them, quickly, and with very little post-opening tweaking (metadata z vs c issue, for instance). I can share them, of course.
    • I finally did a test by skipping SciFIO, and building a pyramidal Source<T>, cached, that uses the stored pyramid, directly from BioFormat. The opening time is instantaneous, and even if it's currently failing because of me testing it quick and dirty, it's very promising imo (https://twitter.com/nKiaru/status/1133387957203800065)

    I'll continue along this path, and I'll be happy to help in order to provide a better support for VSI format. Also maybe I missed something big which can solve the problem instantaneously ;-), I'd be happy to hear it!

    Curtis Rueden
    @ctrueden
    @NicoKiaru Could you file issues in the SCIFIO bug tracker? I think most of your issues are not VSI-specific, but rather SCIFIO's lack of support for pyramidal image structures. Unfortunately, this is a tall order to address, and no one currently has time.
    Glad to hear that calling Bio-Formats directly has the potential to be fast.
    Curtis Rueden
    @ctrueden
    @frauzufall Here is an issue reported in fiji/fiji that stumbled across the false vs. true color issue of SCIFIO & Bio-Formats: fiji/fiji#168
    I have a feeling we should do something about it, because otherwise people will keep hitting it, but I don't know what to do.
    Maybe I should start a forum thread asking for opinions in the Development category?