Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 25 20:56
    ctrueden commented #238
  • Feb 25 20:54
    ctrueden closed #71
  • Feb 25 20:54
    ctrueden commented #71
  • Feb 21 16:59
    carandraug commented #61
  • Feb 21 16:48
    bethac07 commented #61
  • Feb 21 16:06
    carandraug commented #61
  • Feb 21 15:42
    bethac07 commented #61
  • Feb 21 10:49

    tinevez on master

    Better logo color scheme. (compare)

  • Feb 19 20:36
    ctrueden milestoned #32
  • Feb 19 20:36
    ctrueden assigned #32
  • Feb 19 20:36
    ctrueden edited #32
  • Feb 19 20:35
    ctrueden labeled #32
  • Feb 19 20:35
    ctrueden unlabeled #32
  • Feb 19 20:35
    ctrueden commented #32
  • Feb 17 18:35

    etadobson on coloc

    (compare)

  • Feb 17 18:34

    etadobson on coloc-multithread-saca

    WIP: First attempt to multithre… (compare)

  • Feb 17 17:43

    etadobson on coloc

    Add progress bar to AdaptiveCol… AdaptiveSmoothedKendallTau: use… (compare)

  • Feb 17 15:33
    etadobson commented #613
  • Feb 17 15:32

    etadobson on coloc

    Add progress bar to AdaptiveCol… (compare)

  • Feb 17 15:31

    etadobson on coloc

    Add status bar to AdaptiveColoc… (compare)

NicoKiaru
@NicoKiaru
@hanslovsky cool! I fear a bit the fact, that I'm wrapping many times the source (Source -> AffineTransformedSource -> WarpedSource (-> DataSource)). But that looks like a very good solution. (this interface, right ? https://github.com/saalfeldlab/paintera/blob/master/src/main/java/org/janelia/saalfeldlab/paintera/data/DataSource.java). However why is your function getDataSource(int t, int level); returning a RandomAccessibleInterval ? Why not a Source ?
Philipp Hanslovsky
@hanslovsky
Because all the shared meta data (transforms, etc) are stored within a single source, that way.
And I can just query the data or volatile as I need it
NicoKiaru
@NicoKiaru
Ah I see, so you track somewhere else the list of sources which should be transformed in an identical manner ?
Philipp Hanslovsky
@hanslovsky
Each source (and data source) has its own transform in my use case. It's just a matter of accessing data and volatile voxels in a consistent way
NicoKiaru
@NicoKiaru
Ok, I'll have to think about it, thanks for the explanations
Christian "Tischi" Tischer
@tischi
@NicoKiaru I discussed this once with Tobias and I think the conclusion was that one usually only needs Volatile pixels types within the Bdv for rendering, such that you can rotate a volume quickly, even is the data is not yet loaded (i.e. not valid). For any sort of computation one typically wants to have the actual valid data. Thus, I think, one way of seeing the issue is that we would never have to care about Volatile data outside Bdv, but upon saying BdvFuntion.show( source ), Bdv would internally VolatileViews.wrapAsVolatile( ... ) for you. Currently however there are some limitations, e.g. VolatileViews.wrapAsVolatile( ) is only implemented for RandomAccessible and not for a Source and also VolatileViews.wrapAsVolatile( ) does not always work, e.g. if the RAI is internally derived from too complicated Views and Converters. With the help of @hanslovsky I added some functionality to VolatileViews.wrapAsVolatile( ) to make it cover more scenarios, see e.g. https://github.com/tischi/fiji-plugin-bigDataProcessor2/blob/master/src/main/java/de/embl/cba/bdp2/volatiles/VolatileViews.java#L186 but there are still cases where it cannot handle it.
NicoKiaru
@NicoKiaru
cool! Thanks @tischi that's very good to know. Maybe we can work on this Source wrapping in BdvFunctions @tischi ? Would this class be generic enough ? https://github.com/BIOP/bigdataviewer-bioformats/blob/master/src/main/java/ch/epfl/biop/bdv/bioformats/bioformatssource/VolatileBdvSource.java
For now, while waiting for BdvFunctions to wrap a Source into a volatile Source, I think we have to keep the (optional) wrapping in the bigdataviewer-bioformats repo.
NicoKiaru
@NicoKiaru
Something which is not clear to me is the image loader: outside of BdvFunctions, it seems that a SetupLoader which extends an AbstractViewerSetupImgLoader should also provide a Volatile source. As a consequence the source volatile wrapping has to be done internally to the repository anyway.
Christian "Tischi" Tischer
@tischi

it seems that a SetupLoader which extends an AbstractViewerSetupImgLoader should also provide a Volatile source.

This is exactly the kind of stuff I was also just thinking about. To me, right now it seems as if there are two ways to provide Volatile pixels: (a) via the SetupLoader logic and (b) via this kind of logic: https://github.com/tischi/fiji-plugin-bigDataProcessor2/blob/master/src/main/java/de/embl/cba/bdp2/volatiles/VolatileViews.java#L118 where a CachedCellImg can be made to return Volatile pixel values. Honestly, I think I would need a Skype, probably with someone like @tpietzsch or @hanslovsky and a white-board to draw how all these things are connected :-)

Philipp Hanslovsky
@hanslovsky

The SetupLoader existed before the CachedCellImg. I use CachedCellImgs exclusively because it is extremely convenient. There may be benefits of SetupLoader that I am not aware of.

VolatileViews.wrapAsVolatile should be sufficient for any/most standard ImgLib2 native types. It gets a little more tricky when more complex types are being used, e.g. LabelMultisetType in Paintera, or Composite<T> vector-like types.

Christian "Tischi" Tischer
@tischi
@hanslovsky very interesting! Do you use SpimData at all? Or do you mainly work with Source?
Philipp Hanslovsky
@hanslovsky
I do not use SpimData, only Source.
Curtis Rueden
@ctrueden
@tpietzsch This article reminds me of all the equality semantics woes we have when composing multiple interfaces. :grin:
tpietzsch
@tpietzsch
@tischi @NicoKiaru Quick explanation of the volatile/non-volatile stuff:
There is ViewerSetupImgLoader. It extends BasicMultiResolutionSetupImgLoader and adds methods
RandomAccessibleInterval<V> getVolatileImage( final int timepointId, final int level, ImgLoaderHint... hints );
V getVolatileImageType();
on top of the existing
RandomAccessibleInterval< T > getImage( final int timepointId, final int level, ImgLoaderHint... hints );
T getImageType();
tpietzsch
@tpietzsch
So it can provide both volatile and non-volatile version of each image (types would be eg T == UnsignedByteType and V == VolatileUnsignedByteType
The implicit idea is that both images are backed by the same data, just that the non-volatile version blocks when invalid data is encountered until that data is loaded. So for example, if a pixel is accesses in the non-volatile image, that pixel is then automatically already valid in the volatile version. (This is the same with VolatileViews.wrapAsVolatile btw).
tpietzsch
@tpietzsch
The BDV will show sources of any type. The Bdv knows about sources always as SourceAndConverter (because for rendering there must be a converter from the source's pixel type to ARGB).
SourceAndConverter has a method
SourceAndConverter< ? extends Volatile< T > > asVolatile();
that returns a volatile version of the source(andconverter) if it exists.
(otherwise asVolatile returns null)
The idea is that always the non-volatile version is added to the BDV, maybe with the volatile version nested underneath
This way, if somebody takes the source and does image processing, movie rendering, etc, they get the non-volatile version and there are no surprises.
The renderer will ask for the asVolatile() version, and, if it exists, use that.
With the VolatileViews.wrapAsVolatile this idea was a bit weakened, because now you effectively add a volatile source directly (with nothing non-volatile on top of it)
tpietzsch
@tpietzsch
It would be probably good, to
(1) make a way to expose adding SourceAndConverter in the vistools BdvFunctions, and
(2) make some version of VolatileViews.wrapAsVolatile that constructs a volatile view, creates appropriate RGBA converters, and then wraps it with the original RandomAccessible into a non-volatile/volatile SourceAndConverter pair. The code for all of these steps is there, it just needs to be reorganized and exposed...
Curtis Rueden
@ctrueden
I sometimes come across posts/comments/discussions like this one: "what would python integration offer that isn’t already taken care of by cupy/Numba/pytorch/tensorflow etc".
Would it make sense to, together with experts in Python-based scientific image processing, and possibly also C++ and/or JS, do some kind of more deliberate comparison of pros/cons of the ecosystems? It always feels like all the specialists are talking past each other to me—or speaking from places of ignorance w.r.t. the other available tools. I wonder if e.g. a detailed blog post with summary would keep a whole slew of people from reinventing wheels... or at least reinvent them in better and more impactful directions.
NicoKiaru
@NicoKiaru

Thanks @tpietzsch ! It seems that the method BdvFunctions.show(Source src); has also two limitations: (3) - the source is not wrapped into a volatile one; (4) - We cannot specify the number of timepoints.

(5) Another feature which would be nice is also the capability to create a viewer without any Source.

Do you think working on these three extra points would make sense ?

NicoKiaru
@NicoKiaru
Ah, I see that BdvFunctions can handle 4D Source. However I'm not sure it integrates very well with SpimDatastructures and plugins like BigStitcher.
tpietzsch
@tpietzsch
@NicoKiaru yes, it would make sense to work on those
NicoKiaru
@NicoKiaru
Hi @tpietzsch , I wanted to give a shot at these modifications. I've forked the repo, but I found this issue when launching the ExampleSpimData:
Exception in thread "main" java.lang.NoSuchMethodError: net.imglib2.cache.queue.BlockingFetchQueues.<init>(I)V
    at bdv.img.hdf5.Hdf5ImageLoader.open(Hdf5ImageLoader.java:215)
    at bdv.img.hdf5.Hdf5ImageLoader.<init>(Hdf5ImageLoader.java:166)
    at bdv.img.hdf5.Hdf5ImageLoader.<init>(Hdf5ImageLoader.java:152)
    at bdv.img.hdf5.Hdf5ImageLoader.<init>(Hdf5ImageLoader.java:147)
    at bdv.img.hdf5.XmlIoHdf5ImageLoader.fromXml(XmlIoHdf5ImageLoader.java:71)
    at bdv.img.hdf5.XmlIoHdf5ImageLoader.fromXml(XmlIoHdf5ImageLoader.java:50)
    at mpicbg.spim.data.generic.sequence.XmlIoAbstractSequenceDescription.fromXml(XmlIoAbstractSequenceDescription.java:111)
    at mpicbg.spim.data.generic.XmlIoAbstractSpimData.fromXml(XmlIoAbstractSpimData.java:153)
    at bdv.spimdata.XmlIoSpimDataMinimal.load(XmlIoSpimDataMinimal.java:86)
    at bdv.util.ExampleSpimData.main(ExampleSpimData.java:15)
Is it a problem of Java version ? Do you have any idea how I could solve this ?
Simple examples work however
Philipp Hanslovsky
@hanslovsky
@NicoKiaru that looks like version skew, probably between imglib2-cache and bdv-vistools
What versions of those are you using?
NicoKiaru
@NicoKiaru
Just forked the current version of bigdataviewer-vistools
My forked repo is here actually : https://github.com/BIOP/bigdataviewer-vistools
Philipp Hanslovsky
@hanslovsky
Idk if pom-scijava sets the imglib2-cache version to the latest release. You should probably double-check and then set the imglib2-cache version in your pom, if necessary.
NicoKiaru
@NicoKiaru
Will try, thanks! Tha latest version on scijava maven is <version>1.0.0-beta-14-SNAPSHOT</version>. Hope that's recent enough
NicoKiaru
@NicoKiaru
Ok, like this it works:
        <dependency>
            <groupId>sc.fiji</groupId>
            <artifactId>bigdataviewer-core</artifactId>
            <version>7.0.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>net.imglib2</groupId>
            <artifactId>imglib2</artifactId>
            <!--<version>5.8.1-SNAPSHOT</version>-->
        </dependency>
        <dependency>
            <groupId>net.imglib2</groupId>
            <artifactId>imglib2-cache</artifactId>
            <version>1.0.0-beta-14-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>net.imglib2</groupId>
            <artifactId>imglib2-realtransform</artifactId>
            <version>2.2.2-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>net.imglib2</groupId>
            <artifactId>imglib2-ui</artifactId>
            <version>2.0.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.scijava</groupId>
            <artifactId>ui-behaviour</artifactId>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
Thanks @hanslovsky !
Philipp Hanslovsky
@hanslovsky
:plus1:
NicoKiaru
@NicoKiaru
So I don't know if that's the place to share that, but I've made some modifications on BdvFunctions:
  • [a helper class to download and cache datasets for testing] (BIOP/bigdataviewer-vistools@a4e414c)
  • [a way to wrap Source when required and possible according to the functionVolatileViews.wrapAsVolatile] (BIOP/bigdataviewer-vistools@17eeeaa). There are issues, notably regarding the way the cache is handled I guess. It's working for my use case, but I'd like to hear from you regarding what should be improved. Maybe the whole approach is wrong.