These are chat archives for fiji/fiji

17th
May 2019
Stephan Saalfeld
@axtimwalde
May 17 01:16
done what I could find, thanks for the reminder
bigcat has to stick with 19.2.0 for old approaches to stuff, but it seems that it builds at least
Stephan Saalfeld
@axtimwalde
May 17 01:21
javafx works with oraclejdk8 on travis and locally with openjdk + openjfx. The Ubuntu distros unfortunately broke the openjfx package for openjdk8 so you have to nail the versions as described here https://github.com/saalfeldlab/paintera
Curtis Rueden
@ctrueden
May 17 02:32
What does “old approaches to stuff” mean?
Regarding openjfx… multiview-reconstruction also has a broken Travis build now because I kicked it over to openjdk8 without ensuring openjfx is installed. Sorry @StephanPreibisch. I can work more tomorrow on fixing that… I’m not sure how we should proceed with Fiji with the bundled JDK—whether to go to Java 11 now, or try to create an OpenJDK8 bundle that includes OpenJFX? @axtimwalde any thoughts?
Stephan Preibisch
@StephanPreibisch
May 17 06:24
As far as I remember @axtimwalde mentioned that Java 11 has problems with imglib2-cache?
Ola Tarkowska
@olatarkowska
May 17 12:49
Hello, could I please ask what is the status of fiji docker image https://hub.docker.com/u/fiji/, is it maintained anywhere?
Josh Moore
@joshmoore
May 17 12:51
Hi @olatarkowska . See fiji/dockerfiles#8 for the most recent progress. I have an outstanding todo to get things deployed. But you can help me validate that PR :)
Ola Tarkowska
@olatarkowska
May 17 12:51
THanks @joshmoore, I updated my question
where is your PR ?
Josh Moore
@joshmoore
May 17 12:52
#8.
Ola Tarkowska
@olatarkowska
May 17 12:52
yes thanks
Josh Moore
@joshmoore
May 17 12:52
(It’s not from me)
Ola Tarkowska
@olatarkowska
May 17 12:53
ohh is this year, excellent, why noone push it to DH ?
Josh Moore
@joshmoore
May 17 12:54
First I was waiting on privileges, and then you gave me lots of other stuff to do :)
Why don’t you build locally and help me verify, then I’ll push?
Ola Tarkowska
@olatarkowska
May 17 12:55
Ok let me get that running in the pipeline
Josh Moore
@joshmoore
May 17 12:55
Ta
Ola Tarkowska
@olatarkowska
May 17 12:55
will come back to you shortly
thank you
Ola Tarkowska
@olatarkowska
May 17 13:24
one more Q is there any example how to install plugin headless ?
Josh Moore
@joshmoore
May 17 13:40
I had that somewhere in a docker. Looking
Ola Tarkowska
@olatarkowska
May 17 13:40
that would be awesome thanks
Ola Tarkowska
@olatarkowska
May 17 13:41
excellent thank !
Josh Moore
@joshmoore
May 17 13:50
Stephan Saalfeld
@axtimwalde
May 17 13:53
java 11 currently breaks imglib2-cache disk cache. People are looking into this but it may take a while.
Stephan Saalfeld
@axtimwalde
May 17 14:00
@ctrueden "old stuff" means that bigcat is an old project that depends on old libraries and isn't easy or meaningful to move to the latest greatest dependencies. The reason for that is that it predates them all. The concepts and ideas that it implements were later extracted from it and have lead to new and improved API design in those library components. Among others, these are BDV, BDVVisTools, label-multisets, N5, ImgLib2, imglib2-cache. We do not further develop BigCAT because Paintera takes over but we still need it for legacy work and projects. It builds and works with pom-scijava-19.2.0 which, in my opinion, is a legal state of existence :). It is not a Fiji or ImageJ component and does therefore not interfere with any of it.
Ola Tarkowska
@olatarkowska
May 17 14:59
@joshmoore I have issues with that docker container, is that downloading latest fiji pre-8 ?
Curtis Rueden
@ctrueden
May 17 15:05
@StephanPreibisch I have local updates to the multiview-reconstruction build that keep it targeting Java 8, while using OpenJDK11 to actually build the thing. I can push it and we’ll see what happens.. even if the build passes (which I expect it will, but you never know)—it would be good to then test it in a real Fiji installation with Java 8.
Alternately: I could try to update the Travis build to install openjfx8 manually before invoking Maven.
@axtimwalde Thanks for the explanation. I hope I didn't come off as critical there—I think extending pom-scijava 19.2.0 makes sense then. If changes need to be pushed and deployed in future, though, then we’ll have to figure something out. The goal is to inherit the new-enough build configuration, while retaining the needed dependency versions as painlessly as possible. I don’t see a trivial way of doing that, but there are ways it can be done with effort (e.g. enumerating all needed versions explicitly as properties on top of pom-scijava 26).
Curtis Rueden
@ctrueden
May 17 15:11
@olatarkowska FYI, the newest Fiji release should be 2.0.0-pre-10. And thank you to both you and @joshmoore for keeping the Docker stuff going. I have no bandwidth at all for that at the moment, but I know it is helpful to have. If there’s anything you need from me to keep that functional, let me know.
Also, I expect to release and upload 2.0.0-pre-11 before end of the month.
@StephanPreibisch @axtimwalde I think I am going to try creating an openjdk8-based Travis build that also installs openjfx first. This seems cleanest to me. Let me know if you disagree.
Josh Moore
@joshmoore
May 17 15:13
@ctrueden : can you just confirm that you’d expect + curl -O https://update.imagej.net/bootstrap.js && jrunscript bootstrap.js update-force-pristin to work with openjdk-8?
@olatarkowska : what error are you seeing?
Curtis Rueden
@ctrueden
May 17 15:24
@joshmoore Hehehe, try using https://downloads.imagej.net/bootstrapJ8.js instead.
Josh Moore
@joshmoore
May 17 15:24
Wonders never cease! Thank you, sir.
Josh Moore
@joshmoore
May 17 15:25
will do.
Curtis Rueden
@ctrueden
May 17 15:25
I hate that it is that complicated—in particular, that I do two update-force-pristines in a row.
The goal is to migrate away from this bootstrap.js nonsense toward mvn with the scijava-maven-plugin. But it doesn’t do all the needed things yet.
Josh Moore
@joshmoore
May 17 15:26
You let me know when there’s something I should try ;)
Curtis Rueden
@ctrueden
May 17 15:26
I diffed those two bootstrap.js files, BTW, and the only difference is one single line adding the Java-8 update site also.
Will do, for sure. :-)
Josh Moore
@joshmoore
May 17 15:27
Ha! Ok. So conceivably it could even be a flag of some sort (or auto-detected?)
Curtis Rueden
@ctrueden
May 17 15:27
We could probably just update the original bootstrap.js at this point to point to the good one. I don’t know what the old one is even used for. Probably nothing.
In the meantime, if you end up doing something similar to that Travis build script, maybe we want to put that into fiji/fiji’s bin folder as a dedicated script, and just curl and call that from everywhere it’s useful.
Finally: is there some reason the Docker build can’t/shouldn't just curl the latest Fiji bundle from https://downloads.imagej.net/fiji/latest directly?
Josh Moore
@joshmoore
May 17 15:32
Looks like that’s what chigozienri did in https://github.com/fiji/dockerfiles/pull/8/files#diff-3f2130049dd552495f062fd9895b10fdR32 . I can certainly move openjdk-8 to do the same.
(probably won’t get that done before this weekend though)
Curtis Rueden
@ctrueden
May 17 15:36
No rush on my side. I think it will save you effort though. As long as it gets the job done!
Josh Moore
@joshmoore
May 17 15:37
Are there any update sites other than Java8 you’d see being on by default in the base image? or a set of sites that you’d see having on in a secondary image?
Ola Tarkowska
@olatarkowska
May 17 15:58
so with BIOP plugin installed running macro
docker run --rm -v /Users/at22/PIPELINE/stitching/img:/img fiji:latest fiji-linux64 --headless -macro /img/macro.ijm /img/test#/img/out
it is complaining
Script1.groovy: 1: expecting '!', found '@' @ line 1, column 2.
   #@File(label="Select your directory with your exported images", style="directory") theDir
    ^
;-)
I dont see that when I run locally
I hardcoded paths in the macro, and didnt help
Josh Moore
@joshmoore
May 17 16:01
I’m going to need to head out soon. I pushed a commit with the suggestion from @ctrueden if you want to give it a try. If you can also post your macro (maybe on that issue?), that’d be great.
Ola Tarkowska
@olatarkowska
May 17 16:04
looking
woudl be really cool if it would be possible to run cotnainer and say -e “add-update-site=http://plugin_site"
and it would install
:)
what you think?
Step 7/11 : RUN bash ImageJ.sh --headless --update add-update-site PTBIOP http://biop.epfl.ch/Fiji-Update
 ---> Running in 7791d6ae90a4
Error: Could not find or load main class net.imagej.updater.CommandLine
The command '/bin/sh -c bash ImageJ.sh --headless --update add-update-site PTBIOP http://biop.epfl.ch/Fiji-Update' returned a non-zero code: 1
how .sh accept add-update-site ?
Curtis Rueden
@ctrueden
May 17 16:09
The only update sites for Fiji Docker images should be ImageJ, Fiji and Java-8. We could make images for other combos, but it quickly gets crazy.
We could make a “fiji + kitchen sink” that includes a bunch of vetted ones, but I feel it would be breakage-prone.
Ola Tarkowska
@olatarkowska
May 17 16:10
ok never mind it is installing now
Curtis Rueden
@ctrueden
May 17 16:10
@olatarkowska Hmm, that invocation looks reasonable to me.
Oh, you got it working?
Ola Tarkowska
@olatarkowska
May 17 16:10
RUN /fiji/Fiji.app/ImageJ-linux64 --headless --update add-update-site
Curtis Rueden
@ctrueden
May 17 16:11
Yeah, going through the native launcher executable will be safest—it has all the features.
Ola Tarkowska
@olatarkowska
May 17 16:11
So somthing different with bash script
Curtis Rueden
@ctrueden
May 17 16:11
Whereas the ImageJ.sh is a complex shell script attempting to emulate a bunch of them...
Ola Tarkowska
@olatarkowska
May 17 16:11
should /fiji/Fiji.app/ImageJ-linux64 in the container be launched in CMD ?
I need to admit that Entrypoint is bad idea
Curtis Rueden
@ctrueden
May 17 16:12
That said, the ImageJ.sh script does support --update. It looks like a classpath issue, with that error about not finding the right main class.
Ola Tarkowska
@olatarkowska
May 17 16:12
possible, but it is harder to override
but that is up to you guys
Curtis Rueden
@ctrueden
May 17 16:14
Regarding the error about expecting '!', found '@' the problem is that you are invoking an ImageJ1 macro with the ImageJ1-style -macro flag, which does not understand ImageJ2 script parameters.
You need to run the macro the ImageJ2 way.
Ola Tarkowska
@olatarkowska
May 17 16:14
awesome
--->We will try to work on 1 wells in parallel and 41 extra threads to process your data

Processing Well [1, 2]
Ola Tarkowska
@olatarkowska
May 17 16:15
thanks @ctrueden will have a look at that
anyway works now
it was a problem of wrong version I suppose
Josh Moore
@joshmoore
May 17 16:21
(Thanks both. I’ll follow up on the above when I’m back. @ctrueden : if you get a chance, can you explain what the benefits of ImageJ.sh are?)
Curtis Rueden
@ctrueden
May 17 16:21
The main benefit of ImageJ.sh is: it works on POSIX-friendly systems that don't have a native ImageJ launcher executable. :-)
But if you have the native executable handy, I see no advantage.
Prime example: Raspberry Pi systems.
We could enhance the ImageJ Launcher code to run on Pi, but it is a non-trivial endeavor.
One other possible advantage, I suppose: the ImageJ.sh script ultimately invokes java, so if there is some weirdness with the ImageJ native executable linking to libjvm, and it also fails to "fall back to system Java" (i.e., call java or javaw) for some reason, then the shell script is a potential way forward.
Josh Moore
@joshmoore
May 17 16:36
None of which sounds like it holds for Docker. Cool, thanks.
Curtis Rueden
@ctrueden
May 17 19:50

@frauzufall @HedgehogCode I ran across this in my notes from last December:

Plan for TensorFlow: ship native libs both CPU and GPU in lib/<platform> directly; load them with System.loadLibrary so that TensorFlow Java library is already happy. Name libtensorflow_gpu for the GPU version; load that by default first; if it fails, load CPU. UNLESS! a system property is set to "cpu" then use that preferentially.

Did that work get completed? Or, does it make sense to file an issue in imagej/imagej-tensorflow so we don't forget the details?

(Also relevant is this link.)
Also "TensorFlow update sites and reconcile to new TF version -- add tests"
Josh Moore
@joshmoore
May 17 21:04
Does anyone recognize this type of error:
fiji-win32.exe done
[ERROR] Error updating
java.lang.RuntimeException: Incorrect file size for /home/fiji/Fiji.app/fiji-win32.exe: 964 (expected 382976)
    at net.imagej.updater.Installer.verify(Installer.java:293)
    at net.imagej.updater.Installer$VerifyFiles.itemDone(Installer.java:270)
The other error I’ve seen is:
Downloading... (fiji-win32.exe)
[ERROR] Error updating
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
Curtis Rueden
@ctrueden
May 17 21:37
@joshmoore Hmm, nope, but that is unfortunate.
Looks like networking issues though.
@StephanPreibisch @axtimwalde Yay, fixing the multiview-reconstruction build was easy! PreibischLab/multiview-reconstruction@1e17e5b
Just gotta tell Travis to use xenial and install openjfx. No need to pin either because xenial ships the right version already.
I'll go through the other JavaFX-enabled projects in the Fiji ecosystem and fix them analogously.
But now: gotta run catch a bus. TTYAL
Deborah Schmidt
@frauzufall
May 17 23:01
@ctrueden I completed the work on tensorflow/NativeLibrary.javato enable loading the GPU if possible and the CPU as fallback but never created a PR there... :( It's on my list though..
Also on my list: a command that downloads a specific GPU TF Version into your Fiji lib folder, depending on OS and CUDA version (maybe showing a list and the user can pick what fits the system if CUDA version is not detectable). This would avoid creating an update site for each version and add some support in choosing the right one.