These are chat archives for thunder-project/thunder

30th
Sep 2016
mior12
@mior12
Sep 30 2016 11:40
Hello everyone!
I want to load nifti data with thunder. Is it possible?
Dan Kessler
@dankessler
Sep 30 2016 12:49
@mior12 I don't know if it's natively possible with thunder, but if you use nibabel it's pretty easy to address the contents of the nifti file as a numpy array
mior12
@mior12
Sep 30 2016 12:55
@dankessler Can i use nibabel with cluster? I have 10 TB of nifti data, because i need use cluster with spark
Dan Kessler
@dankessler
Sep 30 2016 12:56
@mior12 good question. Here's a thread that discusses nifti files in thunder: https://groups.google.com/forum/#!topic/thunder-user/z1RyeDWABKY
mior12
@mior12
Sep 30 2016 13:04
@dankessler i looked it and, as I understand, data transform from nifti format to binary format and after analyze. I don't like it.
alexandrelaborde
@AlexandreLaborde
Sep 30 2016 13:47

@boazmohar OK so the error is:
File "<stdin>", line 1, in <module> File "C:\Users\alex_\Anaconda3\lib\site-packages\thunder\images\readers.py", line 488, in fromexample data = fromtif(path=path, npartitions=1, engine=engine) File "C:\Users\alex_\Anaconda3\lib\site-packages\thunder\images\readers.py", line 394, in fromtif labels=labels, engine=engine, credentials=credentials) File "C:\Users\alex_\Anaconda3\lib\site-packages\thunder\images\readers.py", line 199, in frompath recursive=recursive, npartitions=npartitions) File "C:\Users\alex_\Anaconda3\lib\site-packages\thunder\readers.py", line 372, in read raise FileNotFoundError("No objects found for '%s'" % path) thunder.readers.FileNotFoundError: No objects found for 's3n://thunder-sample-data/images/fish\*.tif'

This is what you get when running thunder.images.fromexample('fish') . I had the same problem when load the series example so I went in an changed the file thunder/series/readers.py from for key in bucket.list(os.path.join('series', name) + '/'): to for key in bucket.list('series/' + name + '/'):

My thunder version is 1.4.2

@boazmohar I forgot to mention that I am on Windows but you probably realized that already, that might be the reason why this fails.
Jeremy Freeman
@freeman-lab
Sep 30 2016 15:15
thanks @AlexandreLaborde definitely looks like a good old windows specific path issue!
would be awesome if you have a PR that fixes for Windows without breaking OS X / Linux
alexandrelaborde
@AlexandreLaborde
Sep 30 2016 15:22
Ok I will do that :)
Jeremy Freeman
@freeman-lab
Sep 30 2016 16:03
great thanks!