Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Davis Bennett
    @d-v-b
    i'm asking because it might be possible to do image compression using the tif format
    since you can specify a compression level when you save as tif
    the compressed tif stacks in my testing are ~40% the size of the original
    but they remain fully accessible to image readers, unlike zipped images
    Nikita Vladimirov
    @nvladimus
    about 30%
    is it loss-less compression?
    Davis Bennett
    @d-v-b
    i believe so
    Nikita Vladimirov
    @nvladimus
    actually, more like 35% for bz2
    Davis Bennett
    @d-v-b
    the tif compression tag allows lossy and lossless compression
    Nikita Vladimirov
    @nvladimus
    if it's lossless and 40%, this would be really good for us. But can Thunder load compressed TIFF files?
    Davis Bennett
    @d-v-b
    i need to test this too, but anything that loads tif files should be able to load compressed tif files
    ....i will test it if I can get space on the cluster :(
    Davis Bennett
    @d-v-b
    compressed tifs can be read by spark
    Nikita Vladimirov
    @nvladimus
    @d-v-b Davis, have you actually used your code snippet dat = tsc.loadImages(rawDir + 'TM*.tif', inputFormat='tif’) for tif file reading? It does not work in my case. Throws errors.
    Davis Bennett
    @d-v-b
    @nvladimus what kind of errors?
    Nikita Vladimirov
    @nvladimus
    ok, get ready
    ImportError: ("fromMultipageTif requires a successful 'from PIL import Image'; the PIL/pillow library appears to be missing or broken.", ImportError('/groups/ahrens/home/vladimirovn/.python-eggs/Pillow-3.0.0-py2.7-linux-x86_64.egg-tmp/PIL/_imaging.so: undefined symbol: TIFFSetWarningHandlerExt',))
    Davis Bennett
    @d-v-b
    here's a snippet from my code right now:
    dat = tsc.loadImages(rawDirs[curExp] + 'TM*.tif', inputFormat='tif', npartitions=numFrames).subsample((1,1,1))
    what version of thunder are you using?
    Nikita Vladimirov
    @nvladimus
    the latest, I believe. I updated 2 weeks ago or so.
    Davis Bennett
    @d-v-b
    @freeman-lab any ideas what might be happening here?
    i can load multipage tifs and nikita cannot
    Jeremy Freeman
    @freeman-lab
    ah! i might know the error, @nvladimus can you type cat ~/.bashrc
    and print the result here
    Nikita Vladimirov
    @nvladimus
    # .bashrc
    
    # Source global definitions
    if [ -f /etc/bashrc ]; then
            . /etc/bashrc
    fi
    
    # User specific aliases and functions
    export PATH="$PATH:/usr/local/matlab-2010a/bin"
    export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/matlab-2010a/bin/glnxa64:/usr/\
    local/matlab-2010a/runtime/glnxa64:$LD_LIBRARY_PATH
    export MCR_INHIBIT_CTF_LOCK=1
    Jeremy Freeman
    @freeman-lab
    ok great, so now open that file in a text editor, e.g. emacs ~/.bashrc
    and comment out the last four lines
    (add a # in front)
    the ones that start with export ...
    Nikita Vladimirov
    @nvladimus
    ok. Then restart everything?
    Jeremy Freeman
    @freeman-lab
    should be fine just to relaunch python / thunder
    to figure out if it's fixed, it's enough to time ipython
    then from PIL import Image
    if that fails, it's the same error you were seeing above
    if it succeeds, you should be good to go!
    Nikita Vladimirov
    @nvladimus
    yes, it does fail after I changed the bash profile:
    ```
    from PIL import Image
    
    ---------------------------------------------------------------------------
    ImportError                               Traceback (most recent call last)
    <ipython-input-3-0f6709e38f49> in <module>()
    ----> 1 from PIL import Image
    
    build/bdist.linux-x86_64/egg/PIL/Image.py in <module>()
    
    build/bdist.linux-x86_64/egg/PIL/_imaging.py in <module>()
    
    build/bdist.linux-x86_64/egg/PIL/_imaging.py in __bootstrap__()
    
    ImportError: /groups/ahrens/home/vladimirovn/.python-eggs/Pillow-3.0.0-py2.7-linux-x86_64.egg-tmp/PIL/_imaging.so: undefined symbol: TIFFSetWarningHandlerExt
    Jeremy Freeman
    @freeman-lab
    ah you might need to source ~/.bash_profile
    to be safe, log out and log back in
    Nikita Vladimirov
    @nvladimus
    aha, now it worked
    it loaded TIFs, too. You rock, @freeman-lab
    Jeremy Freeman
    @freeman-lab
    great! :+1:
    don't ask why that fixed it =)
    Nikita Vladimirov
    @nvladimus
    I will not understand anything anyway. Just a magic, I understand.
    thank you!
    can I somehow swap dimension(0) and dimension(1) in the data? Importing TIFs changed X and Y dimensions in the data, from binary format.
    Davis Bennett
    @d-v-b
    I use im_data.applyValues(lambda v: v.swapaxes(0,1))
    this probably indicates a bug in the stack - tif conversion
    or a missing feature, at least
    Nikita Vladimirov
    @nvladimus
    the resulting TIFs are OK, from the standpoint of opening them in FiJi
    Davis Bennett
    @d-v-b
    so maybe it's an issue on the thunder side?