Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 10 15:20
    Blastorios closed #77
  • Jan 10 15:20
    Blastorios commented #77
  • Jan 10 15:19
    Blastorios opened #79
  • Dec 24 2021 23:02
    kushalkolar commented #77
  • Dec 24 2021 23:01

    kushalkolar on new-readme

    (compare)

  • Dec 16 2021 02:31
    kushalkolar closed #78
  • Dec 16 2021 02:31

    kushalkolar on master

    updated tifffile.imsave to save… Merge pull request #78 from Eri… (compare)

  • Dec 16 2021 02:31
    kushalkolar commented #78
  • Dec 15 2021 15:30
    EricThomson opened #78
  • Nov 15 2021 19:08
    Blastorios synchronize #77
  • Nov 15 2021 18:56
    Blastorios synchronize #77
  • Nov 15 2021 17:59
    Blastorios synchronize #77
  • Nov 15 2021 15:53
    TundeSzungyi closed #75
  • Nov 15 2021 15:53
    TundeSzungyi commented #75
  • Nov 15 2021 12:10
    Blastorios opened #77
  • Nov 14 2021 21:29
    kushalkolar commented #76
  • Nov 14 2021 21:08
    kushalkolar commented #75
  • Nov 14 2021 14:25
    TundeSzungyi edited #75
  • Nov 14 2021 14:24
    TundeSzungyi edited #75
  • Nov 14 2021 14:23
    TundeSzungyi edited #76
Kushal Kolar
@kushalkolar
you will get an export window like this
katherinekelly2012
@katherinekelly2012
Ok do you know why I would be getting this error: AssertionError: Unknown encoder extension? I've installed ffmpeg and x264. I previously thought it was functioning because I got the popup but when I go further it throws the previously mentioned error.
Kushal Kolar
@kushalkolar
@katherinekelly2012 I need the entire traceback
And what OS are you on etc.
BTW the issue tracker on GitHub is better for these types of things :)
katherinekelly2012
@katherinekelly2012
Ok thanks I can post there
Pradeep Rajasekhar
@pr4deepr
Hi @kushalkolar
If I import ROIs from ImageJ, instead of CNMF how will it affect the downstream analysis?
Cheers
Pradeep
i.e, can I still do peak analysis, k-shape etc?
Kushal Kolar
@kushalkolar
there's no difference, everything gets packaged into a project sample regardless of the signal extraction method
Pradeep Rajasekhar
@pr4deepr
Thanks @kushalkolar
Also, is there a spatiotemporal analysis component as well? Perhaps am missing something obvious.
Kushal Kolar
@kushalkolar
what do you mean?
Pradeep Rajasekhar
@pr4deepr
i.e., using coordinates or locations of the cells and their peak features to figure out changes in intercellular communication?
Kushal Kolar
@kushalkolar
You can add any number of labels you want to ROIs, which then get stored as categorical variables and you could see if peak features or the kshape clusters that data fall into are different based on those categoricla variables
Pradeep Rajasekhar
@pr4deepr
Maybe its the terminology I use. I should rephrase it to I am looking at correlations in activity between cells, so it still applies?
Kushal Kolar
@kushalkolar
correlation in activity is a different thing
Pradeep Rajasekhar
@pr4deepr
ahh
Pradeep Rajasekhar
@pr4deepr
So with the correlation we are using the temporal component, right?
Is it possible to add the spatial component to it as well?
Kushal Kolar
@kushalkolar
you mean ROI tags?
If you tag ROIs and then use the cross correlation GUI you can see the corresponding ROI tags in the plot http://docs.mesmerizelab.org/en/master/_images/cross_cor.gif
Eric Thomson
@EricThomson
I have many users that now want to use mesmerize, and we are wondering what the kind of minimal requirements are? I assume similar to caiman. I know caiman doesn't require NVidia GPU but does benefit from high RAM, high core count, good CPU, etc.. Is there a kind of "minimum suggested build" for mesmerize? There are people with gov't issued laptops that are not amazing. In terms of RAM it seems 64 is good can we get away with 32?
Eric Thomson
@EricThomson
(We are doing cnmf-e so don't use cnn based component exclusion so no need for gpu)
Kushal Kolar
@kushalkolar

@EricThomson Yup in general the same requirements as Caiman if you're using those modules.

CNMFE is more resource intensive than CNMF. If your imaging resolution is ~512x512 and your movies are shorter than 10 mins you might be able to get away with 32GB or less. For longer movies I think it's recommended to use CNMFE online anyways. Faster RAM and quad channel RAM also seems to help for both CNMFE and motion correction (quad channel is usually only availalbe on Xeon and Threadripper I think).

For certain downstream things like computing Earth Mover's Distances like in our papers, very high thread count (64+) is necessary for large datasets (thousands of traces) and the computation will take days. Similar for kshape clustering.

Eric Thomson
@EricThomson
Thanks that's really helpful I will take this into consideration
Eric Thomson
@EricThomson
To run caiman motion correction in mesmerize without the piecewise motion correction, should it be enough to check the Use motion correction kwargs box and manually add in pw_rigid=False, and then just set the rigid correction params I want (and ignore the Elastic correction params)?
Kushal Kolar
@kushalkolar
Nope, I haven't added the option to perform just the rigid correction, is there a usecase where non-rigid doesn't work?
Eric Thomson
@EricThomson
There are cases where rigid is sufficient. Since it is a lot faster it is preferable in those cases (e.g., we have animals under anesthesia where there is not much movement, and rigid works well).
Eric Thomson
@EricThomson
To "delete" a project (e.g., a test project I was using to play with) I can just delete its associated directory? Or I guess my question is: is this the recommended way? :)
Kushal Kolar
@kushalkolar
Yes, and yup that's the recommended way
Irene Ayuso
@AyusoAa
Hi all! I am trying to use the Mesmerize VM. I successfully installed it and mesmerize is in the venvs folder. However, when I double click in the desktop icon, mesmerize doe snot start. A terminal window opens, no message pops up, and the it disappears. Mesmerize also does not start when typing mesmerize in the terminal. Did anyone encounter such issues? Cheers
Kushal Kolar
@kushalkolar
@AyusoAa When you say you installed it, do you mean that you imported the VM? You don't need to install Mesmerize within the VM, Mesmerize is pre-installed.
Irene Ayuso
@AyusoAa
Yes, I meant, I imported it. I did not install mesmerize.
Kushal Kolar
@kushalkolar
Can you provide screenshots and all the details for reproducing it? You're welcome to post an issue on the repo
Irene Ayuso
@AyusoAa
image.png
that's the terminal window that shows up, and then, dissapears, when I double click at the mesmerize icon. Alright, I will post an issue on the repo with all the details
Serapio M. Baca
@smbaca

Got mesmerize installed onto an ubuntu linux box but can't get caiman to install... any thoughts on this error message? Installing collected packages: caiman
Running setup.py develop for caiman
ERROR: Command errored out with exit status 1:
command: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /home/serapio/Desktop/mesmer/CaImAn/
Complete output (25 lines):
running develop
running egg_info
creating caiman.egg-info
writing caiman.egg-info/PKG-INFO
writing dependency_links to caiman.egg-info/dependency_links.txt
writing top-level names to caiman.egg-info/top_level.txt
writing manifest file 'caiman.egg-info/SOURCES.txt'
reading manifest file 'caiman.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'caiman.egg-info/SOURCES.txt'
running build_ext
building 'caiman.source_extraction.cnmf.oasis' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/caiman
creating build/temp.linux-x86_64-3.6/caiman/source_extraction
creating build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf
/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -I/home/serapio/Desktop/mesmer/lib/python3.6/site-packages/numpy/core/include -I/home/serapio/Desktop/mesmer/include -I/usr/include/python3.6m -c caiman/source_extraction/cnmf/oasis.cpp -o build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf/oasis.o
In file included from /usr/include/python3.6m/Python.h:8:0,
from caiman/source_extraction/cnmf/oasis.cpp:26:
/usr/include/python3.6m/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.6m/pyconfig.h: No such file or directory

 #  include <x86_64-linux-gnu/python3.6m/pyconfig.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1
----------------------------------------

ERROR: Command errored out with exit status 1: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Kushal Kolar
@kushalkolar
@smbaca Can you please post an issue on the repo? It's easier to read when you put it in a code block. Thanks!
1 reply
Irene Ayuso
@AyusoAa
image.png
Hi all! I would like to ask for help with my mesmerize project. With a colleague, we're trying to set up the mesmerize pipeline in the lab, but we're encountering a number of issues. We're dealing with 1p recordings in tiff format, with their respective json files and 1D behavioral files. We have managed to run motion correction and do CNMF-E, and as well, we've managed to import our stimulus map containing the behavior file. However (1) DF/F is not calculated, nor displayed. (2) We are not managing to plot the data correlated to our stimuli (PSTH). (3) When trying to export the zscore values associated with each ROI in csv format for plotting elsewhere, we get a file which contains characters instead of values. Shall I provide more details here, or shall I post an issue in github? for now, see avobe. Any help is very much appreciated :D
Kushal Kolar
@kushalkolar
Kushal Kolar
@kushalkolar
@AyusoAa and the tutorial series on youtube has instructions for doing stimulus/behavior mapped analysis.
Irene Ayuso
@AyusoAa
@kushalkolar thanks! managed the DF/F, and watched again all the tutorial series. I still don't manage to export zcore or df/f data for each ROI as csv files. Help with that would be super welcome. Thanks in advance!
Kushal Kolar
@kushalkolar
@AyusoAa You can use the Save node to save the data stream at any point in the flowchart. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/nodes.html#save
You can then use the Transmission.from_hdf5 to load the data and access the pandas dataframe for any further analysis. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/overview.html#transmission-files
Michelle Jin
@mjin1812
Hi Kushal! This software is absolutely amazing and thank you so much for all the time and effort you put into crafting it. I'm currently piloting using mesmerize on a 1-p dataset to use cnmfe. I'm also interested in exporting the processed cnmfe data as a pickle file to potentially analyze separately. I just have a quick questions about something I noticed: After exporting and reading the saved file, I saw there was a discrepancy in the fps metadata I set for this dataset and the fr stored in the data dictionary in the parameters of the cnmf_data_dict. I was wondering which frame rate is actually being used during cnmfe?
Kushal Kolar
@kushalkolar
@mjin1812 Eric Thomson made a very nice notebook that goes through how to explore/use the cnmfe hdf5 files that are produced through the mesmerize batch manager:
https://github.com/kushalkolar/MESmerize/blob/master/notebooks/demo_batch_ingestion.ipynb
@mjin1812 for the fps discrepancy that's weird, I'm assuming you're using the GUI to set the CNMFE params? The GUI fetches the fps from the current viewer meta data and sets it as fr. If you are using the batch manager through the mesmerize scripting interface you probably have to set the fr manually for each file if it differs.
Michelle Jin
@mjin1812
Thanks for the fast response! I will definitely check out the notebook. For reference here is a link to a screenshot showing the difference in frame rates https://imgur.com/a/HMLush2
perhaps I just entered the metadata wrong. I followed the instructions only on entering it manually
Kushal Kolar
@kushalkolar
what method are you using to create the cnmfe batch items?
Michelle Jin
@mjin1812
I used the batch manager gui