MESmerize/community

Kushal Kolar

@kushalkolar

Yea, you can't make a movie from the max projection

But you can export the movie itself

@katherinekelly2012 click the same button but not in the max projection and you should be able to export

Kushal Kolar

@kushalkolar

you will get an export window like this

katherinekelly2012

@katherinekelly2012

Ok do you know why I would be getting this error: AssertionError: Unknown encoder extension? I've installed ffmpeg and x264. I previously thought it was functioning because I got the popup but when I go further it throws the previously mentioned error.

Kushal Kolar

@kushalkolar

@katherinekelly2012 I need the entire traceback

And what OS are you on etc.

BTW the issue tracker on GitHub is better for these types of things :)

katherinekelly2012

@katherinekelly2012

Ok thanks I can post there

Pradeep Rajasekhar

@pr4deepr

Hi @kushalkolar
If I import ROIs from ImageJ, instead of CNMF how will it affect the downstream analysis?
Cheers
Pradeep

i.e, can I still do peak analysis, k-shape etc?

Kushal Kolar

@kushalkolar

there's no difference, everything gets packaged into a project sample regardless of the signal extraction method

Pradeep Rajasekhar

@pr4deepr

Thanks @kushalkolar
Also, is there a spatiotemporal analysis component as well? Perhaps am missing something obvious.

Kushal Kolar

@kushalkolar

what do you mean?

Pradeep Rajasekhar

@pr4deepr

i.e., using coordinates or locations of the cells and their peak features to figure out changes in intercellular communication?

Kushal Kolar

@kushalkolar

You can add any number of labels you want to ROIs, which then get stored as categorical variables and you could see if peak features or the kshape clusters that data fall into are different based on those categoricla variables

Pradeep Rajasekhar

@pr4deepr

Maybe its the terminology I use. I should rephrase it to I am looking at correlations in activity between cells, so it still applies?

Kushal Kolar

@kushalkolar

correlation in activity is a different thing

Pradeep Rajasekhar

@pr4deepr

ahh

Pradeep Rajasekhar

@pr4deepr

So with the correlation we are using the temporal component, right?
Is it possible to add the spatial component to it as well?

Kushal Kolar

@kushalkolar

you mean ROI tags?

If you tag ROIs and then use the cross correlation GUI you can see the corresponding ROI tags in the plot http://docs.mesmerizelab.org/en/master/_images/cross_cor.gif

Eric Thomson

@EricThomson

I have many users that now want to use mesmerize, and we are wondering what the kind of minimal requirements are? I assume similar to caiman. I know caiman doesn't require NVidia GPU but does benefit from high RAM, high core count, good CPU, etc.. Is there a kind of "minimum suggested build" for mesmerize? There are people with gov't issued laptops that are not amazing. In terms of RAM it seems 64 is good can we get away with 32?

Eric Thomson

@EricThomson

(We are doing cnmf-e so don't use cnn based component exclusion so no need for gpu)

Kushal Kolar

@kushalkolar

@EricThomson Yup in general the same requirements as Caiman if you're using those modules.

CNMFE is more resource intensive than CNMF. If your imaging resolution is ~512x512 and your movies are shorter than 10 mins you might be able to get away with 32GB or less. For longer movies I think it's recommended to use CNMFE online anyways. Faster RAM and quad channel RAM also seems to help for both CNMFE and motion correction (quad channel is usually only availalbe on Xeon and Threadripper I think).

For certain downstream things like computing Earth Mover's Distances like in our papers, very high thread count (64+) is necessary for large datasets (thousands of traces) and the computation will take days. Similar for kshape clustering.

Eric Thomson

@EricThomson

Thanks that's really helpful I will take this into consideration

Eric Thomson

@EricThomson

To run caiman motion correction in mesmerize without the piecewise motion correction, should it be enough to check the Use motion correction kwargs box and manually add in pw_rigid=False, and then just set the rigid correction params I want (and ignore the Elastic correction params)?

Kushal Kolar

@kushalkolar

Nope, I haven't added the option to perform just the rigid correction, is there a usecase where non-rigid doesn't work?

Eric Thomson

@EricThomson

There are cases where rigid is sufficient. Since it is a lot faster it is preferable in those cases (e.g., we have animals under anesthesia where there is not much movement, and rigid works well).

Eric Thomson

@EricThomson

To "delete" a project (e.g., a test project I was using to play with) I can just delete its associated directory? Or I guess my question is: is this the recommended way? :)

Kushal Kolar

@kushalkolar

Yes, and yup that's the recommended way

Irene Ayuso

@AyusoAa

Hi all! I am trying to use the Mesmerize VM. I successfully installed it and mesmerize is in the venvs folder. However, when I double click in the desktop icon, mesmerize doe snot start. A terminal window opens, no message pops up, and the it disappears. Mesmerize also does not start when typing mesmerize in the terminal. Did anyone encounter such issues? Cheers

Kushal Kolar

@kushalkolar

@AyusoAa When you say you installed it, do you mean that you imported the VM? You don't need to install Mesmerize within the VM, Mesmerize is pre-installed.

Irene Ayuso

@AyusoAa

Yes, I meant, I imported it. I did not install mesmerize.

Kushal Kolar

@kushalkolar

Can you provide screenshots and all the details for reproducing it? You're welcome to post an issue on the repo

Irene Ayuso

@AyusoAa

that's the terminal window that shows up, and then, dissapears, when I double click at the mesmerize icon. Alright, I will post an issue on the repo with all the details

Serapio M. Baca

@smbaca

Got mesmerize installed onto an ubuntu linux box but can't get caiman to install... any thoughts on this error message? Installing collected packages: caiman
Running setup.py develop for caiman
ERROR: Command errored out with exit status 1:
command: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /home/serapio/Desktop/mesmer/CaImAn/
Complete output (25 lines):
running develop
running egg_info
creating caiman.egg-info
writing caiman.egg-info/PKG-INFO
writing dependency_links to caiman.egg-info/dependency_links.txt
writing top-level names to caiman.egg-info/top_level.txt
writing manifest file 'caiman.egg-info/SOURCES.txt'
reading manifest file 'caiman.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'caiman.egg-info/SOURCES.txt'
running build_ext
building 'caiman.source_extraction.cnmf.oasis' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/caiman
creating build/temp.linux-x86_64-3.6/caiman/source_extraction
creating build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf
/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -I/home/serapio/Desktop/mesmer/lib/python3.6/site-packages/numpy/core/include -I/home/serapio/Desktop/mesmer/include -I/usr/include/python3.6m -c caiman/source_extraction/cnmf/oasis.cpp -o build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf/oasis.o
In file included from /usr/include/python3.6m/Python.h:8:0,
from caiman/source_extraction/cnmf/oasis.cpp:26:
/usr/include/python3.6m/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.6m/pyconfig.h: No such file or directory

 #  include <x86_64-linux-gnu/python3.6m/pyconfig.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1
----------------------------------------

ERROR: Command errored out with exit status 1: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Kushal Kolar

@kushalkolar

@smbaca Can you please post an issue on the repo? It's easier to read when you put it in a code block. Thanks!

1 reply

Irene Ayuso

@AyusoAa

Hi all! I would like to ask for help with my mesmerize project. With a colleague, we're trying to set up the mesmerize pipeline in the lab, but we're encountering a number of issues. We're dealing with 1p recordings in tiff format, with their respective json files and 1D behavioral files. We have managed to run motion correction and do CNMF-E, and as well, we've managed to import our stimulus map containing the behavior file. However (1) DF/F is not calculated, nor displayed. (2) We are not managing to plot the data correlated to our stimuli (PSTH). (3) When trying to export the zscore values associated with each ROI in csv format for plotting elsewhere, we get a file which contains characters instead of values. Shall I provide more details here, or shall I post an issue in github? for now, see avobe. Any help is very much appreciated :D

Kushal Kolar

@kushalkolar

This node can give you df/f with 1p data http://docs.mesmerizelab.org/en/master/user_guides/flowchart/nodes.html#staticdfofo

Kushal Kolar

@kushalkolar

@AyusoAa and the tutorial series on youtube has instructions for doing stimulus/behavior mapped analysis.

Irene Ayuso

@AyusoAa

@kushalkolar thanks! managed the DF/F, and watched again all the tutorial series. I still don't manage to export zcore or df/f data for each ROI as csv files. Help with that would be super welcome. Thanks in advance!

Kushal Kolar

@kushalkolar

@AyusoAa You can use the Save node to save the data stream at any point in the flowchart. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/nodes.html#save

You can then use the Transmission.from_hdf5 to load the data and access the pandas dataframe for any further analysis. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/overview.html#transmission-files

Michelle Jin

@mjin1812

Hi Kushal! This software is absolutely amazing and thank you so much for all the time and effort you put into crafting it. I'm currently piloting using mesmerize on a 1-p dataset to use cnmfe. I'm also interested in exporting the processed cnmfe data as a pickle file to potentially analyze separately. I just have a quick questions about something I noticed: After exporting and reading the saved file, I saw there was a discrepancy in the fps metadata I set for this dataset and the fr stored in the data dictionary in the parameters of the cnmf_data_dict. I was wondering which frame rate is actually being used during cnmfe?

Kushal Kolar

@kushalkolar

@mjin1812 Eric Thomson made a very nice notebook that goes through how to explore/use the cnmfe hdf5 files that are produced through the mesmerize batch manager:
https://github.com/kushalkolar/MESmerize/blob/master/notebooks/demo_batch_ingestion.ipynb

@mjin1812 for the fps discrepancy that's weird, I'm assuming you're using the GUI to set the CNMFE params? The GUI fetches the fps from the current viewer meta data and sets it as fr. If you are using the batch manager through the mesmerize scripting interface you probably have to set the fr manually for each file if it differs.

Where communities thrive

People

Repo info

Activity