Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Nov 15 19:08
    Blastorios synchronize #77
  • Nov 15 18:56
    Blastorios synchronize #77
  • Nov 15 17:59
    Blastorios synchronize #77
  • Nov 15 15:53
    TundeSzungyi closed #75
  • Nov 15 15:53
    TundeSzungyi commented #75
  • Nov 15 12:10
    Blastorios opened #77
  • Nov 14 21:29
    kushalkolar commented #76
  • Nov 14 21:08
    kushalkolar commented #75
  • Nov 14 14:25
    TundeSzungyi edited #75
  • Nov 14 14:24
    TundeSzungyi edited #75
  • Nov 14 14:23
    TundeSzungyi edited #76
  • Nov 14 14:19
    TundeSzungyi labeled #76
  • Nov 14 14:19
    TundeSzungyi assigned #76
  • Nov 14 14:19
    TundeSzungyi opened #76
  • Nov 14 13:42
    TundeSzungyi labeled #75
  • Nov 14 13:42
    TundeSzungyi assigned #75
  • Nov 14 13:42
    TundeSzungyi opened #75
  • Nov 12 15:43

    kushalkolar on master

    Update README.md (compare)

  • Nov 12 06:04

    kushalkolar on v0.8.0

    (compare)

  • Nov 12 06:01

    kushalkolar on master

    Update README.md (compare)

Kushal Kolar
@kushalkolar
Nope, I haven't added the option to perform just the rigid correction, is there a usecase where non-rigid doesn't work?
Eric Thomson
@EricThomson
There are cases where rigid is sufficient. Since it is a lot faster it is preferable in those cases (e.g., we have animals under anesthesia where there is not much movement, and rigid works well).
Eric Thomson
@EricThomson
To "delete" a project (e.g., a test project I was using to play with) I can just delete its associated directory? Or I guess my question is: is this the recommended way? :)
Kushal Kolar
@kushalkolar
Yes, and yup that's the recommended way
Irene Ayuso
@AyusoAa
Hi all! I am trying to use the Mesmerize VM. I successfully installed it and mesmerize is in the venvs folder. However, when I double click in the desktop icon, mesmerize doe snot start. A terminal window opens, no message pops up, and the it disappears. Mesmerize also does not start when typing mesmerize in the terminal. Did anyone encounter such issues? Cheers
Kushal Kolar
@kushalkolar
@AyusoAa When you say you installed it, do you mean that you imported the VM? You don't need to install Mesmerize within the VM, Mesmerize is pre-installed.
Irene Ayuso
@AyusoAa
Yes, I meant, I imported it. I did not install mesmerize.
Kushal Kolar
@kushalkolar
Can you provide screenshots and all the details for reproducing it? You're welcome to post an issue on the repo
Irene Ayuso
@AyusoAa
image.png
that's the terminal window that shows up, and then, dissapears, when I double click at the mesmerize icon. Alright, I will post an issue on the repo with all the details
Serapio M. Baca
@smbaca

Got mesmerize installed onto an ubuntu linux box but can't get caiman to install... any thoughts on this error message? Installing collected packages: caiman
Running setup.py develop for caiman
ERROR: Command errored out with exit status 1:
command: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: /home/serapio/Desktop/mesmer/CaImAn/
Complete output (25 lines):
running develop
running egg_info
creating caiman.egg-info
writing caiman.egg-info/PKG-INFO
writing dependency_links to caiman.egg-info/dependency_links.txt
writing top-level names to caiman.egg-info/top_level.txt
writing manifest file 'caiman.egg-info/SOURCES.txt'
reading manifest file 'caiman.egg-info/SOURCES.txt'
adding license file 'LICENSE.txt'
writing manifest file 'caiman.egg-info/SOURCES.txt'
running build_ext
building 'caiman.source_extraction.cnmf.oasis' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/caiman
creating build/temp.linux-x86_64-3.6/caiman/source_extraction
creating build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf
/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc -DNDEBUG -g -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -I/home/serapio/Desktop/mesmer/lib/python3.6/site-packages/numpy/core/include -I/home/serapio/Desktop/mesmer/include -I/usr/include/python3.6m -c caiman/source_extraction/cnmf/oasis.cpp -o build/temp.linux-x86_64-3.6/caiman/source_extraction/cnmf/oasis.o
In file included from /usr/include/python3.6m/Python.h:8:0,
from caiman/source_extraction/cnmf/oasis.cpp:26:
/usr/include/python3.6m/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.6m/pyconfig.h: No such file or directory

 #  include <x86_64-linux-gnu/python3.6m/pyconfig.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
error: command '/opt/anaconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1
----------------------------------------

ERROR: Command errored out with exit status 1: /home/serapio/Desktop/mesmer/bin/python3.6 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"'; file='"'"'/home/serapio/Desktop/mesmer/CaImAn/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

Kushal Kolar
@kushalkolar
@smbaca Can you please post an issue on the repo? It's easier to read when you put it in a code block. Thanks!
1 reply
Irene Ayuso
@AyusoAa
image.png
Hi all! I would like to ask for help with my mesmerize project. With a colleague, we're trying to set up the mesmerize pipeline in the lab, but we're encountering a number of issues. We're dealing with 1p recordings in tiff format, with their respective json files and 1D behavioral files. We have managed to run motion correction and do CNMF-E, and as well, we've managed to import our stimulus map containing the behavior file. However (1) DF/F is not calculated, nor displayed. (2) We are not managing to plot the data correlated to our stimuli (PSTH). (3) When trying to export the zscore values associated with each ROI in csv format for plotting elsewhere, we get a file which contains characters instead of values. Shall I provide more details here, or shall I post an issue in github? for now, see avobe. Any help is very much appreciated :D
Kushal Kolar
@kushalkolar
Kushal Kolar
@kushalkolar
@AyusoAa and the tutorial series on youtube has instructions for doing stimulus/behavior mapped analysis.
Irene Ayuso
@AyusoAa
@kushalkolar thanks! managed the DF/F, and watched again all the tutorial series. I still don't manage to export zcore or df/f data for each ROI as csv files. Help with that would be super welcome. Thanks in advance!
Kushal Kolar
@kushalkolar
@AyusoAa You can use the Save node to save the data stream at any point in the flowchart. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/nodes.html#save
You can then use the Transmission.from_hdf5 to load the data and access the pandas dataframe for any further analysis. http://docs.mesmerizelab.org/en/master/user_guides/flowchart/overview.html#transmission-files
Michelle Jin
@mjin1812
Hi Kushal! This software is absolutely amazing and thank you so much for all the time and effort you put into crafting it. I'm currently piloting using mesmerize on a 1-p dataset to use cnmfe. I'm also interested in exporting the processed cnmfe data as a pickle file to potentially analyze separately. I just have a quick questions about something I noticed: After exporting and reading the saved file, I saw there was a discrepancy in the fps metadata I set for this dataset and the fr stored in the data dictionary in the parameters of the cnmf_data_dict. I was wondering which frame rate is actually being used during cnmfe?
Kushal Kolar
@kushalkolar
@mjin1812 Eric Thomson made a very nice notebook that goes through how to explore/use the cnmfe hdf5 files that are produced through the mesmerize batch manager:
https://github.com/kushalkolar/MESmerize/blob/master/notebooks/demo_batch_ingestion.ipynb
@mjin1812 for the fps discrepancy that's weird, I'm assuming you're using the GUI to set the CNMFE params? The GUI fetches the fps from the current viewer meta data and sets it as fr. If you are using the batch manager through the mesmerize scripting interface you probably have to set the fr manually for each file if it differs.
Michelle Jin
@mjin1812
Thanks for the fast response! I will definitely check out the notebook. For reference here is a link to a screenshot showing the difference in frame rates https://imgur.com/a/HMLush2
perhaps I just entered the metadata wrong. I followed the instructions only on entering it manually
Kushal Kolar
@kushalkolar
what method are you using to create the cnmfe batch items?
Michelle Jin
@mjin1812
I used the batch manager gui
Or mean the cnmfe gui to add to the batch manager
For reference, I'm using mesmerize through the VM installation
Kushal Kolar
@kushalkolar
@mjin1812 just took a look, the GUI sets the framerate at the component evaluation stage which appears to be when CNMFE uses the framerate, though I don't know why this doesn't show up in the cnmf_data_dict
Michelle Jin
@mjin1812
Thanks for your help! Does the gui automatically set the frame rate using the metadata dictionary? If so, then maybe I’ll just go with that value
Kushal Kolar
@kushalkolar
@mjin1812 The GUI does set the fps but only at the evaluate_components() step which might be why it's not showing up in the cnmf_data_dict. Anyways I'll put in a fix so that it's set at the very beginning and then it should show up in cnmf_data_dict
Michelle Jin
@mjin1812
Hi Kushal! Can I clarify whether 'cnmf_idx' in the 'ROI_states' series of a transmission object dataframe is corresponding to the roi indices in a traditional cnmf estimates object?
Kushal Kolar
@kushalkolar
yes, it's the originating cnmf.estimates.idxvalue for that ROI
Michelle Jin
@mjin1812
Thank you!
Serapio M. Baca
@smbaca
Does anyone have any experience with using mesmerize on a cluster? We have to request number of cores, RAM, GPU or no but then we have a 10TB scratch space and can use a Desktop environment running CentOS below to access mesmerize. So far, I have been able to run the motion correction but then the cnmf-e fails with not being able to find a file.
I am using Eric Thomson's instructions on the windows installation so I know that it is not a problem with the tiff movie.
Kushal Kolar
@kushalkolar
@smbaca this will probably be easiest to solve if you post an issue with all the details :)
You will probably have to set a workdir and it will need to be mounted at the same location on the compute nodes and the submission machine, but if you post an issue I can help with the details
Serapio M. Baca
@smbaca
Ok, posted the error message in the other location. Program seems to want to create a file in /var/www/ood... and that is not an installed directory.
Michelle Jin
@mjin1812
Hi Kushal! I follow instructions online to export and open up a transmission object externally in python. I detected what I think may be an indexing issue in the transmission dataframe. The "_DFOF" traces and "_RAW_CURVE" traces aren't lining up for me. I plotted the calculated dfof and raw curves for three consecutive rois using the code below where t is the transmission object and n is my roi index:
ax.plot( t.df["_DFOF"].iloc[n]); ax.plot( t.df["_RAW_CURVE"].iloc[n]);
Here is the plot for the three consecutive rois: https://imgur.com/a/nHUJ5ly Orange = "_RAW_CURVE" blue = "_DFOF"
It looks like the DFOF traces in blue matches with the orange trace of the previous roi
Kushal Kolar
@kushalkolar
@mjin1812 I'm away until next Wednesday so I'll get back to you then.
Eric Thomson
@EricThomson
Congratulations on some great software and a great paper @kushalkolar !!!
Kushal Kolar
@kushalkolar
@mjin1812 Did manually remove ROIs in the Mesmerize ROI manager or from the cnmf.estimate object?
2 replies
Irene Ayuso
@AyusoAa
Hi Kushal, congratulations for the paper! I was wondering if it is possible that the MC output file get the name of the orinial file instead of a collection of letters and numbers. When trying a bunch of parameters for CNMF-E, It is very slow to have to check the match between the original filename and the mc filename everytime. Coud you give some tips on how to change that in the code?
Kushal Kolar
@kushalkolar
@mjin1812 I'm not sure what was causing the issue then in that one case but it seems like it only occurred once so I'm not too concerned.
@AyusoAa I'm not sure what you're trying to do. If you use the batch manager interface you don't need to know the uuids for each item to iterate over parameter variants.
WhereWolf72
@WhereWolf72
@kushalkolar Congrats on the paper! I'm having some trouble getting the metadata (specifically the frames per second) to be loaded properly to run CaImAn's CNMF-E. I am almost certain the problem is on my end, but I can't seem to troubleshoot it. The files were acquired via Nikon Elements, so they were originally .nd2 files. I manually added "fps = __" into the metadata for each as our Elements only uses frame acquisition rate (seconds per frame), and then converted to TIFFs via Elements. Finally, I have deinterleaved the two-channel images into just the channel for the calcium imaging using FIJI. In the end, none of the metadata loaders will recognize the image file as containing the fps metadata. Any help or advice would be greatly appreciated, thank you!