Hi Kushal, congratulations for the paper! I was wondering if it is possible that the MC output file get the name of the orinial file instead of a collection of letters and numbers. When trying a bunch of parameters for CNMF-E, It is very slow to have to check the match between the original filename and the mc filename everytime. Coud you give some tips on how to change that in the code?
@AyusoAa I'm not sure what you're trying to do. If you use the batch manager interface you don't need to know the uuids for each item to iterate over parameter variants.
@kushalkolar Congrats on the paper! I'm having some trouble getting the metadata (specifically the frames per second) to be loaded properly to run CaImAn's CNMF-E. I am almost certain the problem is on my end, but I can't seem to troubleshoot it. The files were acquired via Nikon Elements, so they were originally .nd2 files. I manually added "fps = __" into the metadata for each as our Elements only uses frame acquisition rate (seconds per frame), and then converted to TIFFs via Elements. Finally, I have deinterleaved the two-channel images into just the channel for the calcium imaging using FIJI. In the end, none of the metadata loaders will recognize the image file as containing the fps metadata. Any help or advice would be greatly appreciated, thank you!
@WhereWolf72 you can create a json file with the same name as the tiff file that has the structure of the "json_minimal" described here http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html#module-tifffile-metadata
2 replies
@danielmichaelvirga See this: http://docs.mesmerizelab.org/en/master/user_guides/viewer/overview.html#open-image
Find the mesmerize config.json file and change the "_MESMERIZE_WORKDIR" key.
if you're on Linux or Mac the config file is located in the .mesmerize dir in the home dir. ON windows it's in the user profile dir.
I successfully ran CNMFe for one of my samples as a test and want to optimize my parameters. However, now I am running into a pickling error that I'm not sure how to fix whenever I try to run CNMFe. Any advice would be appreciated. Here's then end of the error code (I can post the entire error if needed): multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f4f6cdd3da0>'. Reason: 'PicklingError("Can't pickle <class '_flapack.error'>: import of module '_flapack' failed",)'
2 replies
Hi, I just ran a small test batch through Mesmerize with good results so far (really loving the GUI and the video tutorials, excellent work). However my actual data has multiple Z-planes and despite being reasonably comfortable with the GUI, I cannot figure out how to get the TIFF file importer to recognize my TIFFs as 3D. I searched the docs for a template meta data file for 3d (such as the meta min provided for single plane imaging) and I couldn't find what the metadata should look like in order to format the json appropriately to get the importer to sort the planes correctly. At the moment, I'm using a ThorLabs scope which exports OME Tiffs along with an XML. In one of the video tutorials Kushal mentioned he would provide a link explaining how to add more meta data import methods but I didn't see the link or info about in the docs. Any help would be appreciated, thanks!
@cawarwick you don't need additional metadata to import 3D files, if the default
http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html
tzxy axes order isn't working you can enter the axes order that your data are in:http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html
6 replies
ok, so new issue. If I import the files in the default order tzxy, the image is rotated 90 degrees, which would suggest my files are tzyx and (256X512 (Y,X)) and using a custom order, when I ran the 3D CNMF, the ROIs appear to be rotated 90 degrees and don't align with the image. Did I do something out of the ordinary?
My post was not as clear as it could have been. The default order tzxy makes my image rotated 90 degrees, so I then instead used tzyx which did import it correctly as pictured in the screenshot (that is a horizontal orientation like a widescreen monitor), but the ROIs generated are rotated 90 degrees. For reference, I did not have this happen with a single Z-plane image, but it imported correctly with the default order of txy. I'm not sure why the multiplane image is different, they were both created in ImageJ from the same source file.
e.g. My first batches were with a simpler single-plane image and it ran exactly as I expected with the default order of txy (screenshot is from a single plane image that ran fine). When I then included the other Z-planes in the TIF the order had to be tzyx for it to be oriented correctly.
error in trying to load script editor: Windows OS:
Traceback (most recent call last):
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 141, in <lambda>
self.ui.actionScript_Editor.triggered.connect(lambda: self.run_module(script_editor.ModuleGUI))
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 299, in run_module
m = module_class(self, self._viewer)
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 27, in init
self.open_temp_script()
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 72, in open_temp_script
home_dir = os.environ['HOME']
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'HOME'
Traceback (most recent call last):
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 141, in <lambda>
self.ui.actionScript_Editor.triggered.connect(lambda: self.run_module(script_editor.ModuleGUI))
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 299, in run_module
m = module_class(self, self._viewer)
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 27, in init
self.open_temp_script()
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 72, in open_temp_script
home_dir = os.environ['HOME']
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'HOME'
I'm guessing this is a computer managed by the university or something?
In the console in the mesmerize welcome window can you try:
import os
os.environ['HOME'] = 'C:\\Users\\kates\\'
@kas5dv Windows is inconsistent with how it sets the
HOME env variable, if you don't want to do this each time you launch mesmerize you can HOME systemwide https://phoenixnap.com/kb/windows-set-environment-variable
1 reply
Hi everywhere there will be an in-person training event at CCN in NYC for Mesmerize & Caiman June 2-3. More information: https://indico.flatironinstitute.org/event/2800/
status: 0
output_info: Traceback (most recent call last):
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/mesmerize/viewer/modules/batch_run_modules/CNMFE.py", line 173, in run
cnm.fit(Y)
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/cnmf.py", line 627, in fit
self.update_temporal(Yr, use_init=False)
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/cnmf.py", line 903, in update_temporal
*self.params.get_group('temporal'))
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/temporal.py", line 212, in update_temporal_components
YrA = YA - AA.T.dot(Cin).T
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/scipy/sparse/base.py", line 359, in dot
return self other
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/scipy/sparse/base.py", line 516, in mul
raise ValueError('dimension mismatch')
ValueError: dimension mismatch
Y.shape: [1448, 600, 600]
processing_time: 11.938894434769947
output_info: Traceback (most recent call last):
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/mesmerize/viewer/modules/batch_run_modules/CNMFE.py", line 173, in run
cnm.fit(Y)
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/cnmf.py", line 627, in fit
self.update_temporal(Yr, use_init=False)
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/cnmf.py", line 903, in update_temporal
*self.params.get_group('temporal'))
File "/opt/caiman/CaImAn/caiman/source_extraction/cnmf/temporal.py", line 212, in update_temporal_components
YrA = YA - AA.T.dot(Cin).T
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/scipy/sparse/base.py", line 359, in dot
return self other
File "/opt/caiman/caimanvenv/lib/python3.6/site-packages/scipy/sparse/base.py", line 516, in mul
raise ValueError('dimension mismatch')
ValueError: dimension mismatch
Y.shape: [1448, 600, 600]
processing_time: 11.938894434769947
Hi, I just started working with mesmerize so it's probably an easy question. I'm trying to "add to project" but it keeps telling me to wait. Any tips?
@kushalkolar, thank you for the fast reply! I let it run overnight but no movement. It looks like it has to do with the metadata. I don't have a separate file. is there a way to add the fps manually?
We have a few more slots available for the CaImAn, Mesmerize & FIOLA workshop in June. It will be in-person in NYC. Register by this Sunday! https://indico.flatironinstitute.org/event/2800/
Dear Kushal
Thank you for the great work in sharing, documenting and troubleshooting MESmerize for the community. In our lab we are looking with great interest to the tools you developed. Unfortunately I'm getting stuck with an early issue when trying to replicate the tutorial 1 (Caiman motion correction on the allen dataset pvc7). The batch execution does not seem to start (see screen capture below).
Do you have any idea why it does not start ?
I am running the installed MESmerize 0.6.0 version via VM on Windows 10.. Let me know if you need more info.
Thank you very much.
Alberto
