Does anyone have any experience with using mesmerize on a cluster? We have to request number of cores, RAM, GPU or no but then we have a 10TB scratch space and can use a Desktop environment running CentOS below to access mesmerize. So far, I have been able to run the motion correction but then the cnmf-e fails with not being able to find a file.
I am using Eric Thomson's instructions on the windows installation so I know that it is not a problem with the tiff movie.
You will probably have to set a workdir and it will need to be mounted at the same location on the compute nodes and the submission machine, but if you post an issue I can help with the details
Hi Kushal! I follow instructions online to export and open up a transmission object externally in python. I detected what I think may be an indexing issue in the transmission dataframe. The "_DFOF" traces and "_RAW_CURVE" traces aren't lining up for me. I plotted the calculated dfof and raw curves for three consecutive rois using the code below where t is the transmission object and n is my roi index:
ax.plot( t.df["_DFOF"].iloc[n]);
ax.plot( t.df["_RAW_CURVE"].iloc[n]);
Here is the plot for the three consecutive rois: https://imgur.com/a/nHUJ5ly Orange = "_RAW_CURVE" blue = "_DFOF"
It looks like the DFOF traces in blue matches with the orange trace of the previous roi
The paper is out now! https://twitter.com/kushalkolar/status/1460152541015121920?t=wFgSpClBaIMfFeBtrzVBDg&s=19
Hi Kushal, congratulations for the paper! I was wondering if it is possible that the MC output file get the name of the orinial file instead of a collection of letters and numbers. When trying a bunch of parameters for CNMF-E, It is very slow to have to check the match between the original filename and the mc filename everytime. Coud you give some tips on how to change that in the code?
@AyusoAa I'm not sure what you're trying to do. If you use the batch manager interface you don't need to know the uuids for each item to iterate over parameter variants.
@kushalkolar Congrats on the paper! I'm having some trouble getting the metadata (specifically the frames per second) to be loaded properly to run CaImAn's CNMF-E. I am almost certain the problem is on my end, but I can't seem to troubleshoot it. The files were acquired via Nikon Elements, so they were originally .nd2 files. I manually added "fps = __" into the metadata for each as our Elements only uses frame acquisition rate (seconds per frame), and then converted to TIFFs via Elements. Finally, I have deinterleaved the two-channel images into just the channel for the calcium imaging using FIJI. In the end, none of the metadata loaders will recognize the image file as containing the fps metadata. Any help or advice would be greatly appreciated, thank you!
@WhereWolf72 you can create a json file with the same name as the tiff file that has the structure of the "json_minimal" described here http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html#module-tifffile-metadata
2 replies
@danielmichaelvirga See this: http://docs.mesmerizelab.org/en/master/user_guides/viewer/overview.html#open-image
Find the mesmerize config.json file and change the "_MESMERIZE_WORKDIR" key.
if you're on Linux or Mac the config file is located in the .mesmerize dir in the home dir. ON windows it's in the user profile dir.
I successfully ran CNMFe for one of my samples as a test and want to optimize my parameters. However, now I am running into a pickling error that I'm not sure how to fix whenever I try to run CNMFe. Any advice would be appreciated. Here's then end of the error code (I can post the entire error if needed): multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f4f6cdd3da0>'. Reason: 'PicklingError("Can't pickle <class '_flapack.error'>: import of module '_flapack' failed",)'
2 replies
Hi, I just ran a small test batch through Mesmerize with good results so far (really loving the GUI and the video tutorials, excellent work). However my actual data has multiple Z-planes and despite being reasonably comfortable with the GUI, I cannot figure out how to get the TIFF file importer to recognize my TIFFs as 3D. I searched the docs for a template meta data file for 3d (such as the meta min provided for single plane imaging) and I couldn't find what the metadata should look like in order to format the json appropriately to get the importer to sort the planes correctly. At the moment, I'm using a ThorLabs scope which exports OME Tiffs along with an XML. In one of the video tutorials Kushal mentioned he would provide a link explaining how to add more meta data import methods but I didn't see the link or info about in the docs. Any help would be appreciated, thanks!
@cawarwick you don't need additional metadata to import 3D files, if the default
http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html
tzxy axes order isn't working you can enter the axes order that your data are in:http://docs.mesmerizelab.org/en/master/user_guides/viewer/modules/tiff_file.html
6 replies
ok, so new issue. If I import the files in the default order tzxy, the image is rotated 90 degrees, which would suggest my files are tzyx and (256X512 (Y,X)) and using a custom order, when I ran the 3D CNMF, the ROIs appear to be rotated 90 degrees and don't align with the image. Did I do something out of the ordinary?
My post was not as clear as it could have been. The default order tzxy makes my image rotated 90 degrees, so I then instead used tzyx which did import it correctly as pictured in the screenshot (that is a horizontal orientation like a widescreen monitor), but the ROIs generated are rotated 90 degrees. For reference, I did not have this happen with a single Z-plane image, but it imported correctly with the default order of txy. I'm not sure why the multiplane image is different, they were both created in ImageJ from the same source file.
e.g. My first batches were with a simpler single-plane image and it ran exactly as I expected with the default order of txy (screenshot is from a single plane image that ran fine). When I then included the other Z-planes in the TIF the order had to be tzyx for it to be oriented correctly.
error in trying to load script editor: Windows OS:
Traceback (most recent call last):
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 141, in <lambda>
self.ui.actionScript_Editor.triggered.connect(lambda: self.run_module(script_editor.ModuleGUI))
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 299, in run_module
m = module_class(self, self._viewer)
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 27, in init
self.open_temp_script()
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 72, in open_temp_script
home_dir = os.environ['HOME']
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'HOME'
Traceback (most recent call last):
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 141, in <lambda>
self.ui.actionScript_Editor.triggered.connect(lambda: self.run_module(script_editor.ModuleGUI))
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\main_window.py", line 299, in run_module
m = module_class(self, self._viewer)
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 27, in init
self.open_temp_script()
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\viewer\modules\script_editor.py", line 72, in open_temp_script
home_dir = os.environ['HOME']
File "C:\Users\kates\anaconda3\envs\mesmerize\lib\os.py", line 669, in getitem
raise KeyError(key) from None
KeyError: 'HOME'
I'm guessing this is a computer managed by the university or something?