Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Oct 27 16:35
    kevbarb closed #109
  • Oct 27 16:35
    kevbarb commented #109
  • Oct 26 22:48
    kevbarb edited #109
  • Oct 26 22:46
    kevbarb opened #109
  • Oct 24 09:54
    JCKMes commented #99
  • Oct 11 09:15
    kushalkolar commented #99
  • Sep 21 01:54

    kushalkolar on master

    Update index.rst (compare)

  • Sep 21 01:53

    kushalkolar on master

    Update conf.py (compare)

  • Sep 21 01:50

    kushalkolar on master

    Update README.md (compare)

  • Sep 19 01:49
    kushalkolar closed #62
  • Sep 19 01:48
    kushalkolar closed #69
  • Sep 19 01:48
    kushalkolar closed #81
  • Sep 19 01:48
    kushalkolar closed #91
  • Sep 19 01:48
    kushalkolar closed #96
  • Sep 19 01:48
    kushalkolar closed #99
  • Sep 19 01:47
    kushalkolar closed #103
  • Sep 19 01:47
    kushalkolar closed #101
  • Sep 13 02:28
    kushalkolar closed #108
  • Sep 13 02:16
    Robkozol commented #108
  • Sep 12 22:05
    kushalkolar commented #108
maissa_95
@maissa_95:matrix.org
[m]
Hello again, sorry for taking so much to aswer so, my final idea is to go through the statistic DF_F node in the flowchart because my laptop couldn't handle the CNMF so I went back to the manual ROI( sorry for the different directions that I'm applying but I really want to solve the problem) and finally it worked perfectly until the peak detect as you can see it's marked in red and I couldn't open the GUI of it so please what could be the problem?? also I made the plot in the background until the node of the derivative so I think it's fine until there only the setp toward the peak detect seem to be the problem.
maissa_95
@maissa_95:matrix.org
[m]
:point_up: Edit: Hello again, sorry for taking so much to aswer so, my final idea is to go through the statistic DF_F node in the flowchart because my laptop couldn't handle the CNMF so I went back to the manual ROI( sorry for the different directions that I'm applying but I really want to solve the problem) and finally it worked perfectly until the peak detect as you can see it's marked in red and I couldn't open the GUI of it so please what could be the problem?? also I made the plot in the background until the node of the derivative so I think it's fine until there only the setp toward the peak detect seem to be the problem.
Kushal Kolar
@kushalkolar
@paulleogit I'm not sure, maybe there are characters that it's unable to decode, see if your OS is set to use utf8, change the language to English, install utf8?
@maissa_95:matrix.org you can click on a node to see what's wrong
Kushal Kolar
@kushalkolar
@WhereWolf72 the numbers of vertices that represent the edge of the ROI are downsampled for freehand ROIs from imagej
1 reply
paulleogit
@paulleogit
@kushalkolar thank you! I'll try
Hello, This is what appears when I select the peak detect and nothing else, I'm really sorry to bother you
Kushal Kolar
@kushalkolar
@maissa_95:matrix.org you will need to downgrade to pandas v0.25
maissa_95
@maissa_95:matrix.org
[m]
And an extra question are they any option to detect peak frequency in the Mesmerize?
Kushal Kolar
@kushalkolar
@maissa_95:matrix.org nope
WhereWolf72
@WhereWolf72
Anyone else having trouble getting new installs of mesmerize to recognize opencv? I keep getting an error saying that it doesn't have cv2, which is what opencv is imported as. I seem to remember @kushalkolar mentioning this was a more common error on Windows, but I'm on Ubuntu 20.04
1 reply
Dylan Black
@dylanablack

Hello @kushalkolar , I'm just getting started with MESmerize and think it's great so far!

For some experiments, I am working with low magnification (and mesoscale) recordings to measure travelling waves. As such, I don’t have cellular resolution and don’t think that CNMF to identify ROIs is applicable in these cases. Instead, I am looking to apply a grid of rectangular ROIs over all of, or parts of, the field of view (please see the example image below).

I started using the desktop application and understand that I should switch to mesmerize-core. I thought there might be a straightforward way to implement rectangular ROIs using the MESmerize backend (perhaps using the ManagerManual or ManagerScatterROI class). However, this is proving more complicated than I expected and I’m wondering if you have any suggestions?

I attempted to generate these ROIs using an ImageJ macro, however, I receive an error when importing the zipped ROI set into the desktop application (please see the error message below). Strangely, ImageJ ROIs that were generated without the macro import without error.

example_ROI.tif
error_msg.PNG
Kushal Kolar
@kushalkolar
@dylanablack CNMF can detect neurpil so it may still be worth an attempt with a small gSig and params typically optimized for neuropil.
If you just want to get changes in fluorescence by setting a regular square grid you can just use index your arrays and get the fluorescence from those indexed views of the array.
Kushal Kolar
@kushalkolar
Hi everyone, there will be a workshop around late September on the new Mesmerize packages. More information: https://twitter.com/kushalkolar/status/1554927353251262464
bcfvlad
@bcfvlad

Hi everyone, I am using mesmerize to visualize the output of CaImAn with the help of an HDF5 importer. Viewer and ROI manager allow me to visualize the spatial footprints of cells and their traces. Is there an easy and intuitive way to see the individual cells' SNR and spatial correlations (which are also stored in the HDF5 file)?

I would like to see this: a cell with the following spatial footprint has this signal, as well as these values of SNR and spatial correlation.

Perhaps it was mentioned somewhere in the tutorials, but maybe I just missed it...

Eric Thomson
@EricThomson
@bcfvlad I usually do that kind of thing outside of mesmerize, in caiman using their nb component viewer where that info is displayed after importing results from a batch into a notebook.
2 replies
Kushal Kolar
@kushalkolar
@bcfvlad Nope does not exist, maybe a good idea to add for mesmerize-viz which is in development kushalkolar/mesmerize-viz#9
Robkozol
@Robkozol
Hello everyone, I am new to both time lapse gcamp imaging/analysis and looking to use mesmerize to stimulus map or cluster light and dark tuned neurons in fish hypothalamus. I can get the tiff, metadata and ROI's to load, as well as the light stimuli mapped, but I am having trouble with the analysis part. I would like to have a flow chart that loads data frame - normalizes - cross correlates - clusters to stimulus mapp, but I have errors with red highlighted boxes in that configuration. I did try the CNMF tutorial route and it provided some cool segmented groups, but not exactly what I was looking for. Does anyone have a similar example or suggestion for me to try?
Kushal Kolar
@kushalkolar
@Robkozol my time is limited but if you open an issue with all relevant information, screenshots, including the error message shown when you click on the flowchart node (box) that is red I might b able to help. I recommend the youtube tutorial vids, it's a lot but pretty exhaustive.
Robkozol
@Robkozol
@kushalkolar thanks for the reply. I will go back to the documentation and youtube vids and try other routes. I am thankful for the resource and strongly believe mesmerize is a treasure for all of us researchers.
Kushal Kolar
@kushalkolar
Hi everyone, the next in-person CaImAn and Mesmerize Workshop will be January 29-Feb 1 at Flatiron Institute, NYC! We will be teaching the new mesmerize ecosystem of packages!
Charles Warwick
@cawarwick

Running through the demo 'mcorr_cnmf' for Mesmerize-Core and hit an issue post CNMF that one path is relative and one is absolute? This is once the batch has been created and we're doing 'process = df.iloc[i].caiman.run()' Did I miss something or put the paths incorrectly? Thanks

Full Error:
ValueError: 'C:\Users\charl\caiman_data\mesmerize-core-batch\e8887b1f-d56d-4f90-857a-b5573b1d477b\e8887b1f-d56d-4f90-857a-b5573b1d477b.hdf5' is not in the subpath of '\Users\charl\caiman_data\mesmerize-core-batch' OR one path is relative and the other is absolute.

Input Paths:
set_parent_raw_data_path("/Users/charl/caiman_data/")
batch_path = "/Users/charl/caiman_data/mesmerize-core-batch/batch.pickle"
movie_path = "/Users/charl/caiman_data/example_movies/Sue_2x_300040-46.tif"

Kushal Kolar
@kushalkolar
Looks like you're missing C:/ maybe?
Charles Warwick
@cawarwick
image.png
Kushal Kolar
@kushalkolar
if you'r eon windows you'll need the drive letters
Charles Warwick
@cawarwick
Ah, so in Linux you can provide relative but in windows you should enter each of the paths as absolute paths?
Kushal Kolar
@kushalkolar
In that first cell those paths need to be absolute to set things up.
But on windows absolute paths start with a drive letter AFAIK
Kushal Kolar
@kushalkolar
from the explanation in the nb:
image.png
Kushal Kolar
@kushalkolar
If it doesn't exist already, I'll add a check to make sure that the parent path is valid and exists to try and prevent this issue in the future
Eric Thomson
@EricThomson
Yes be careful those are linux paths in the notebook -- in windows it needs C etc etc (at least when using the absolute path).. I've started using Pathlib finally to avoid pitfalls when switching between operating systems.
Charles Warwick
@cawarwick
Yep, that's all it was. Thanks for correcting me on that, for some reason what you said in the workshop about relative paths (since you're on linux) stuck with me more than the actual text in comments. Everything else works so no issues there.
Kushal Kolar
@kushalkolar
opened a PR for it, does a simple check to make sure the path passed to set_parent_raw_data_path() exists. will merge into master when tests pass nel-lab/mesmerize-core#118
glad to hear you got it working!
Kathryn Salvati
@kas5dv
has anyone ever created a singularity image of mesmerize/mesmerize-core to be used on a HPC? My local OS is windows 10 and don't have access to a linux machine...has anyone installed or used mesmerize/mesmerize-core via WSL (windows subsystem for linux)?
Kushal Kolar
@kushalkolar
@kas5dv
WSL is probably more trouble than it's worth, and you won't be able to usefastplotlib for visualization since Vulkan isn't properly supported on WSL.
To use it on an HPC, if it has python3.9 or greater I would just create a virtual environment. Otherwise compile pyton3.9 or 3.10 locally from source (it's not that hard as long as the HPC is modern, if the HPC is old it's probably better to just use a workstation anyways.).
slothToes
@snMousavi
Quick question. What is the recommended method for giving mesmerize-core large image files? If I save all of my data in one tif stack, that would be over 40GB.
Kushal Kolar
@kushalkolar
As long as you have enough RAM ro create a memmap for caiman you should be fine.
Depending on how your tiff file is formatted I might need to add a new method for random-access of frames
slothToes
@snMousavi

Got it. Writing my data to a bigTiff file and following the example notebook on the repo, I get the following error when running process = df.iloc[0].caiman.run() :

KeyError                                  Traceback (most recent call last)
Cell In [44], line 1
----> 1 process = df.iloc[0].caiman.run()
      2 process.wait()

File /opt/anaconda3/envs/mesmerize-core/lib/python3.10/site-packages/mesmerize_core/caiman_extensions/cache.py:214, in Cache.invalidate.<locals>._invalidate.<locals>.__invalidate(instance, *args, **kwargs)
    208 if pre:
    209     self.cache.drop(
    210         self.cache.loc[self.cache["uuid"] == u].index,
    211         inplace=True
    212     )
--> 214 rval = func(instance, *args, **kwargs)
    216 if post:
    217     self.cache.drop(
    218         self.cache.loc[self.cache["uuid"] == u].index,
    219         inplace=True
    220     )

File /opt/anaconda3/envs/mesmerize-core/lib/python3.10/site-packages/mesmerize_core/caiman_extensions/common.py:393, in CaimanSeriesExtensions.run(self, backend, **kwargs)
    390     args_str += f" --data-path {get_parent_raw_data_path()}"
    392 # make the runfile
--> 393 runfile_path = make_runfile(
    394     module_path=os.path.abspath(
    395         ALGO_MODULES[self._series["algo"]].__file__
...
    678     # raise KeyError with the original key value
--> 679     raise KeyError(key) from None
    680 return self.decodevalue(value)

KeyError: 'CONDA_PREFIX_1'

Running 32 GB of memory on M1 Mac. Not sure if it's the chipset / OS that might have an incompatability

3 replies
Kushal Kolar
@kushalkolar
@snMousavi what OS is this on? how did you install mesmerize-core? also, as far as I know M1 macs don't support the scientific python stack very well, I don't know if that's changed in the past few months (it was a pain to get tensorflow installed last time I heard)
slothToes
@snMousavi

@kushalkolar I have tried this both on Mac M1 and now on Ubuntu 20.04.3. Conda version is 22.9.0. I installed mesmerize-core using mamba install -c conda-forge mesmerize-core on Python 3.10.6. Installation did not output any warnings or errors.

The error message on Ubuntu is slightly different but traces back to the 'CONDA_PREFIX_1':

FileNotFoundError                         Traceback (most recent call last)
File ~/anaconda3/envs/mesmerize-core/lib/python3.10/site-packages/mesmerize_core/caiman_extensions/common.py:401, in CaimanSeriesExtensions.run(self, backend, **kwargs)
    400 try:
--> 401     self.process = getattr(self, f"_run_{backend}")(
    402         runfile_path, **kwargs
    403     )
    404 except:

File ~/anaconda3/envs/mesmerize-core/lib/python3.10/site-packages/mesmerize_core/caiman_extensions/common.py:329, in CaimanSeriesExtensions._run_subprocess(self, runfile_path, **kwargs)
    328 if not IS_WINDOWS:
--> 329     self.process = Popen(runfile_path, cwd=parent_path)
    330 else:

File ~/anaconda3/envs/mesmerize-core/lib/python3.10/subprocess.py:969, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize)
    966             self.stderr = io.TextIOWrapper(self.stderr,
    967                     encoding=encoding, errors=errors)
--> 969     self._execute_child(args, executable, preexec_fn, close_fds,
    970                         pass_fds, cwd, env,
    971                         startupinfo, creationflags, shell,
    972                         p2cread, p2cwrite,
    973                         c2pread, c2pwrite,
    974                         errread, errwrite,
    975                         restore_signals,
    976                         gid, gids, uid, umask,
...
export CONDA_PREFIX_1=/home/jim/anaconda3
export PYTHONPATH=/home/jim/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles:/home/jim/.vscode/extensions/ms-toolsai.jupyter-2022.9.1202862440/pythonFiles/lib/python
export OPENBLAS_NUM_THREADS=1
export MKL_NUM_THREADS=1
python /home/jim/anaconda3/envs/mesmerize-core/lib/python3.10/site-packages/mesmerize_core/algorithms/mcorr.py --batch-path batch.pickle --uuid eb2b3c47-ef89-4e88-801d-a229aac63142 --data-path .
5 replies