@all see above if you were having trouble
seems to have been a error in file content not getting caught correctly
didn't show up till now cause we hadn't seen these particular errors
@/all new deadline may 4th! should give everybody who wanted to submit the chance to do so.
My guess is that the convolutional networks are not really easily overfitting the training data. I tried fitting a single neuron with a CNN, and it could still nicely predict other neurons' spikes. However, the optimal temporal filters are different for the different datasets, and I guess that this point (choice of a filter set for a specific dataset) is where overfitting can happen. But I'm curious as well ...
i will try to analyse the results quickly and announce the winners!
this is the mean of the correlation values for the five test datasets
however, a weighted mean (by number of cells per datasets) gives slightly different ordering of the top 3
therefore:
we have three winners: Patrick Mineault, Nikolay, Peter&Stephan! Congratulations. The absolute difference between these is less extremely small, so i think it is fair to award three first prizes
Here is the analysis code: https://github.com/berenslab/spikefinder_analysis
all these submitters were better than STM, the state of the art, so these submissions qualify for being included in a review/overview paper of the challenge
@philippberens I've got a suggestion for something to analyze for the paper: One could select for each neuron in the test (or also training) dataset the submission that was best at predicting the spiking of this very neuron. Then order the neurons according to this highest correlation value (which tells roughly how well overall this neuron can be predicted) - neurons of which the spikes can be easily predicted would be plotted to the right hand side, and neurons of which the spikes cannot be well predicted to the left. Plotting the performance of the different models would then visualize that e.g. model 1 is exceptionally good at fitting neurons that are easy to fit, and model 2 is exceptionally good at fitting neurons that are difficult to fit, etc.
Of course, if the result of the plot is just a mess, better discard it, but I'd be curious to see this, if it is not too much work. I hope I have explained this well enough (otherwise I could make a small dummy figure for visual explanation).
Of course, if the result of the plot is just a mess, better discard it, but I'd be curious to see this, if it is not too much work. I hope I have explained this well enough (otherwise I could make a small dummy figure for visual explanation).
thanks
once I have the traces I can do it
Hi @phreeza : Agree. I have done it like this, and if you like it you can use it as well: https://www.dropbox.com/sh/oi1c1zy9dxichaj/AADuZe6z_sx8vHWAAdfs3Gffa?dl=0
The numbers in the boxes indicate the output sizes. But I'm open to any other ideas ...
The numbers in the boxes indicate the output sizes. But I'm open to any other ideas ...
Hi, I'm starting to look at the spikefinder data and some of the models to hopefully apply them to our own datasets, but it's not obvious to me how a neuron from a series of calcium images (TIFFs) gets transformed into a single column vector? Maybe it's something as simple as taking the mean of all pixels in the neuron at each time step? Is the code for doing this transformation available somewhere? I appreciate any tips.
Hi @alexklibisz , this is typically done as you have described it:
1) Take the mean of all pixels in the neurons (ROI) at each time step. The ROI is drawn either manually (you could use e.g. ImageJ's ROImanager) or using a more elaborate semiautomatic/manual/automatic approach (there are many: http://neurofinder.codeneuro.org/). For starters, manual ROI-drawing should be fine.
2) For the spikefinder competition, the timetraces (single column vectors) have been normalized. The normalization process follows Theiss et al. (2016) and can be found here: https://github.com/lucastheis/c2s/blob/master/c2s/c2s.py under the method "preprocess". In the end it is just a sort of normalization, and Philipp could give you more details on that ...
1) Take the mean of all pixels in the neurons (ROI) at each time step. The ROI is drawn either manually (you could use e.g. ImageJ's ROImanager) or using a more elaborate semiautomatic/manual/automatic approach (there are many: http://neurofinder.codeneuro.org/). For starters, manual ROI-drawing should be fine.
2) For the spikefinder competition, the timetraces (single column vectors) have been normalized. The normalization process follows Theiss et al. (2016) and can be found here: https://github.com/lucastheis/c2s/blob/master/c2s/c2s.py under the method "preprocess". In the end it is just a sort of normalization, and Philipp could give you more details on that ...
One thing to add: When using TIFFs to extract calcium signals, you might want to subtract an offset (if there is any). After that, the signal is averaged over the ROI, and the dF/F0 is calculated. If you do not know what that is, check this out: https://www.quora.com/Why-is-the-equation-%CE%94F-F-used-to-measure-changes-in-fluorescence-intensity ;)
Plus, many datasets suffer from background contamination. Therefore, many people perform a 'neuropil subtraction' of the neuron-timetraces. However, this was not done for this competition, if I'm not wrong.
@PTRRupprecht I'm still a little confused on the details, so to make sure I understand it correctly I'll describe the steps and indicate where I'm unsure. If you could correct me that would be very much appreciated.
Given an ROI from a series of TIFFs (e.g. one of the neurofinder datasets), I would take the following steps to get a spikefinder format column vector:
- Subtract an offset -- what constitutes an offset? Do I want to subtract such that the min is 0, the data is centered at 0, etc?
- Average all pixels at each timestep.
- Transform each value F_t as
F_t = (F_0 - F_t) / F_0-- when I look at the datasets without preprocessing (https://goo.gl/GoEPfH), F_0 is much larger than the remaining values, but this isn't the case with the Neurofinder data, which makes me think I'm still misunderstanding this step. - Call the
preprocessfunction from c2s with fps/Hz = 100.
Is there maybe a snippet of code somewhere for converting a Neurofinder ROI into Spikefinder format?
Dear all,
I’d like to download the raw, unprocessed data set. However, the link http://spikefinder.codeneuro.org/assets/nopreprocessing.zip does not work. Can someone provide me with a new link ?
Best regards,
Jerome Tubiana
I’d like to download the raw, unprocessed data set. However, the link http://spikefinder.codeneuro.org/assets/nopreprocessing.zip does not work. Can someone provide me with a new link ?
Best regards,
Jerome Tubiana
Hi Jerome, Philipp mentioned it somewhere above, it's on Dropbox (but only for datasets 1-5): https://www.dropbox.com/s/ugmet5eky8g0l5n/datasets%201-5%20nopreprocessing.zip?dl=0
Best,
Best,
I had a question regarding these data. I know the competition is over, but I was playing around with the data for a class. The training dataset 07, with GCaMP6f data, has 37 cells in it. However, the source data provided on CRCNS says there were only 11 GCaMP6f cells in that dataset (cai-1). Is there a reason for the discrepancy?
I could see a couple plausible things, but I thought I'd ask here to see why that might be. Are the cai-1 recordings really long, so they were split up for SpikeFinder (So in reality the 37 cells come from those same 11 cells), or were additional datasets beyond what the documentation of cai-1 says provided for SpikeFinder?