Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 15 20:11

    rossant on dev

    Update README (compare)

  • Jun 15 19:58
    rossant closed #835
  • Jun 15 19:58
    rossant commented #835
  • Jun 15 19:58
    rossant closed #834
  • Jun 15 19:58
    rossant commented #834
  • Jun 15 19:58

    rossant on dev

    Fix installation.md (compare)

  • Jun 15 19:45

    rossant on dev

    Update install instructions (compare)

  • Jun 15 18:35

    rossant on dev

    Update doc table of contents (compare)

  • Jun 15 18:26

    rossant on dev

    Update API doc (compare)

  • Jun 15 08:23

    rossant on dev

    Better position of window size … Relax zooming constrain in grid… Add cluster_group colormap and 3 more (compare)

  • Jun 14 20:36

    rossant on dev

    Add mean firing rate column in … Press escape to clear the filte… Fix bug in batch text visual wh… and 5 more (compare)

  • Jun 14 13:13
    rossant labeled #532
  • Jun 14 13:13
    rossant unassigned #532
  • Jun 14 13:12

    rossant on dev

    Add GUI action to dynamically c… Add option to change the marker… Add option to change the marker… and 2 more (compare)

  • Jun 14 11:16
    rossant commented #644
  • Jun 14 11:11
    rossant labeled #841
  • Jun 14 11:11
    rossant labeled #841
  • Jun 14 11:11
    rossant assigned #841
  • Jun 14 11:11
    rossant opened #841
  • Jun 14 11:04
    rossant commented #782
Shabnam Kadir
@shabnamkadir
this would favour local only
Cyrille Rossant
@rossant
yeah. right now there isn't going to be support for multiple views of the same type, but it will come later
I think this is gonna be a rare scenario. you'll always be able to access the API and change the scaling of every single view by hand if you want to
Shabnam Kadir
@shabnamkadir
multiple correlogram views would be very useful
Cyrille Rossant
@rossant
but I'm not sure why you'd want different scaling for different waveform views
yes this one is high priority
but I think that's the main use case for multiple views of the same kind
also: there isn't gonna be text atm, but views will display some information in the status bar
like the channels displayed
Shabnam Kadir
@shabnamkadir
how easy will it be for the dilettante developer to make new views?
Cyrille Rossant
@rossant
not harder than matplotlib :)
this is the plotting part in the feature view
Max Hunter
@nippoo
multiple featureviews are pretty common, though we might be able to cover that with the FeatureGridView
ok so basically: the mouse + modifier key shortcuts in KV were really useful
unless you use the program really regularly, nobody (inc. me!) can remember the pure-keyboard shortcuts
while it's ok to have them, I suspect 99% of the users will use the mouse + modifier ones if they're there
Max Hunter
@nippoo

so: e.g. WaveformView:
mouse drag = move all waveforms (pan)
ctrl + mouse drag = modify waveform box spacing
shift + mouse drag = modify waveform box height/width (aka zoom)
shift + mouse wheel = modify waveform height

FeatureView:
mouse drag = move all features (pan)
shift + mouse drag = modify features height/width (aka zoom)
shift + mouse wheel = modify feature point size (inc. background point size, I think!)

TraceView:
mouse drag = move traces (pan). NB it would be nice to implement continuous scaling, but it doesn't matter if the same drag causes it to jump in chunks, I don't think that would be too weird.
shift + mouse drag = modify traces height/width (aka zoom)
shift + mouse wheel = modify traces channel height

CorrelogramView:
mousewheel = change total width (in discrete steps - with text on axes!)
shift + mousewheel = change bin size (in discrete steps - with text "bin size = x"!)

with regards to global shortcuts: I think it's less user-friendly to have view-specific global shortcuts. To start off with, it makes having multiple views more difficult! But also it's much more difficult to remember multiple UI paradigms for similar views than one, does that make sense? If we introduce an ISI view or a separate Template View, we don't want to have different modifier keys for that. My original idea was to have a big red box around the "selected view" and one set of keys (ie shift + / control + / alt + arrow keys for everything) and then you can move around views by clicking on them or with a shortcut, like, the arrow keys
Max Hunter
@nippoo

so effectively you have like, about 4 total view-related context-dependent shortcut functions

  • arrow keys (no modifier): move around 'selected view'
  • shift + arrow keys: zoom
  • ctrl + arrow keys: change spacing
  • alt + arrow keys: change height/size

this is all just a vague idea, I suspect things might change (particularly going between selected views, I don't know if arrow keys will be intuitive with views on different screens) but this is a vague idea?

ps - we should move this conversation to kwikteam/kwikdev to avoid spamming the user chatroom :)
Shabnam Kadir
@shabnamkadir
Well users may have suggestions for keyboard shortcuts that they favour
Cyrille Rossant
@rossant
having more mouse shortcuts is a good idea! but i'm not convinced by having a notion of selected view and many local shortcuts. That might a bit over-engineering and perhaps we shouldn't have too many ways of doing the same thing. If we have good mouse shortcuts that will already be good. Regarding global view-specific actions we could have less keyboard shortcuts for those. In any case there is now a menu bar with the totality of the keyboard shortcuts, along with their names, description, organized per type of action and per view. This will improve the discoverability of the actions and shortcuts. Now, if users have a way to define their own shortcuts, most of them will probably be happy. Don't forget we also have snippets for all actions as well so there's plenty of possibilities already!
Max Hunter
@nippoo
ok! but the problem is - how will we distinguish between different views? if people do want an ISIView or multiple CorrelogramViews or multiple FeatureViews or whatever....
Cyrille Rossant
@rossant
yes.
  • while most of the architecture is ready for multiple views, it's not enabled yet (it's not completely trivial)
  • I don't know if we'll enable this feature generically for all views, or only for the CCG view. adding this feature for all views does add some complexity to the code so we need to be sure we absolutely need it
  • if we ensure that everything is possible with mouse shortcuts, then having global shortcuts that only work for the first view or on all views is not a big deal I think
Antonin Blot
@ablot
Hi all, is it possible to access to the quality measure of the wizard without using the gui? And are there already options to calculate the isolation distance or other quality metrics?
Carl Schoonover
@carlschoonover

Hi,

Following up on Jeremy's post--and copying over from the Klutaviewas message board, since this might be the more relevant place to ask these questions.

[Jeremy, for comparison, our typical datasets, averaging 3M spikes off of 32 channels, take ~6 hours in the detection phase, and ~72 hours in the clustering phase when we process on 16-core nodes (E5-2650v2) with 256 GB RAM. ]

We are considering migrating our sorting pipeline to phy now that it plays nice with KlustaViewa. (For the record, in our hands phy runs in one quarter to one half of the walltime required by KK1 using equivalent parameters and computing resources--does this match up with other people's experiences?)

Before we take the plunge, we wanted to check in on whether the following fixes have been implemented:

- ability to perform detection and clustering steps separately (and, critically, to pass different parameters after the detection step)
- phy parameter equivalent to KK_Subset

More generally, we are in early conversations with the high performance computing people here regarding a new cluster that will be built for the neuroscience community at Columbia. It would be helpful for us to get a sense of where phy is heading long-term so that we can put in our two cents when decisions are being made concerning the hardware design. Early message board posts on phy had raised the possibilities of (1) porting to GPU, and (2) implementing it for a distributed computing architecture. So far my impression is that phy is still multiprocessor like its predecessor but cannot run on a distributed system (do I have this right?). Does the long-term vision still include either GPU or distributed processing; i.e. when it comes time, should we advocate for GPU modules, a fast distributed processing architecture (e.g. something like InfiniBand) or simply big nodes with many, powerful cores?

napvasconcelos
@napvasconcelos
This message was deleted

Hi,

I have a bunch of datasets to do the spike sorting. Now, is more often find this kind of error during the the spike extraction part, when messages like the following can be found in several lines in the log file:
spikedetekt/group_all/components/chunk_253775625.npy` doesn't exist.chunk_253775625.npy` doesn't exist.

Any clue ?

Thanks in advance,

Antonin Blot
@ablot
Hi,
Yesterday I installed the new version of klusta on a new system. Klusta itself work out of the box but to get kwik-gui working I had to add a few steps compared to the install instructions (http://klusta.readthedocs.org/en/latest/).
Obviously I had to install phy
  • pip install phy phycontrib
  • edit ~/.phy/phy_config.py and type "import phycontrib" in it as suggested by Cyrille on the google group
  • install vispy (with pip)
  • install joblib (with conda)
Cyrille Rossant
@rossant
Thanks! The "klusta" installation instructions are normally for KlustaViewa, whereas phy was meant to be installed separated in another conda environment (as per the installation instructions on the phy webpage). However I think it makes sense to include phy too in the klusta installation instructions, since it is now relatively stable. I just updated the .yml files accordingly in the klusta installation process.
Antonin Blot
@ablot
With the new klusta, all the information about the current clustering are stored in .klustakwik2/, .phy/, .spikedetekt and phy.log in the local directory, is that right? Does that mean that I have to make sure that I have only one single prm file per folder?
Cyrille Rossant
@rossant
Yes!
Antonin Blot
@ablot
Good. That might explains the error I had (the kwik-gui was trying to access spikes 70000 in a file where there were only 50000 spikes in total but I had done 2 clusterings in the same folder, it probably got confused)
Cyrille Rossant
@rossant
Yes we have to precise this in the docs -- or feel free to propose a PR :)
Antonin Blot
@ablot
a PR for to change the docs? Are they part of the klusta repository?
Cyrille Rossant
@rossant
yes, i just updated the docs but feel free to propose any further improvements https://github.com/kwikteam/klusta/blob/master/docs/sorting.md#preparing-your-files
Antonin Blot
@ablot
Thanks. What's the link between this local documentation and what is in the readthedocs (linked on the README.md)?
Cyrille Rossant
@rossant
it should be the same... the readthedocs page should be automatically updated when the docs/ dir in the repo is updated (takes a few minutes)
Antonin Blot
@ablot
The RamLimitGB option has disappeared from klusta? Running klusta seems to take a lot of RAM for the spike detection. Does klusta need to load the whole dat file in RAM for filtering/thresholding?
Cyrille Rossant
@rossant
how much ram does it take, and how big is your dataset (time and nchannels)? spike detection loads 50 chunks of 1sec each for thresholding
Antonin Blot
@ablot
32 channels, 30kHz, I'd have to check for the length but the dat file is 26gb. I killed the process when it reached ~40gb or RAM
Cyrille Rossant
@rossant
ah, this must be a bug... can you send me your PRM?
sin-mike
@sin-mike
Hi all! I've become interested in your package after reading your article http://www.nature.com/neuro/journal/v19/n4/full/nn.4268.html. I would like to join the phy community and get an insightful view on the spike sorting algorithms and develop new ones. I was wondering if I could contribute to the community. I was mastering machine learning and pattern recognition in the field of computer vision using python as well as processing extracellular recordings in matlab.
Antonin Blot
@ablot
I'm still using phy v0.2.2 to open kwik files using phy.session.Session to access the data after clustering. The whole session class seems to have disappear in newer phy releases. What is the best way now to access the same informations? (metadata of clustering, cluster properties (including probe position), waveforms read from the dat files...)
Optrode
@Optrode
Hi, I have a few questions about the Phy / KiloSort interface and how to integrate it with existing sorting routines in MatLab. Anyone around who could help me out?
Oh... This is all pretty old, isn't it.
Bionerdess
@Bionerdess
Hi guys, someone here that knows how to add the "channel" (in cluster view)as an additional column to the cluster_group.tsv file