BlueBrain/BluePyOpt

Blue Brain Python Optimisation Library

Werner Van Geit

@wvangeit

@strawcup , does this stop the execution? It's a warning and normally shouldn't affect the execution itself.

Tanguy Damart

@DrTaDa

@Aman-A Hello, have you checked if your NEURON installation works outside of BluePyOpt ?

1 reply

strawcup

@strawcup

@wvangeit , thanks for the response. You're right-- it doesn't stop the execution itself, but I get the following error message when I run the next line 'responses = twostep_protocol.run(cell_model=simple_cell, param_values=default_params, sim=nrn)':

ProcessExpired Traceback (most recent call last)

<ipython-input-16-b41867be1955> in <module>
1 default_params = {'gnabar_hh': 0.1, 'gkbar_hh': 0.03}
----> 2 responses = twostep_protocol.run(cell_model=simple_cell, param_values=default_params, sim=nrn)

~/opt/anaconda3/lib/python3.7/site-packages/bluepyopt/ephys/protocols.py in run(self, cell_model, param_values, sim, isolate, timeout)
81 sim=sim,
82 isolate=isolate,
---> 83 timeout=timeout)
84 except TypeError as e:
85 if "unexpected keyword" in str(e):

~/opt/anaconda3/lib/python3.7/site-packages/bluepyopt/ephys/protocols.py in run(self, cell_model, param_values, sim, isolate, timeout)
233 timeout=timeout)
234 try:
--> 235 responses = tasks.result()
236 except TimeoutError:
237 logger.debug('SweepProtocol: task took longer than '

~/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
433 raise CancelledError()
434 elif self._state == FINISHED:
--> 435 return self.__get_result()
436 else:
437 raise TimeoutError()

~/opt/anaconda3/lib/python3.7/concurrent/futures/_base.py in get_result(self)
382 def get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result

ProcessExpired: Abnormal termination

1 reply

mphillipsneuro

@mphillipsneuro

Hi @wvangeit,
I'm currently attempting to use bluepyopt to optimize parameters for a model of NMDA receptor kinetics (i.e. rate constants for transitions between states, number of receptors, etc.). My stimulus (agonist and antagonist applications on a single vclamped cell) is built into my .mod file and works great for simulations and parameter optimization just using NEURON, but I'm having trouble figuring out how to use it with bluepyopt. You mention above that it is possible to create a custom "subclass of Stimulus and use your mod mechanism in the implementation of that new class". Could you elaborate on how this can be done, or point me in the direction of an example?
Thanks!

Werner Van Geit

@wvangeit

@strawcup do you mind creating an issue in the bluepyopt tracker, then we can investigate it further over there

@mphillipsneuro is your MOD file open source, could we have a look at it? For an example how e.g. a protocol is overwritten, you could have a look here: https://github.com/BlueBrain/BluePyOpt/blob/master/examples/thalamocortical-cell/CellEvalSetup/protocols.py

mphillipsneuro

@mphillipsneuro

@wvangeit Thanks for the response and example. I believe I've successfully written a voltage clamp stimulus subclass, but the agonist/drug applications are proving to be tricky. The MOD file is currently unpublished (and very much a work in progress) and thus not currently available online. The closest is found here https://senselab.med.yale.edu/ModelDB/ShowModel?model=145836&file=/MoradiEtAl2012/SynNMDA16_2.mod#tabs-2
but the method I'm using for modeling the stimulus is entirely different. I can send it to you via email. Thanks again!

IlaCar

@IlaCar

Hi @wvangeit and hi everyone, I have this strange error :

File "/home/miseno/.local/lib/python3.6/site-packages/bluepyopt/ephys/morphologies.py", line 128, in instantiate
imorphology.input(str(self.morphology_path))
RuntimeError: hoc error

when I run a code that works perfectly on another machine. Can you please point me to the right direction to solve the problem?
Thank you.

Ilaria

Werner Van Geit

@wvangeit

@DrTaDa could you have a look what might be causing the above error? Thanks.

Tanguy Damart

@DrTaDa

Hello @IlaCar, do you have the same version of NEURON, Python and BluePyOpt and the two machines ? Also, what is the format and the origin of the morphology file you are using ? If you morphology is open, could you send it to us ?

IlaCar

@IlaCar

Hello @DrTaDa, yes, I have the same version of NEURON, Python and BluePyOpt. I use the swc format. Here you find the morphologies:
https://kg.ebrains.eu/search/instances/Model/a6458de3-a176-4378-9b03-34a26f5da3bd

9 replies

sunzhenyang2018

@sunzhenyang2018

Respected devs of BluePyOpt, I wonder whether your code works on windows. My concerns lie in the use of mod files, because compiling mod files is different between windows and linux/mac. (e.g. in linux libnrnmech.so is stored somwhere in the x86 folder, and in windows nrnmech.dll is created in the folder where the mod files are; windows has nrniv.exe while linux has something else. etc.) Since my model is quite dependent on these mod files, it would be really nice if BluePyOpt is compatible to using mod files on windows, or can be configured to do so easily.

Werner Van Geit

@wvangeit

dear @sunzhenyang2018, unfortunately we don't have access to Windows machines, nor do we have access to testing infrastructure for Windows, so we're not really supporting it at the moment. Would it be possible for you to use a VM or the Windows Linux subsystem? Is there any particular error you're getting? Another would be that you subclass the Simulator class to handle the loading of the dll file.

sunzhenyang2018

@sunzhenyang2018

Dear @wvangeit, it worked perfectly for me on windows without using any VM or linux-like environment! That was quite surprising but a welcomed one.

Werner Van Geit

@wvangeit

@sunzhenyang2018 , ok, good to hear that it worked for you

FrancescoCavarretta

@FrancescoCavarretta

if I try to run BluePyOpt in parallel using ipyparallel,
do I need to pass a map_function to DEAPOptimisation?
What kind of map function I may use?

Werner Van Geit

@wvangeit

@FrancescoCavarretta I've answer your question in https://github.com/BlueBrain/BluePyOpt/issues/319#issuecomment-675307276

FrancescoCavarretta

@FrancescoCavarretta

@wvangeit just seen today. thank you so much.

sunzhenyang2018

@sunzhenyang2018

Hi, I also have a question about parallelization, but mine is about setting it up on Neuroscience Gateway (NSG). I have a optimization that has 200 generations and 100 offspring, with python's built in map function. How many processes are present in each generation, and how many cores should I assign to this optimization job?

3 replies

Aman Aberra

@Aman-A

Hi, I've had a couple optimization runs fail due to out of memory error while saving the checkpoint file on our cluster, which causes the checkpoint file to be empty and all progress to be lost. I can just restart with more memory from the beginning in this case, but for longer runs, this would be pretty unpleasant. Have you thought about implementing a version of checkpointing where each generation is a different file, or they're batched together in some way, so that the new checkpoint file isn't overwriting the old file. This way any error or interruption wouldn't lose the data from past generations

Werner Van Geit

@wvangeit

@Aman-A I'm not sure we ever encountered this issue, but it's a valid point. I'm rather reluctant to save all the checkpoints to save disk space. But one thing we could do is save the checkpoint first to a temporary filename and then overwrite the main checkpoint file with a copy operation (which should be failry atomic). We can also keep the temporary file as an extra backup. What do you think?

Aman Aberra

@Aman-A

@wvangeit Ya, I was gonna say the previous generation files could be deleted at some point, but this sounds like a better solution to the main problem. So with each generation, you would over-write the temporary file then copy the temporary file over the cp_filename?

Werner Van Geit

@wvangeit

Yep, that's the idea

I created this: BlueBrain/BluePyOpt#323

Aman Aberra

@Aman-A

Hi, I was wondering if there's a way to work with custom sectionlists using the Morphology class. So if I write an swc file and use 5+ for e.g. myelin, how could I specify this when instantiating a Morphology and CellModel? Or would I need to use the HocModel class?

Werner Van Geit

@wvangeit

I'm not 100% sure, but I don't think Neuron will create sectionlists other than the default ones

If you know which sections have the specific type, you could however read the morphology using the normal class, and assign the relevant sections to sectionlists you create

Aman Aberra

@Aman-A

Ok I think a better approach may be to load the somatodendritic morphology in the swc, then generate custom axons with neuron code using the replace_axon_hoc option. Is there any reason not to make seclist_names and secarray_names customizable in the CellModel class?

Werner Van Geit

@wvangeit

There is no specific api to changes seclist_names/secarray_names. But after constructing the object, they are just attributes of the object. So normally you can customize them that way.

9 replies

William Scott Thompson

@wstho

Hi! Wondering if anyone has come across this error before: " File "/cfs/klemming/nobackup/w/wst23/local/tegner/lib/python3.6/site-packages/bluepyopt/ephys/parameterscalers.py", line 81, in scale
return self.multiplier * value + self.offset
TypeError: can't multiply sequence by non-int of type 'float' ". Have had bluepyopt running fine on this machine in the past, not sure what would have caused this. Cheers!

Werner Van Geit

@wvangeit

It looks like one of the values in 'self.multiplier * value' is a sequence (probably a string). Possibly somewhere you feed a string instead of a float into the arguments of the methods.

Russell Jarvis

@russelljjarvis

Hi Werner I finally made a Pull Request on Github concerning the integration of BluePyOpt with NeuronUnit. I am having trouble getting the tox continuous integration tests to pass, I think thats because my version of setup.py has requirements that are git repositories. If possible I am hoping to get feedback about the Pull Request. Best, Russell.

Russell Jarvis

@russelljjarvis

Also I wonder if there is any apetite for a streamlit interactive frontend for BluePyOpt? I have done something like this for my examples, but I didn't include it in the Pull Request. The approach can be integrated with standard blue py opt examples.

sunzhenyang2018

@sunzhenyang2018

Hi, I am confused about how to how to set up point process. The few things that confused my are like suffix and location. As far as I know, mod files for point process doesn't explicitly declare a suffix. The other thing is location, specifically NrnPointProcessLocation. The constructor expect the second argument to be a string, but then in instantiate, the line self.pprocess_mech.pprocesses is run which doesn't work if pprocess_mech is a str. If it is not a str, then do I need create the point process before the location, and pass the point process to it? If so, what location value should I pass to the point process's constructor?

It would be really helpful if you could provide me with an example of setting up a default point process from NEURON

Werner Van Geit

@wvangeit

here there is an example of setting up a point process: https://github.com/BlueBrain/BluePyOpt/tree/master/examples/expsyn

you're right, the naming is confusing. Suffix is similar to a channel, but indeed, in a point process it's not explicitely called Suffix

I basically meant the name of the point process: https://github.com/BlueBrain/BluePyOpt/blob/master/examples/l5pc/mechanisms/ProbAMPANMDA_EMS.mod#L69

I think the notebook in the example I showed could help you with your question of the location

let me know if you have more questions

just pinging @sunzhenyang2018 to make sure you see the reply

sunzhenyang2018

@sunzhenyang2018

Oh thank you so much. That example is very helpful!

sunzhenyang2018

@sunzhenyang2018

Hi, I have a question about CVODE compatibility in mod files. Currently I am trying to run this stochastic synapse model (https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model=8115&file=/fluct/Gfluct.mod#tabs-2) with ephys. But when I try to run, it says "NEURON: Gfluct2 cannot be used with CVODE ".

But I could run this mod on NEURON on its own

A search through the NEURON forum tells me that this is because this mod file is not CVODE compatible and instead NEURON runs it using DASPK. Can ephys also do this?

Werner Van Geit

@wvangeit

Normally yes, if the only problem is Cvode, then you can disable it in the NrnSimulator class: https://github.com/BlueBrain/BluePyOpt/blob/master/bluepyopt/ephys/simulators.py#L19

sunzhenyang2018

@sunzhenyang2018

It worked. Thanks!

Johanna Frost Nylen

@jofrony

Hi, I was wondering how I can access logs from Bluepyopt or even NEURON - we are having a problem of crashes - and we think NEURON is the issue, but we are not getting any helpful error messages.

Werner Van Geit

@wvangeit

Are you using ipyparallel? Can you reproduce the crash on a single machine? Maybe it could help if you disable ipyparallel for a sec, that might allow you to see more the output logging. @DrTaDa, you have maybe any other ways of introspecting things?

Where communities thrive

People

Repo info

Activity