Sin wave current stimulus [v3.last] proposal [update]

[SimoneTorsello]

Hello everyone! Here we are again! :)

Originally posted by @SimoneTorsello in dbbs-lab/bsb-core#721 (comment)

I finally managed to create a package called sinclamp, but I'm still unable to insert it into the simulation. The procedure is as follows and requires Astrocyte and nrn-glia (I'm using version v3.latest):

1. Install Astrocyte and create the package:

   astro create package my-package

2. Generate the desired .mod file and load it:

   astro add mod /path/to/mod/file

3. Run the commands to build and install the package:

   astro build
   astro install

4. I don't know if it's necessary, but I also used:

   glia compile

5. After confirming the installation, I used:

   glia list

The installed packages are:

sinclamp, patch_extensions, dbbs_mod_collection

Great!

Now, however, I can't manage to load the tool into the .json simulation file, as in the following example:

"sinclamp_stim": {
     "io": "input",
     "device": "sinclamp",
     "targetting": "cell_type",
     "cell_types": [
          "purkinje_cell"
     ],
     "section_types": [
          "soma"
     ],
     "parameters": {
          "delay": 0.0,
          "duration": 0.0,
          "amplitude": 0.0,
          "dur": 400.0,
          "pkamp": 1.0,
          "freq": 1.0,
          "phase": 0.0,
          "bias": 0.0
     }
}

I am obviously getting this error:

  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/core.py", line 123, in _intialise_components
    self._initialise_simulations()
  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/core.py", line 185, in _initialise_simulations
    self._initialise_simulation(simulation)
  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/core.py", line 194, in _initialise_simulation
    device.initialise(self)
  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/simulation/targetting.py", line 9, in initialise
    super().initialise(scaffold)
  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/helpers.py", line 53, in initialise
    self.validate()
  File "/home/simone/.pyenv/versions/BSB_tool/lib/python3.9/site-packages/bsb/simulators/neuron/adapter.py", line 110, in validate
    raise ConfigurationError(
bsb.exceptions.ConfigurationError: Unknown device 'sinclamp' for simulations.test_oscillazioni.devices.sinclamp_stim
-------------------------------------------------------
Primary job terminated normally, but 1 process returned a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was:

  Process name: [[43726,1],12]
  Exit code:    1
--------------------------------------------------------------------------

Now: I have observed the __init__.py file, and the mechanisms are included in the collection; however, they are not actual NEURON tools but only ion channels.

Am I lost again, or am I finally close to a solution?

Thanks a lot, and have a great evening.

Originally posted by @SimoneTorsello in dbbs-lab/bsb-core#721 (comment)

[Helveg]

Hi! Can you try to move to v4 of the BSB: it is the only major version we currently offer open source support for.

Then you'd have to clarify a bit what you are trying to do, I'm seeing mixed things happening here.

From your description it appears that you are trying to package your own NMODL file using Glia? Then you follow it up with snippets of BSB configurations, and I'm seeing what I believe to be an MPI related error message tail in your logs.

That's a lot of moving parts at the same time: problems with Glia, and doing MPI-parallel tasks with the BSB. Why don't we start simple and see if we can simply use Glia to insert your mechanism in a NEURON section.

The latest version of Glia is actually called nmodl-glia, uninstall the one you have, even better, throw away the whole environment, start a new virtual environment, and install the newest version: https://nmodl-glia.readthedocs.io/en/latest/

Then there is a guide that explains how to create your own Glia package and add your own NMODL file to it: https://nmodl-glia.readthedocs.io/en/latest/index.html#document-package (Glia 4 also supports a "local library", so you wouldn't even need a package, you could just do glia add mod --local sinclamp.mod)

The guide mentions to flit publish to make your package public, but you can also install and use it locally with pip install -e .

Afterwards, say you have a sinclamp named asset you should be able to:

import glia
from patch import p

s = p.Section()
glia.insert(s, 'sinclamp')

Let me know how that goes, and then we can continue helping you getting started with NEURON simulations in BSB 4

[SimoneTorsello]

Thanks for the reply! Ok, in the meantime I will try to migrate to v4. Maybe it could be a quicker solution.

My intention is to integrate an oscillatory electrical stimulation tool (precisely sine wave).

In v3.10 using the json file I can indicate all the simulation commands.
For this:

import glia
from patch import p

s = p.Section()
glia.insert(s, 'sinclamp')

I wouldn't know how to integrate it.

I generated this Python file to automate the whole process which works great with the stock version of BSB v3.10.:

import os
import shutil
import time
import datetime
os.system('ls')
from CHANGE_CONFIG_HDF5 import name_simulation, sim_dur, net_file

start = time.time()
os.system(f'mpirun -n 16 bsb -v=3 simulate {name_simulation} --hdf5 {net_file}.hdf5')
total = time.time() - start

path = os.path.abspath(os.getcwd())
test = os.listdir(path)

time1 = datetime.datetime.now().isoformat(sep='_', timespec='hours')[:]
time2 = datetime.datetime.now().isoformat(sep='_', timespec='minutes')[14:]

for item in test:
    if item.startswith(f"results_{name_simulation}"):
        NAME_new = f'{time1}_{time2}_{name_simulation}_{sim_dur}_NEURON.hdf5'
        clean_name = NAME_new.replace("-", "_")
        os.rename(item, clean_name)
        new_path = path+"/SIMULATIONS/"
        shutil.move(path + "/" + clean_name, 
                    new_path + "/" + clean_name)

Then I will gradually start moving all the work to another environment with v4.

In the meantime, thanks again for the reply.

I will update you in future developments.

[Helveg]

My intention is to integrate an oscillatory electrical stimulation tool (precisely sine wave).

For which scope? For NEURON users, or for BSB users? You do not necesarily need to involve the BSB for this purpose. You can write your NMODL file, and package/distribute it with Glia. And then all NEURON users can already use your tool. It is another additional step to write a BSB component for your NEURON tool. Let's go step by step

I wouldn't know how to integrate it.

We're not yet at the stage of integrating anything, the code snippet I gave you is a standalone test that confirms whether your Glia package works. First make the Glia package, and prove that it works with this code snippet. (PS: It is not something to "integrate", it is plain python code that you should run with python myfile.py)

I generated this Python file to automate the whole process which works great with the stock version of BSB v3.10.:

So what your script does is run an MPI-BSB simulation on 16 nodes on an (to me) unknown network reconstruction file, and then moves the HDF5 result files around. I'm still missing what configuration file was used to reconstruct that network?

[SimoneTorsello]

For which scope? For NEURON users, or for BSB users? You do not necesarily need to involve the BSB for this purpose. You can write your NMODL file, and package/distribute it with Glia. And then all NEURON users can already use your tool. It is another additional step to write a BSB component for your NEURON tool. Let's go step by step

I would like this sine wave device to be integrated inside BSB to test the behavior of the network as naturally as possible, so I assume that the last part of the answer is the key step. I'll start by trying to create the package with Glia. The goal however is to use it with BSB and use the device on the network (as if it were a NEURON icurrent device).

We're not yet at the stage of integrating anything, the code snippet I gave you is a standalone test that confirms whether your Glia package works. First make the Glia package, and prove that it works with this code snippet. (PS: It is not something to "integrate", it is plain python code that you should run with python myfile.py)

Ok, I'll try with Glia package in meanwhile.

So what your script does is run an MPI-BSB simulation on 16 nodes on an (to me) unknown network reconstruction file, and then moves the HDF5 result files around. I'm still missing what configuration file was used to reconstruct that network?

Exact. A JSON configuration file with the appropriate settings (cerebellar network) is loaded and the results moved to hdf5.

[Helveg]

Exact. A JSON configuration file with the appropriate settings (cerebellar network) is loaded and the results moved to hdf5.

That's not what appears to be happening:

os.system(f'mpirun -n 16 bsb -v=3 simulate {name_simulation} --hdf5 {net_file}.hdf5')

This uses {net_file}.hdf5, a reconstructed network file! This does not show which configuration file you have used to create {net_file}.hdf5, please post it, if you're allowed to publicly share it, if not you can email it confidentially to [email protected]. The file I am looking for is a JSON file that describes the network. If you don't know which configuration was used, that's fine, you'll have to recreate it at a later point either way, but it helps me understand your context.

[SimoneTorsello]

That's not what appears to be happening:

os.system(f'mpirun -n 16 bsb -v=3 simulate {name_simulation} --hdf5 {net_file}.hdf5')

This uses {net_file}.hdf5, a reconstructed network file! This does not show which configuration file you are using to create that file, please post it, if you're allowed to publicly share it, if not you can email it confidentially to [email protected]. The file I am looking for is a JSON file that describes the network.

This is because I use some scripts to handle the configuration file for the simulation:

from bsb.config import HDF5Formatter, JSONConfig
import json

name_simulation = "test"
sim_duration = 400
net_file = "60_8_CORE"

with open('JS_TEST.json', 'r+') as f:

    data = json.load(f)
    test = data['simulations']
    old = test.keys()
    old_key = list(old)[0]
    if name_simulation == old_key:
        pass
    else:
        test[name_simulation] = test[old_key]
        del test[old_key]

    sim = test[name_simulation]

    # DURATION OF SIM
    sim_dur = sim['duration'] = sim_duration

    # BACKGROUND STIMULATION PARAMETERS
    devices = sim['devices']
    bckgr = devices['background']
    bp_param = bckgr['parameters']
    dl = bp_param['delay'] = 0.0
    dr = bp_param['duration'] = sim_dur
    amp = bp_param['amplitude'] = 0.0045

[ecc.....]

network_file = f"{net_file}.hdf5"
new_config = JSONConfig("JS_TEST.json")
HDF5Formatter.reconfigure(network_file, new_config)
print(f'\nNEW CONFIG DONE!\nSIMULATION {name_simulation} LOADED.')

The network has been created ex novo by using:

from bsb.core import Scaffold
from bsb.config import JSONConfig
from bsb.reporting import set_verbosity
import warnings
import numpy as np
warnings.filterwarnings("ignore", category=np.VisibleDeprecationWarning) 
import time

set_verbosity(3)
config = JSONConfig(file='JS_TEST.json')
scaffoldInstance = Scaffold(config)

start = time.time()
scaffoldInstance.compile_network()
total = time.time() - start
print(total/60, 'minuti')

Don't worry, for now it isn't confidential. I can post it here.
JS_TEST.json

Just to be clear, I would like to make a device in order to integrate it into BSB's JSON file as well:

                "devices": {
                    "NEW_DEVICE": {
                         "io": "input",
                         "device": "NEW_DEVICE",
                         "targetting": "cell_type",
                         "cell_types": [
                              "CELL"
                         ],
                         "section_types": [
                              "SECTION"
                         ],
                         "parameters": {
                              "PARM_NEW_DEVICE1": nnn,
                              "PARM_NEW_DEVICE2": mmm,
                              "PARM_NEW_DEVICE3": xxx
                         }
                    },

[Helveg]

OK, so your configuration is based on the cerebellar model of the DBBS laboratory. I would urge you, when developing new things, to make your test cases as simple as possible, to eliminate moving parts and sources of errors, and to reduce the complexity of your test scenario.

Let me know what happens once you have created the Glia package and have tried to run the test snippet I sent you.

[SimoneTorsello]

Of course, I am already using a scaffold of very small size and few neurons which in a few minutes is able to return a preliminary result without too much effort. Clearly the problem is to get at least as far as a simulation. These days I will work on the transfer to 4.0

I will keep the 3.10 for basic analysis, as I know how it works very well.

I'll keep you updated soon

[Helveg]

Ok, but just to confirm you understand that at this point we are not ready to use any BSB at all. Please send me the output of the following code, ran in a Python interpreter:

import glia
from patch import p

s = p.Section()
glia.insert(s, 'sinclamp')

[SimoneTorsello]

Of course, I understood.
We are far from solution.

This is the output:

/home/simone/.pyenv/versions/3.9.0/envs/BSB_tool/bin/python /home/simone/PROGETTI_NEURO/NEURO_SIM/Cereb_NEURON/TEST.py
/home/simone/.pyenv/versions/3.9.0/envs/BSB_tool/lib/python3.9/site-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.14) or chardet (5.2.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "

numprocs=1

[Helveg]

/home/simone/.pyenv/versions/3.9.0/envs/BSB_tool/lib/python3.9/site-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.14) or chardet (5.2.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "

This warning is probably fine. If not, try updating urllib to v2

numprocs=1

This is a message NEURON sends when it initializes MPI, so that's fine too.

---

It looks like you've succesfully inserted your sinclamp in the section. Why don't you set some parameters and make some plots to confirm that it works? Please refer to the Glia/Patch docs for setting parameters, or go oldschool and use the original NEURON interface. I can not offer you any support on the NMODL/NEURON side of things, but if you run into problems you can always ask for support through the regular NEURON support channels. Glia injects the mechanism under a mapped name, but is still accessible through the NEURON interface, in case any of the supporting NEURON devs wonder about that :) Feel free to link them this issue as well should they have further questions about Glia in your code snippet!

I'm looking forward to seeing some plots :) If all is well we can move to include your Glia package in a BSB4 model :)

[SimoneTorsello]

Please refer to the Glia/Patch docs for setting parameters, or go oldschool and use the original NEURON interface.

On NEURON, I have already developed and introduced this tool (I later discovered that there was already one called 'izap').

I'm looking forward to seeing some plots :) If all is well we can move to include your Glia package in a BSB4 model :)

That would be fantastic! For now, these are simple equations, but it would be interesting to introduce this oscillatory tool into much more complex circuits!

Great, I'll get to work on testing Glia! I’ll try to produce some graphs. ;)