Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arbor cable cell exporter and backend #393

Merged
merged 53 commits into from
Jan 17, 2023
Merged
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
3d9e7fa
Arbor cable cell label_dict and decor generation from create_hoc (tes…
lukasgd Mar 10, 2022
fc52a06
Arbor cable cell, label dict and decor generation fixed and done sepa…
lukasgd Mar 10, 2022
3e20642
Unit tests for create_acc (Arbor cable cell output)
lukasgd Mar 14, 2022
dbf766f
Fixed Arbor mechanism output, separated create_hoc from create_acc ex…
lukasgd Mar 31, 2022
fba5376
Separate module for create_acc
lukasgd Apr 7, 2022
8bd6539
Improved documentation for create_acc
lukasgd Apr 7, 2022
d2f2f42
Fixed membrane capacitance conversion to Arbor, formatting
lukasgd Apr 8, 2022
22ad6da
Fixed pycodestyle errors
lukasgd Apr 11, 2022
2228979
Fixed tox tests/docs, replace_axon handling fixed (unsupported in Arb…
lukasgd May 16, 2022
760f53c
Merge branch 'master' into arbor_integration
lukasgd May 16, 2022
315457c
Merge branch 'master' into arbor_integration
wvangeit Jun 1, 2022
dccc375
Merge branch 'master' into arbor_integration
wvangeit Jun 23, 2022
f72690d
Merge branch 'master' into arbor_integration
wvangeit Jun 24, 2022
e88168f
Support for replace_axon in Arbor, cable cell construction from JSON/…
lukasgd Jul 28, 2022
c0af192
Formatting fix, support for myelinated section in Arbor cable cell (c…
lukasgd Jul 28, 2022
27827fb
Merge branch 'master' into arbor_integration
wvangeit Jul 29, 2022
de52dea
Making Arbor an extra dependency
lukasgd Jul 29, 2022
08c9157
Merge branch 'arbor_integration' of github.com:lukasgd/BluePyOpt into…
lukasgd Jul 29, 2022
152033e
Moved Arbor all to default properties, explicit mechanism qualificati…
lukasgd Aug 2, 2022
f2ca1ed
Support for range parameters in Arbor via iexprs, Arbor mechanism cat…
lukasgd Aug 5, 2022
2c67258
Added existing section lists to instantiated cell to prevent crash on…
lukasgd Aug 5, 2022
ab004f6
Arbor package data fixes
lukasgd Aug 8, 2022
0a8a932
Merge branch 'master' into arbor_integration
lukasgd Aug 8, 2022
e685990
Validation of simulation output with Arbor vs. Neuron for L5PC mechs …
lukasgd Aug 11, 2022
26f3af1
Minor changes on ACC exporter
lukasgd Aug 24, 2022
a7453f8
Adding ACC morphology output with axon replacement
lukasgd Aug 25, 2022
79dda2e
Merge branch 'master' into arbor_integration
wvangeit Aug 29, 2022
fac077d
Support for point processes in Arbor cable cell exporter
lukasgd Sep 24, 2022
6aa15de
Arbor locations/labels, stimuli, protocols and optimization in simple…
lukasgd Oct 2, 2022
cde9b96
Using all segments of Neuron sections in axon replacement
lukasgd Oct 3, 2022
6d76ed1
L5PC optimization with Arbor
lukasgd Oct 4, 2022
3978d31
Axon replacement with Arbor morphologies, split_at/join_at and Neuron…
lukasgd Oct 12, 2022
5d4ad03
Removing old axon replacement impl and synapse implementation with sp…
lukasgd Oct 12, 2022
d98041d
Support for general Arbor labels, expsyn example fixed, Arbor iexpr g…
lukasgd Oct 17, 2022
c425101
External mechanism catalogues for Arbor simulator
lukasgd Oct 18, 2022
9443b12
avoid using string concatenation in joining paths
anilbey Oct 19, 2022
d958c14
Removing assertions, adding another Arbor iexpr generation test
lukasgd Oct 20, 2022
f2d93e3
Merge branch 'arbor_integration' of github.com:lukasgd/BluePyOpt into…
lukasgd Oct 20, 2022
4bc6424
More iexpr tests, mech metadata type in ACC exporter, Arbor label mov…
lukasgd Oct 20, 2022
e83dbc0
Renamed l5pc Arbor-Neuron validation
lukasgd Oct 20, 2022
ebd32d0
Adding docstrings to create_acc, replaced os by pathlib
lukasgd Oct 21, 2022
9bbc68e
Python 3 exception handling in protocols, better create_hoc error mes…
lukasgd Oct 23, 2022
bcada48
Use kwargs for cable cell constructor and swap label-dict and decor
lukasgd Oct 24, 2022
b973162
tox use the same EXTRA_ARBOR defined in setup.py
anilbey Oct 24, 2022
9dd09e6
create a submodule for arbor's dsl inside parameterscalers
anilbey Oct 25, 2022
4148974
Renaming Arbor iexpr module, adding docstring
lukasgd Oct 25, 2022
73772ea
Integrating Anil's review including large extension of the create_acc…
lukasgd Nov 7, 2022
e85d695
Fixing expsyn generate_acc docs
lukasgd Nov 7, 2022
93c0685
Fixing create_acc tests
lukasgd Nov 9, 2022
02a00fb
Anil's review for ACC exporter, fixing create_acc tests
lukasgd Dec 21, 2022
04ce614
Integrating more feedback from Anil, fixed L5PC ACC test
lukasgd Jan 4, 2023
5c9a5bf
Merge branch 'master' into arbor_integration
lukasgd Jan 10, 2023
70fddf8
Merge branch 'master' into arbor_integration
lukasgd Jan 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Improved documentation for create_acc
lukasgd committed Apr 7, 2022
commit 8bd65393a8bb3efd319aa2b448c6bb7ad07bc25c
78 changes: 45 additions & 33 deletions bluepyopt/ephys/create_acc.py
100755 → 100644
Original file line number Diff line number Diff line change
@@ -14,10 +14,11 @@


# Define Neuron to Arbor variable conversions
ArbVar = namedtuple('ArbVar', 'name, conv', defaults=[None,None])
ArbVar = namedtuple('ArbVar', 'name, conv',
defaults=[None,None])

_nrn2arb_var = dict(
cm=ArbVar(name='membrane-capacitance'),
cm=ArbVar(name='membrane-capacitance'), # conv=None implies identity
lukasgd marked this conversation as resolved.
Show resolved Hide resolved
ena=ArbVar(name='ion-reversal-potential \"na\"'),
ek=ArbVar(name='ion-reversal-potential \"k\"'),
v_init=ArbVar(name='membrane-potential'),
@@ -28,45 +29,53 @@


def _nrn2arb_var_name(name):
"""Neuron to Arbor variable renaming."""
return _nrn2arb_var[name].name if name in _nrn2arb_var else name


def _nrn2arb_var_value(param):
"""Neuron to Arbor variable value conversion."""
if param.name in _nrn2arb_var and _nrn2arb_var[param.name].conv is not None:
return _nrn2arb_var[param.name].conv(param.value)
else:
return param.value


def _make_arb_global_param(loc, param):
def _arb_is_global_param(loc, param):
"""Returns if location-specific variable is a global one in Arbor."""
return loc == 'all' and param.name in ['membrane-capacitance']

# Define region mapping (relabeling locations to SWC convention)

# Define BluePyOpt to Arbor region mapping (relabeling locations to SWC convention)
# Remarks:
# - using SWC convetion: dend == basal dendrite, apic == apical dendrite
# - using SWC convetion: 'dend' for basal dendrite, 'apic' for apical dendrite
# - myelinated is unsupported in Arbor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add support for myelinated in Arbor? Do you know how these section are defined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not completely sure where this is used (it doesn't seem to be in l5pc). A quick search finds it in several .hoc files in examples/stochkv. Does somebody else have insights on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'myelinated' is a sectionlist we're using in our latest models. It's a cylinder attached to the end of the AIS to prevent boundary effects. It doesn't have channels but just represent a myelinated chunk of axon

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wvangeit for the explanation. I assume AIS means axon initial segment. Is there an example/morphology with myelinated? Otherwise, I'd suggest to throw an exception for now until we can properly test that in Arbor.

Copy link
Contributor

@wvangeit wvangeit Jun 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have this in open source models yet. But it's fairly simple code though. It's not specified in the morphology file itself, but used in the replace_axon function when constructing the model:

        sim.neuron.h.execute('create myelin[1]', icell)                                                                                                                                                                                 
        icell.myelinated.append(sec=icell.myelin[0])                                                                                                                                                                                    
        icell.all.append(sec=icell.myelin[0])                                                                                                                                                                                           
        icell.myelin[0].nseg = 5                                                                                                                                                                                                        
        icell.myelin[0].L = 1000                                                                                                                                                                                                        
        icell.myelin[0].diam = diams[count-1]                                                                                                                                                                                           
        icell.myelin[0].connect(icell.axon[1], 1.0, 0.0)         

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested - it works with your code above added to replace_axon. I've added the necessary changes to c0af192, but they're currently commented out as I have no way to figure out if the myelin section exists without crashing in Neuron.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure, but i wonder if hasattr():
https://www.w3schools.com/python/ref_func_hasattr.asp
would work on icell?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works on myelin and the len(self.icell.myelin) == 1 looks like a bug given that NEURON crashes, complaining that the section was deleted upon accessing e.g. self.icell.myelin[0]. I've added safe support for myelin now in 2c67258.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's look at this in a separate issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #419.

# - could use ('(all)', None) instead, then "all" undefined
ArbRegion = namedtuple('ArbRegion', 'ref, defn')

def _arb_defined_region(region, expr):
def _make_region(region, expr=None):
"""Create Arbor region with region name and defining expression
(name for decor, defined in label_dict) or region expression only
(for decor, no defined label in label_dict)."""
if expr is not None:
return ArbRegion(ref='(region \"%s\")' % region,
defn='(region-def \"%s\" %s)' % (region, expr))
else:
return ArbRegion(ref=region, defn=expr)


def _arb_tagged_region(region, tag):
return _arb_defined_region(region, '(tag %i)' % tag)
def _make_tagged_region(region, tag):
return _make_region(region, '(tag %i)' % tag)


_loc2arb_region = dict(
all=_arb_defined_region('all', '(all)'),
somatic=_arb_tagged_region('soma', 1),
axonal=_arb_tagged_region('axon', 2),
basal=_arb_tagged_region('dend', 3),
apical=_arb_tagged_region('apic', 4),
myelinated=_arb_defined_region(None, None),
# defining "all" region for convenience here, else use
# all=_arb_defined_region('(all)') to omit "all" in label_dict
all=_make_region('all', '(all)'),
somatic=_make_tagged_region('soma', 1),
axonal=_make_tagged_region('axon', 2),
basal=_make_tagged_region('dend', 3),
apical=_make_tagged_region('apic', 4),
myelinated=_make_region(None),
)

# # Generated with NMODL in arbor/mechanisms
@@ -151,8 +160,9 @@ def _arb_tagged_region(region, tag):
'pas': {'globals': ['e'], 'ranges': ['g']}}
)

def _find_mech_and_split_param_name(param, mechs):
"""TODO: doc"""

def _find_mech_and_convert_param_name(param, mechs):
"""Find a parameter's mechanism and convert parameter name to Arbor convention"""
mech_suffix_matches = numpy.where([param.name.endswith("_" + mech)
for mech in mechs])[0]
if mech_suffix_matches.size == 0:
@@ -168,41 +178,41 @@ def _find_mech_and_split_param_name(param, mechs):
(param.name, repr(mechs[mech_suffix_matches])))


def _split_mech_from_non_mech_params_global(params, channels):
"""TODO: doc"""
mech_params = [_find_mech_and_split_param_name(
def _arb_convert_params_and_group_by_mech_global(params, channels):
"""Group global parameters by mechanism and rename them to Arbor convention"""
lukasgd marked this conversation as resolved.
Show resolved Hide resolved
mech_params = [_find_mech_and_convert_param_name(
Location(name=name, value=value), channels['all'])
for name, value in params.items()]
mechs = {mech: [] for mech, _ in mech_params}
for mech, param in mech_params:
mechs[mech].append(param)
if len(mechs) > 0:
assert list(mechs.keys()) == [None]
return {param.name: param for param in mechs[None]} # correct?
return {param.name: param for param in mechs[None]}
else:
return {}


def _split_mech_from_non_mech_params_local(params, channels):
"""TODO: doc"""
def _arb_convert_params_and_group_by_mech_local(params, channels):
"""Group section parameters by mechanism and rename them to Arbor convention"""
local_params = []
global_params = {}
for loc, params in params:
mech_params = [_find_mech_and_split_param_name(
mech_params = [_find_mech_and_convert_param_name(
param, channels[loc]) for param in params]
mechs = {mech: [] for mech, _ in mech_params}
for mech, param in mech_params:
mechs[mech].append(param)
for i, param in enumerate(mechs.get(None,[])):
if _make_arb_global_param(loc, param):
if _arb_is_global_param(loc, param):
global_params[param.name] = param
del mechs[None][i]
local_params.append((loc, list(mechs.items())))
return local_params, global_params


def _arb_mech_translate(mech_name, mech_params):
"""TODO: doc"""
def _arb_nmodl_global_translate(mech_name, mech_params):
"""Integrate NMODL GLOBAL parameters of Arbor-built-in mechanisms into mechanism name"""
arb_mech = None
for cat in ['bbp', 'default', 'allen']: # in order of precedence
if mech_name in _arb_mechs[cat]:
@@ -227,14 +237,15 @@ def _arb_mech_translate(mech_name, mech_params):
return (arb_mech_name, arb_mech_params)


def _nrn_to_arb_mechs_local(params):
def _arb_nmodl_global_translate_local(params):
ret = []
for loc, mechs in params:
ret.append((loc, [_arb_mech_translate(*mech) for mech in mechs]))
ret.append((loc, [_arb_nmodl_global_translate(*mech) for mech in mechs]))
return ret


def _read_templates(template_dir, template_filename):
"""Expand Jinja2 template filepath with glob and return dict of target filename -> parsed template"""
if template_dir is None:
template_dir = os.path.abspath(
os.path.join(
@@ -314,11 +325,12 @@ def create_acc(mechs,
range_params = template_params['range_params']

global_params = \
_split_mech_from_non_mech_params_global(global_params, channels)
_arb_convert_params_and_group_by_mech_global(global_params, channels)
section_params, additional_global_params = \
_split_mech_from_non_mech_params_local(section_params, channels)
global_params.update(additional_global_params) # TODO: global translate?
section_params = _nrn_to_arb_mechs_local(section_params)
_arb_convert_params_and_group_by_mech_local(section_params, channels)
global_params.update(additional_global_params)
# no nmodl translate on global_params as no mechs
section_params = _arb_nmodl_global_translate_local(section_params)
# TODO: range_params = _split_mech_from_non_mech_params_local(range_params, channels)

template_params['global_params'] = global_params
1 change: 1 addition & 0 deletions bluepyopt/ephys/create_hoc.py
100755 → 100644
Original file line number Diff line number Diff line change
@@ -113,6 +113,7 @@ def _generate_parameters(parameters):


def _read_template(template_dir, template_filename):
"""Read Jinja2 hoc template to render"""
if template_dir is None:
template_dir = os.path.abspath(
os.path.join(
3 changes: 3 additions & 0 deletions examples/l5pc/generate_acc.py
Original file line number Diff line number Diff line change
@@ -13,6 +13,9 @@
decor = arbor.load_component("test_acc/" + cell_json["decor"]).component'
An Arbor cable cell is then created with
cell = arbor.cable_cell(morpho.morphology, labels, decor)
The resulting cable cell can be output to ACC for visual inspection
in the Arbor GUI (File > Cable cell > Load) using
arbor.write_component(cell, "l5pc.acc")
'''
import os
import argparse
3 changes: 3 additions & 0 deletions examples/simplecell/generate_acc.py
Original file line number Diff line number Diff line change
@@ -13,6 +13,9 @@
decor = arbor.load_component("test_acc/" + cell_json["decor"]).component'
An Arbor cable cell is then created with
cell = arbor.cable_cell(morpho, labels, decor)
The resulting cable cell can be output to ACC for visual inspection
in the Arbor GUI (File > Cable cell > Load) using
arbor.write_component(cell, "simple_cell.acc")
'''
import os
import argparse