Shared (abstract) EEG/neuroimaging types

[marcpabst]

Hey all,

Not sure if it would be useful at all, but I'm starting to do more and more EEG analysis in Julia and I was wondering if there is a case for introducing some sort of shared (abstract) EEG and/or other neuroimaging types. They could behave like normal vectors or matrices but could also store EEG metadata and allow for time-based indexing. They could also easily be converted to and from other structures (e.g., to interface with PyMNE).

I know that this is not necessarily a great fit for Unfold.jl, but seeing that you're already putting a lot of infrastructure for plotting and statistical analysis in place, I feel it's a good place to ask if there i any interest.

[behinger]

Hey!
Thanks for reaching out! I have seen this discussion multiple times, most recently I participated in NeuroCore.jl - but this is already a year ago or so...

Prior datastructures are defined in at least

• Neuroimaging.jl (previously EEG.jl) (not active)
• NeuroAnalyzer.jl (active, 1 contributor, very large dependencies, https://discourse.julialang.org/t/ann-neuroanalyzer-jl/89574)
• NeuroAnalysis.jl (new, LFP)
• BrainWave.jl (deprecated)
• NeuroCore.jl (status unclear, https://github.com/JuliaNeuroscience/NeuroCore.jl/ - some discussion on dataformat in here)

I think NeuroAnalyzer.jl has most features, Neuroimaging.jl is the oldest, NeuroCore.jl in principle the right place, but no development even after asking from multiple people.

I'm interested in participating and I could be a long-term maintainer, but I dont have capacity to kickstat it. Does that make sense?

[marcpabst]

Hey Benedikt, thanks for getting back to me!

I actually remember that this topic came up a few times, and I participated in at least one of those discussions. Usually, I then stopped doing EEG for some time, and upon returning, noticed not much progress had been made. However, I think something has changed now - Julia is at a point where it is actually usable and ahead of the alternatives for a lot of numerical stuff!

In regards to the projects you mentioned, I tend to agree:

• Neuroimaging.jl looks nice but hasn't been updated in a long time.
• I'm not terribly excited about the approach NeuroAnalyzer.jl is taking, mostly for the reasons you mentioned.
• NeuroCore.jl indeed seems like the right approach, although I haven't seen it being used anywhere. Honestly, going through the code and discussions, it looks like NeuroCore.jl failed because it tried to do too much at once.. For example, I don't know of anyone who is doing fMRI stuff in Julia, but there seem to be a fair few people working with EEG/MEG/physiological data.

So my preliminary idea would be to

1. Start with a very lightweight set of abstract data types or traits. This could be based on Julia's AbstractArray types, or follow the NeuroCore approach - although I'm not sure traits are the best way to handle this (traits are great, but my understanding is that they lack first-class language support and don't integrate well with other parts of the Julia ecosystem. For example, I'm not sure if you can create Makie recipes based on traits? Also, they tend to be ugly in Julia).
2. Provide some useful infrastructure around these types, mainly by implementing a wrapper around PyMNE objects, so you get access to a lot of pre-existing infrastructure; and by providing Makie integration (I think Unfold.jl plotting tools are a really good place to start). For example, I'm yet to see a nice and repsonsive (!!) viewer for raw EEG data...
3. Ping everyone who is working on these kinds of things in Julia and open PRs in projects to add support for these new types/traits.

Does this approach sound reasonable to you? Are there any other people you know of who might be interested in this project?

[marcpabst]

I mean in principle we could 1:1 copy the BIDS interface and implement all "files" as one AbstractType? E.g. AbstractChannels, AbstractEvents, AbstractElectrodes, AbstractCoordsystem, AbstractEEG (what you called AbstractMetadata - I actually prefer your name there, AbstractEEG should be the container containing all the others maybe, )*

I like that! I did some pondering, and I think we could even split the interface into both types and traits, so that you don't need to use the provided types but can still resort to them for easier implementation. Btw, I think Tables.jl is a good example of leveraging traits for this.

• For example, the events() methods called on a EEG timeseries would be required to return a type that implements the isevents trait (providing the onsets() and durations() methods, which could be part of the default implementation of the AbstractEvents type (which could also implement the Tables.istable() trait).
• Same for electrodes(), which would need to return a type implementing iselectrodes, which ensures compatibility with the BIDS standard for encoding electrode information/location. Default implementation would be provided through the AbstractEEGElectrodes type (which again could implement the Tables.jl interface by default)
• As you said, the default type for metadata() could be called AbstractMetadata (or maybe better AbstractEEGMetadata) which would then implement the iseegmetadata() trait, and in turn either provide all the allowed BIDS fields as methods (and by default just return nothing) or make them accessible through a getfield(m, ::Symbol) method.

Using traits instead of types would allow to implement these things on top of existing AbstractTypes - so you might decide to inherit from an AbstractArray or AbstractVector for your events instead of the provided AbstractEvents type.

At the top of the (logical) hierarchy, we could then have an AbstractMultivariateTimeseries type (and we also could have AbstractEEG and AbstractMEG based on that) - this type in turn implements different useful traits using default/fallback methods, like the istimeseries (see below) trait. On top of that, this could also implemen the Tables.istable trait, so it can be fed to anything that accepts a Tables.jl table, like CSV.jl, AlgebraOfGraphic - you can even do DataFrame(ts::AbstractMultivariateTimeseries). AbstractMultivariateTimeseries could then also provide the AbstractArray interface (if we want that).

And finally, the istimeseries trait would provide standardised access to the underlying data using a data()method and also come with a set of other useful methods like duration(), nsamples, nobservations, and so on. Plus, it could define methods to get metadata through: metadata(), events(), and observations() as described above.

[behinger]

I tried to understand Tables.jl, but the explanations are relatively tricky because quite abstract.

• Regarding AbstractMetadata vs AbstractEEGMetadata - I was thinking that BIDS uses the same metadata everywhere. could be AbstractEEGMetadata <: AbstractMetadata hierarchy, but let's focus on the EEG bit first :-D
• what are observations in your terminology? repetitions/trials?

[marcpabst]

I agree, Tables.jl is not super straightforward... Here's my undertanding of how it works: When you want to accept a Table-like type t::SomeType, you check if Tables.istable(t) is true (this can be done at compile time, Holy-trait-style). If yes, we say t implements the trait istable.

You then check if t allows for column-access (or row-access) by calling Tables.allowscolumnaccess(t). If true, you can use c = Tables.columns(t) to get a type that imlements the AbstractColumns interface, i.e., Tables.getcolumn(c, i::Int), getcolumn(c, nm::Symbol), columnnames(c) (and might also subtype AbstractColumns, but doesnt have to).

And because it's all implemented via traits, you can have Tables.columns(t) = t, if your original type implements the columns interface.

Regarding AbstractMetadata vs AbstractEEGMetadata - I was thinking that BIDS uses the same metadata everywhere. could be AbstractEEGMetadata <: AbstractMetadata hierarchy, but let's focus on the EEG bit first :-D

I think keeping it as non-specific as reasonably possible is a really good idea!

what are observations in your terminology? repetitions/trials?

I mainly tried to avoid "epochs', because it feels so EEG speicfic. "Observation" feels more in line with the general ML lingo (not sure what the "traditional stats" term would be - "repetition"?) Maybe a EEG subtype could use "epochs" instead?

I have some updated code (and a simple notebook with examples) I'll try to share shortl for some feedback (currently using real, unpublished data, so I will need to replace that with some mock data) :)

[behinger]

Hey! Anything you can share?
There is now also https://slopezpereyra.github.io/EEGToolkit.jl/ - a new package

Happy to spend some time working on this :)

[behinger]

@marcpabst bump?