Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix: 1381 change to the axes attribute meaning #1396

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Fix: 1381 change to the axes attribute meaning #1396

Changes from 2 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
b1ad2e6
sometimes referred to as -> also referred to as
woutdenolf Jun 30, 2024
26315aa
NXdata@axes rectification: contains the default axes, not all axes
woutdenolf Jun 30, 2024
d19ff47
clarify what "equal shape" means
woutdenolf Jul 1, 2024
7844bda
rephrase the fact that a list of axis names is not provided by NXdata…
woutdenolf Jul 1, 2024
d85f56c
clarify that one dimension is covered by more than on axis
woutdenolf Jul 1, 2024
61343dd
capitals DATA in analogy to AXISNAME
woutdenolf Jul 1, 2024
939fb7e
switch order of examples for clarity
woutdenolf Jul 1, 2024
ecd53dc
use "spanned" instead of "covered"
woutdenolf Jul 1, 2024
69e53c9
each dimension can only have one default axis
woutdenolf Jul 17, 2024
9a78a11
remove justification for the complex axes definition
woutdenolf Jul 17, 2024
1c61a09
reword
woutdenolf Jul 17, 2024
6a014b5
reword
woutdenolf Jul 17, 2024
115e631
change quotes
woutdenolf Jul 17, 2024
c8e5d6c
change quotes
woutdenolf Jul 17, 2024
27cfde3
reword
woutdenolf Jul 17, 2024
37306ef
express length(axes) == rank(data) in NXDL
woutdenolf Aug 15, 2024
e37a676
NIAC suggestions: start with simple example at the top, histogram axi…
woutdenolf Aug 15, 2024
0bf8672
review
woutdenolf Aug 20, 2024
6b23f02
#edges = #data_points + 1
woutdenolf Aug 20, 2024
c27e2d3
WIP
woutdenolf Sep 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 124 additions & 78 deletions base_classes/NXdata.nxdl.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@

<doc>
The :ref:`NXdata` class is designed to encapsulate all the information required for a set of data to be plotted.
NXdata groups contain plottable data (sometimes referred to as *signals* or *dependent variables*) and their
associated axis coordinates (sometimes referred to as *axes* or *independent variables*).
NXdata groups contain plottable data (also referred to as *signals* or *dependent variables*) and their
associated axis coordinates (also referred to as *axes* or *independent variables*).

The actual names of the :ref:`DATA &lt;/NXdata/DATA-field&gt;` and :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields
can be chosen :ref:`freely &lt;validItemName&gt;`, as indicated by the upper case (this is a common convention in all NeXus classes).
Expand Down Expand Up @@ -85,124 +85,170 @@
.. index:: axes (attribute)
.. index:: coordinates

The :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields contain the axis coordinates associated with the data values.
The names of all :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields are listed in the
:ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute.
The :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields contain the axis coordinates associated with the signal values.

`Rank`
`Default axes`

:ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields are typically one-dimensional arrays, which annotate one of the dimensions.
The :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute provides the names of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;`
fields to be used as the default axis for each dimension of the :ref:`data &lt;/NXdata/DATA-field&gt;` fields.
As a result the length of the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute must be equal to the rank of the :ref:`data &lt;/NXdata/DATA-field&gt;`
woutdenolf marked this conversation as resolved.
Show resolved Hide resolved
fields. When a particular dimension has no default axis, the string “.” is used in that position.

An example of this would be
`Spanned dimensions`

The :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes describe the
:ref:`data &lt;/NXdata/DATA-field&gt;` dimensions spanned by the corresponding :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields.

When :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` is missing for a given :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field,
the position(s) of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field name in the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute
is(are) used. The shape of an :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field must be equal to the shape of the
PeterC-DLS marked this conversation as resolved.
Show resolved Hide resolved
:ref:`data &lt;/NXdata/DATA-field&gt;` dimensions it spans.

Note that an :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field can have more than one dimension and can therefore span
more than one :ref:`data &lt;/NXdata/DATA-field&gt;` dimension. Conversely, one :ref:`data &lt;/NXdata/DATA-field&gt;` dimension
can be spanned by more than one :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field. As stated before, the default
axis name (if any) can be found in the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute.

`Available axes`

With the current definition of the :ref:`axes &lt;/NXdata@axes-attribute&gt;` and :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes
PeterC-DLS marked this conversation as resolved.
Show resolved Hide resolved
it is not straightforward to discover all available axes. All strings in the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute
PeterC-DLS marked this conversation as resolved.
Show resolved Hide resolved
(excluding the “.” string) are axis field names. In addition the prefix of an attribute ending with the string "_indices" is also
an axis field name.

This indirect way of discovering axes exists so that data which does not have the :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes
does not become invalid. Such data requires the length of the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute to be equal to the rank of the
:ref:`data &lt;/NXdata/DATA-field&gt;` fields.

**Axes examples:**

`1. Single-dimensional axes`

:ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields are typically one-dimensional arrays that span a single :ref:`data &lt;/NXdata/DATA-field&gt;` dimension.

An example of this would be 2D data on a regular coordinate grid

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", "y"] --> the order matters
@axes = ["x", "y"]
data: float[10,20]
x: float[10] --> coordinates along the first dimension
y: float[20] --> coordinates along the second dimension

In this example each data point ``data[i,j]`` has axis coordinates ``[x[i], y[j]]``.

However, the fields can also have a rank greater than 1, in which case the rank of each
:ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` must be equal to the number of data dimensions it spans.
Note that `@x_indices` and `@y_indices` attributes can be omitted in this case. However it is strongly encouraged to provide them.

`2. Multi-dimensional axes`

When coordinates do not form a regular grid, multi-dimensional :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields
can be used. Just remember that the shape of an :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field must be equal to the shape
of the :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions it spans.

An example of this would be
An example of this would be 2D scatter data where each data point has unique coordinates

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", "y"] --> the order does NOT matter
@axes = ["x", "y"]
@x_indices = [0, 1]
@y_indices = [0, 1]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions
y: float[10,20] --> coordinates along both dimensions

In this example each data point ``data[i,j]`` has axis coordinates ``[x[i,j], y[i,j]]``.

`Dimensions`

The data dimensions annotated by an :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field are defined by the
:ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attribute. When this attribute is missing,
the position(s) of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` string in the
:ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute are used.
In this example each data point ``data[i,j]`` has axis coordinates ``[x[i,j], y[i,j]]`` and when
plotting, `x` is used along the first data dimension by default and `y` along the second data dimension.
Since `x` and `y` span both dimensions, a reader could choose to use `y` for the first dimension
and `x` for the second as an alternative to the default. A writer could also choose to not specify
any default by specifying `@axes = [".", "."]` and leave the decision up the the reader.

When all :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields are one-dimensional, and none of the data dimensions
have more than one axis, the :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes
are often omitted. If one of the data dimensions has no :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field,
the string “.” can be used in the corresponding index of the axes list.
Note that omitting `@x_indices` would result in `@x_indices = [0]` as derived from the position of `"x"` in `@axes`.
woutdenolf marked this conversation as resolved.
Show resolved Hide resolved
This would be invalid since the shape `[10,20]` of `x` is not equal to the shape `[10]` of the spanned data dimensions.

An example of this would be
Omitting indices for multi-dimensional axes can only be done by repeating the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;`
woutdenolf marked this conversation as resolved.
Show resolved Hide resolved
name in all positions of the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute which they span.

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", ".", "z"] --> the order matters
data: float[10,20,30]
x: float[10] --> coordinates along the first dimension
z: float[30] --> coordinates along the third dimension
@axes = ["x", "x"]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions

Note that since ``NXdata`` does not describe how the data is to be plotted or even the dimensionality of the plot,
a reader would probably treat this particular example as a 1D signal and plot it as such.

`3. Dimensions spanned by more than one axis`
PeterC-DLS marked this conversation as resolved.
Show resolved Hide resolved

When using :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` this becomes
In the case of multi-dimensional axes, single-dimensional axes are often introduced as default axes
to support readers that does not use the :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes
or cannot handle multi-dimensional axes.

The 2D scatter example can be enriched as follows

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", "z"] --> the order does NOT matter
data: float[10,20,30]
@x_indices = 0
@z_indices = 2
x: float[10] --> coordinates along the first dimension
z: float[30] --> coordinates along the third dimension

When providing :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes it is recommended
to do it for all axes.

`Non-trivial axes`
@axes = ["x_set", "y_set"]
@x_indices = [0, 1]
@y_indices = [0, 1]
@x_set_indices = 0 --> or [0]
@y_set_indices = 1 --> or [1]
data: float[10,20]
x: float[10,20] --> coordinates along both dimensions
y: float[10,20] --> coordinates along both dimensions
x_set: float[10] --> coordinates along the first dimension
y_set: float[20] --> coordinates along the second dimension

What follows are two examples where :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes
cannot be omitted.
Technically `@x_set_indices` and `@y_set_indices` can be omitted. However it is strongly encouraged to provide them.

The first is an example where data dimensions have alternative axis coordinates. The NXdata group represents
a stack of images collected at different energies. The ``wavelength`` is an alternative axis of ``energy``
for the last dimension (or vice versa).
Another common case is the need to specify alternative axes for the same dimension

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", "y", "energy", "wavelength"] --> the order does NOT matter
@x_indices = 0
@y_indices = 1
@energy_indices = 2
@wavelength_indices = 2
data: float[10,20,30]
x: float[10] --> coordinates along the first dimension
y: float[20] --> coordinates along the second dimension
energy: float[30] --> coordinates along the third dimension
wavelength: float[30] --> coordinates along the third dimension

The second is an example with coordinates that span more than one dimension. The NXdata group represents data
from 2D mesh scans performed at multiple energies. Each data point ``data[i,j,k]`` has axis coordinates
``[x[i,j,k], y[i,j,k], energy[k]]``.
@axes = ["eta", "q"]
data: float[10,20]
@eta_indices = [0]
@tth_indices = [1]
@q_indices = [1]
eta: float[10] --> coordinates along the first dimension
tth: float[20] --> coordinates along the second dimension (alternative)
q: float[20] --> coordinates along the second dimension (default)

Note that to recognize `tth` as an axis, `@tth_indices` must be present. Readers that do
not use :ref:`AXISNAME_indices &lt;/NXdata@AXISNAME_indices-attribute&gt;` attributes will
have no way to understand that `tth` is an axis and simple ignore it.

`4. Axes without defaults`

Expanding on the 2D scatter example, a stack of 2D scatter data where the stack dimension
does not have an axis can be described as follows

.. code-block::

data:NXdata
@signal = "data"
@axes = ["x", "y", "energy"] --> the order does NOT matter
@x_indices = [0, 1, 2]
@y_indices = [0, 1, 2]
@energy_indices = 2
data: float[10,20,30]
x: float[10,20,30] --> coordinates along all dimensions
y: float[10,20,30] --> coordinates along all dimensions
energy: float[30] --> coordinates along the third dimension
@axes = [".", "x_set", "y_set"]
@x_indices = [1, 2]
@y_indices = [1, 2]
@x_set_indices = 1
@y_set_indices = 2
data: float[5,10,20]
x: float[10,20] --> coordinates along the last two dimensions
y: float[10,20] --> coordinates along the last two dimensions
x_set: float[10] --> coordinates along the second dimension
y_set: float[20] --> coordinates along the third dimension

Note that the first dimension does not have a default axis but this doesn't necessarily mean
the first dimension is not spanned by any axis, although in this example that is the case.

**Uncertainties:**

Expand Down Expand Up @@ -319,8 +365,8 @@
The ``AXISNAME_indices`` attribute is a single integer or an array of integers that defines which :ref:`data &lt;/NXdata/DATA-field&gt;`
dimension(s) are spanned by the corresponding axis. The first dimension index is ``0`` (zero).

When the ``AXISNAME_indices`` attribute is missing for an :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field, its value becomes the index
(or indices) of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` name in the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute.
When the ``AXISNAME_indices`` attribute is missing for a given :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` field, its value becomes
the index (or indices) of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` name in the :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute.

.. note:: When ``AXISNAME_indices`` contains multiple integers, it must be saved as an actual array
of integers and not a comma separated string.
Expand All @@ -331,7 +377,9 @@
.. index:: plotting

The ``axes`` attribute is a list of strings which are the names of the :ref:`AXISNAME &lt;/NXdata/AXISNAME-field&gt;` fields
that contain the values of the coordinates along the :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions.
to be used as the default axis along all :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions. As a result the length must
be equal to the rank of the :ref:`data &lt;/NXdata/DATA-field&gt;` fields. The string "." can be used for
dimensions without default axes.

.. note:: When ``axes`` contains multiple strings, it must be saved as an actual array
of strings and not a single comma separated string.
Expand All @@ -341,14 +389,12 @@
<!-- Data and coordinate fields -->
<field name="AXISNAME" type="NX_CHAR_OR_NUMBER" nameType="any">
<doc>
Coordinate values along one or more :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions. The rank must be equal
to the number of dimensions it spans.
Coordinate values along one or more :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions. The shape of an ``AXISNAME`` field
must be equal to the shape of the :ref:`data &lt;/NXdata/DATA-field&gt;` dimensions it spans.

As the upper case ``AXISNAME`` indicates, the names of the ``AXISNAME`` fields can be chosen :ref:`freely &lt;validItemName&gt;`.
The :ref:`axes &lt;/NXdata@axes-attribute&gt;` attribute can be used to find all datasets in the
``NXdata`` that contain coordinate values.

Most AXISNAME fields will be sequences of numbers but if an axis is better represented using names, such as channel names,
Most ``AXISNAME`` fields will be sequences of numbers but if an axis is better represented using names, such as channel names,
an array of NX_CHAR can be provided.
</doc>
<attribute name="long_name"><doc>Axis label</doc></attribute>
Expand Down
Loading