Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datatype checking for array dtype #555

Open
rettigl opened this issue Feb 18, 2025 · 1 comment · May be fixed by #554
Open

Datatype checking for array dtype #555

rettigl opened this issue Feb 18, 2025 · 1 comment · May be fixed by #554

Comments

@rettigl
Copy link
Collaborator

rettigl commented Feb 18, 2025

Currently, the data type of numpy arrays is not being checked against the NX datatype, i.e. e.g. numeric arrays are treated as valid entries for NX_CHAR fields, or string arrays as valid entries for NX_NUMBER. Instead, the dtype of an array should be checked against the possible primitive datatypes.
Relevant code:

"NX_CHAR": (str, np.ndarray, *np_char),

Steps to reproduce:
Follow the conversion Notebook here: https://nomad-lab.eu/prod/v1/gui/user/uploads/upload/id/iM3-bEttS060E2-GIqaSBA
The @vector attributes of the transformation classes are defined as NX_CHAR in NXmpes_arpes (which itself is a mistake, to be fixed), yet are passed as numeric arrays to the converter, which accepts them without complaint, and writes numeric arrays to the attribute.

@RubelMozumder
Copy link
Collaborator

@rettigl I tried the *_specsscan_conversion_to_NeXus.ipynb with the current modification available here https://github.com/FAIRmat-NFDI/pynxtools/pull/554/files. With that modification I see the warning message from the mentioned notebook as follows:

Using mpes reader to convert the given files:  
• NXmpes_arpes_config.json
• Scan1496.yaml
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/TRANSFORMATIONS[transformations]/analyzer_rotation/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/TRANSFORMATIONS[transformations]/analyzer_elevation/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/TRANSFORMATIONS[transformations]/analyzer_dispersion/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/sample_polar/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/offset_polar/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/sample_tilt/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/offset_tilt/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/sample_azimuth/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/SAMPLE[sample]/TRANSFORMATIONS[transformations]/offset_azimuth/@vector should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/data/@energy_indices should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/data/@angular0_indices should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/data/@angular1_indices should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: Field [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/entry_identifier written without documentation.
WARNING: The value at [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/data/@axes should be one of: (<class 'str'>, <class 'numpy.str_'>, <class 'numpy.bytes_'>), as defined in the NXDL as NX_CHAR.
WARNING: Field [/ENTRY](https://file+.vscode-resource.vscode-cdn.net/ENTRY)[entry]/data/@axes written without documentation.
The output file generated: Scan1496.nxs.

Here I see that warning massage from vector attribute. You may also see that errors from *_indices which are defined as NX_CHAR in app def.

@RubelMozumder RubelMozumder linked a pull request Feb 20, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants