Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No inner sections #525

Merged
merged 33 commits into from
Feb 14, 2025
Merged

No inner sections #525

merged 33 commits into from
Feb 14, 2025

Conversation

sanbrock
Copy link
Collaborator

No description provided.

@mkuehbach mkuehbach mentioned this pull request Feb 7, 2025
5 tasks
Copy link
Collaborator

@lukaspie lukaspie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some code cleanup, the content looks good to me.

Comment on lines 161 to 173
for sec in entry.m_all_contents():
if isinstance(sec, ActivityStep):
sec_c = sec.m_copy()
self.steps.append(sec_c)
ref = NexusActivityStep(name=sec.name, reference=sec)
self.steps.append(ref)
elif isinstance(sec, basesections.Instrument):
ref = InstrumentReference(name=sec.name)
ref.reference = sec
ref = InstrumentReference(name=sec.name, reference=sec)
self.instruments.append(ref)
elif isinstance(sec, CompositeSystem):
ref = CompositeSystemReference(name=sec.name)
ref.reference = sec
ref = CompositeSystemReference(name=sec.name, reference=sec)
self.samples.append(ref)
elif isinstance(sec, ActivityResult):
sec_c = sec.m_copy()
self.results.append(sec_c)
ref = NexusActivityResult(name=sec.name, reference=sec)
self.results.append(ref)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion to use a mapping

mapping = {
    ActivityStep: (NexusActivityStep, self.steps),
    basesections.Instrument: (InstrumentReference, self.instruments),
    CompositeSystem: (CompositeSystemReference, self.samples),
    ActivityResult: (NexusActivityResult, self.results),
}

for sec in entry.m_all_contents():
    for cls, (ref_cls, collection) in mapping.items():
        if isinstance(sec, cls):
            collection.append(ref_cls(name=sec.name, reference=sec))
            break

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@rettigl
Copy link
Collaborator

rettigl commented Feb 13, 2025

With latest version of nomad/develop and this branch, I get a number of parsing errors for the mpes example data:
Several occurrences of errors like this:

INFO:     127.0.0.1:55042 - "POST /fairdi/nomad/latest/api/v1/uploads/OfXtBW8yQim9ak4OcANbdA/action/process HTTP/1.1" 200 OK
WARNING  nomad.processing     2025-02-13T16:27:12 error while setting field
  - exception: Traceback (most recent call last):
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/pynxtools/src/pynxtools/nomad/parser.py", line 336, in _populate_data
        name_metainfo_def = resolve_variadic_name(
                            ^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/util.py", line 373, in resolve_variadic_name
        raise ValueError(f'Cannot find a proper definition for name "{name}".')
    ValueError: Cannot find a proper definition for name "Photoemission intensity__name".
  - exception_hash: qGCggkHx0o3uqHY-Jy8QajweKlzt
  - nomad.commit:
  - nomad.deployment: oasis
  - nomad.entry_id: T8VZzx-qAZlgaukdVNIjNS-1aRil
  - nomad.mainfile: TiTe2/TiTe2_0deg.nxs
  - nomad.processing.logger: nomad.processing
  - nomad.processing.parser: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.proc: Entry
  - nomad.processing.process: process_entry
  - nomad.processing.process_status: RUNNING
  - nomad.processing.process_worker_id: 1D3RTnpvQkqzyfCkp1DBkg
  - nomad.processing.step: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.target_name: DATA__field
  - nomad.service: worker
  - nomad.upload_id: OfXtBW8yQim9ak4OcANbdA
  - nomad.version: 1.3.14.dev154+geeb5b7cd3
  - taskName: None

It can however be that these files indeed don't adhere to the NXmpes standard.
Other errors (these occur quite often):

WARNING  nomad.processing     2025-02-13T16:27:15 error while setting field
  - exception: Traceback (most recent call last):
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/pynxtools/src/pynxtools/nomad/parser.py", line 354, in _populate_data
        current.m_set(stat_metainfo_def, stat)
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 1251, in m_set
        return definition.__set__(self, value, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 720, in wrapper
        return method(self, obj, value, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 3491, in __set__
        raise TypeError(
    TypeError: Cannot set a repeating subsection directly, modify the list, e.a. via append.
  - exception_hash: 2paGVYVBt9CLS4ypINed1amcE6N-
  - nomad.commit:
  - nomad.deployment: oasis
  - nomad.entry_id: ONPVN4bDe87p4zSRfC2wliz5g9Oq
  - nomad.mainfile: Scan1496.nxs
  - nomad.processing.logger: nomad.processing
  - nomad.processing.parser: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.proc: Entry
  - nomad.processing.process: process_entry
  - nomad.processing.process_status: RUNNING
  - nomad.processing.process_worker_id: 3lWqhJH8Qli8aZaIDd46Hg
  - nomad.processing.step: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.target_name: incident_polarization__field
  - nomad.service: worker
  - nomad.upload_id: OfXtBW8yQim9ak4OcANbdA
  - nomad.version: 1.3.14.dev154+geeb5b7cd3
  - taskName: None
WARNING  nomad.processing     2025-02-13T16:27:18 error while setting attribute
  - exception: Traceback (most recent call last):
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/pynxtools/src/pynxtools/nomad/parser.py", line 251, in _populate_data
        current.m_set(metainfo_def, attribute)
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 1251, in m_set
        return definition.__set__(self, value, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 720, in wrapper
        return method(self, obj, value, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/metainfo.py", line 3319, in __set__
        m_quantity.value = self.type.normalize(
                           ^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/data_type.py", line 787, in normalize
        return self._normalize_impl(value, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/data_type.py", line 1072, in _normalize_impl
        return self._validate_value(value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/data_type.py", line 1054, in _validate_value
        if (new_value := value.split('.')[-1]) in self:
                         ^^^^^^^^^^^
    AttributeError: 'int' object has no attribute 'split'
  - exception_hash: TFgZop1f94CmHe-jcoN67tyC8ZoA
  - nomad.commit:
  - nomad.deployment: oasis
  - nomad.entry_id: ONPVN4bDe87p4zSRfC2wliz5g9Oq
  - nomad.mainfile: Scan1496.nxs
  - nomad.processing.logger: nomad.processing
  - nomad.processing.parser: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.proc: Entry
  - nomad.processing.process: process_entry
  - nomad.processing.process_status: RUNNING
  - nomad.processing.process_worker_id: 3lWqhJH8Qli8aZaIDd46Hg
  - nomad.processing.step: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.target_name: vector
  - nomad.service: worker
  - nomad.upload_id: OfXtBW8yQim9ak4OcANbdA
  - nomad.version: 1.3.14.dev154+geeb5b7cd3
  - taskName: None
WARNING  nomad.processing     2025-02-13T16:27:31 error while setting field
  - exception: Traceback (most recent call last):
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/pynxtools/src/pynxtools/nomad/parser.py", line 350, in _populate_data
        stat_metainfo_def = resolve_variadic_name(
                            ^^^^^^^^^^^^^^^^^^^^^^
      File "/home/femtolab_admin/nomad/nomad-distro-dev/packages/nomad-FAIR/nomad/metainfo/util.py", line 373, in resolve_variadic_name
        raise ValueError(f'Cannot find a proper definition for name "{name}".')
    ValueError: Cannot find a proper definition for name "cdeform_field__std".
  - exception_hash: 7PJC0_KHANSaawD2LMtqUSJ3P_Ru
  - nomad.commit:
  - nomad.deployment: oasis
  - nomad.entry_id: Lo_1v2viPSMcsX6F8B0EW2seB_PQ
  - nomad.mainfile: MoTe2.mpes.nxs
  - nomad.processing.logger: nomad.processing
  - nomad.processing.parser: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.proc: Entry
  - nomad.processing.process: process_entry
  - nomad.processing.process_status: RUNNING
  - nomad.processing.process_worker_id: PIAwCfZjRWSL-fr3KyhiFw
  - nomad.processing.step: pynxtools.nomad.entrypoints:nexus_parser
  - nomad.processing.target_name: cdeform_field__field
  - nomad.service: worker
  - nomad.upload_id: OfXtBW8yQim9ak4OcANbdA
  - nomad.version: 1.3.14.dev154+geeb5b7cd3
  - taskName: None

Copy link
Collaborator

@lukaspie lukaspie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -34,7 +34,7 @@ jobs:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv pip install coverage coveralls
- name: Install nomad
if: "${{ matrix.python_version != '3.8'}}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at some point we should just drop support for python<3.10, like NOMAD did. I can do this in a separate PR.

@sanbrock sanbrock merged commit e985c92 into master Feb 14, 2025
17 checks passed
@sanbrock sanbrock deleted the no_inner_sections branch February 14, 2025 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants