Skip to content

Commit

Permalink
Merge pull request #88 from FAIRmat-NFDI/bring-in-examples
Browse files Browse the repository at this point in the history
Bring in NOMAD examples
lukaspie authored Nov 29, 2024
2 parents 1d8c392 + 8175be8 commit be2f10d
Showing 15 changed files with 2,305 additions and 16 deletions.
4 changes: 4 additions & 0 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
@@ -33,6 +33,10 @@ jobs:
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
uv pip install coverage coveralls
- name: Install nomad
if: "${{ matrix.python_version != '3.8'}}"
run: |
uv pip install nomad-lab[infrastructure]@git+https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR.git
- name: Install package
run: |
uv pip install ".[dev]"
5 changes: 5 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
prune *
exclude *
recursive-include src/pynxtools_xps *.py *.json
include pyproject.toml README.md dev-requirements.txt
graft src/pynxtools_xps/nomad/examples
24 changes: 19 additions & 5 deletions docs/tutorial/standalone.md
Original file line number Diff line number Diff line change
@@ -25,11 +25,25 @@ An example script to run the XPS reader in `pynxtools`:
user@box:~$ dataconverter $<xps-file path> $<xps-file path> $<eln-file path> --reader xps --nxdl NXxps --output <output-file path>.nxs
```

Note that none of the supported file format have data/values for all required and recommended fields and attributes in NXxps. In order for the validation step of the XPS reader to pass, you need to provide an ELN file that contains the missing values. Example raw and converted data can be found in [*pynxtools_xps/examples*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples).
Note that none of the supported file format have data/values for all required and recommended fields and attributes in ``NXxps``. In order for the validation step of the XPS reader to pass, you need to provide an ELN file that contains the missing values.

TODO: add more steps! <!--[The Jupyter notebook is available here](https://github.com/FAIRmat-NFDI/pynxtools-em/blob/main/examples/HowToUseTutorial.ipynb) TODO!-->
### Examples

You can find examples how to use `pynxtools-xps` for your XPS research data pipeline in [`src/pynxtools-xps/nomad/examples`](../../src/pynxtools_xps/nomad/examples/). These are designed for working with [`NOMAD`](https://nomad-lab.eu/) and its [`NOMAD Remote Tools Hub (NORTH)`](https://nomad-lab.eu/prod/v1/gui/analyze/north). Feel invited to try out the respective tutorial [here](tutorial/nomad.md).

There are also small example files with raw and converted data for using the `pynxtools` dataconverter with the `mpes` reader and the `NXmpes` application definition in the [`examples`](../../examples/) folder.

**Congrats! You now have a FAIR NeXus file!**
For this tutorial, we will work with the example data for the VAMAS reader (see here [](../../examples/vms/)). You can run the conversion as
```shell
dataconverter \\
--reader xps \\
--nxdl NXmpes \\
regular.vms \\
eln_data_vms.yaml \\
-c config_file.json \\
--output regular.vms.nxs
```

TODO: add more steps! <!--[The Jupyter notebook is available here](https://github.com/FAIRmat-NFDI/pynxtools-em/blob/main/examples/HowToUseTutorial.ipynb) TODO!-->

The above-mentioned parsing is also integrated into the NOMAD research data management system.
Feel invited to try out the respective tutorial [here]((tutorial/nomad.md)
**Congrats! You now have a FAIR NeXus file!**
24 changes: 13 additions & 11 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -25,18 +25,10 @@ dependencies = [
"h5py>=3.6.0",
"igor2",
"xarray>=0.20.2",
"numpy>=1.21.2",
'numpy>=1.22.4,<2.0.0',
"pint>=0.17",
"pynxtools>=0.7.0",
"pynxtools>=0.9.0",
]

[project.entry-points."pynxtools.reader"]
xps = "pynxtools_xps.reader:XPSReader"

[project.urls]
"Homepage" = "https://github.com/FAIRmat-NFDI/pynxtools-xps"
"Bug Tracker" = "https://github.com/FAIRmat-NFDI/pynxtools-xps/issues"

[project.optional-dependencies]
dev = [
"mypy",
@@ -60,6 +52,16 @@ docs = [
"mkdocs-click"
]

[project.urls]
"Homepage" = "https://github.com/FAIRmat-NFDI/pynxtools-xps"
"Bug Tracker" = "https://github.com/FAIRmat-NFDI/pynxtools-xps/issues"

[project.entry-points."pynxtools.reader"]
xps = "pynxtools_xps.reader:XPSReader"

[project.entry-points.'nomad.plugin']
xps_example = "pynxtools_xps.nomad.entrypoints:xps_example"

[tool.setuptools.packages.find]
where = [
"src",
@@ -82,7 +84,7 @@ select = [
"E", # pycodestyle
"W", # pycodestyle
"PL", # pylint
"NPY201",
# "NPY201",
]
ignore = [
"E501", # Line too long ({width} > {limit} characters)
38 changes: 38 additions & 0 deletions src/pynxtools_xps/nomad/entrypoints.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#
# Copyright The NOMAD Authors.
#
# This file is part of NOMAD. See https://nomad-lab.eu for further info.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""Entry points for XPS examples."""

try:
from nomad.config.models.plugins import ExampleUploadEntryPoint
except ImportError as exc:
raise ImportError(
"Could not import nomad package. Please install the package 'nomad-lab'."
) from exc

xps_example = ExampleUploadEntryPoint(
title="X-ray Photoelectron Spectroscopy (XPS)",
category="FAIRmat examples",
description="""
This example presents the capabilities of the NOMAD platform to store and standardize X-ray Photoelectron Spectroscopy XPS data.
It shows the generation of a NeXus file according to the
[NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes)
application definition from an example measurement file and a subseqeuent analysis of that data set.
""",
plugin_package="pynxtools_xps",
resources=["nomad/examples/*"],
)
137 changes: 137 additions & 0 deletions src/pynxtools_xps/nomad/examples/E1 XPS data conversion to NeXus.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "38f1916e-2d0f-4f81-9b00-fedc615a96aa",
"metadata": {},
"source": [
"# XPS conversion example\n",
"\n",
"In this notebook a XPS measurement file from a SPECS detector (using the native SPECS .sle format) is read and converted into the [NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) NeXus standard."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "dc099289-78f5-4357-b7d3-715dd20179da",
"metadata": {},
"source": [
"## Create a NeXus file from measurement data\n",
"\n",
"To convert the available files to the NeXus format we use the convert function readily supplied by [`pynxtools`](https://github.com/FAIRmat-NFDI/pynxtools)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7d2c1df4-1d96-4255-a18e-e323c69d32b4",
"metadata": {},
"outputs": [],
"source": [
"from pynxtools.dataconverter.convert import convert, logger\n",
"import logging\n",
"logger.setLevel(logging.ERROR)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "bf441135-d4c5-4310-ae6e-39c87da1b726",
"metadata": {},
"source": [
"The input parameters are defined as follows:\n",
"\n",
"**input_file**: The input files for the reader. This is a sle file from [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) (v.63), which is the propietary format of SPECS GmbH,and a YAML ELN file containing additional information not contained in the measurement file (e.g. user name).\n",
"\n",
"**reader**: The specific reader which gets called inside `pynxtools`. This is supplied in the [`pynxtools-xps`](https://github.com/FAIRmat-NFDI/pynxtools-xps) reader plugin. For XPS data, the reader is called `xps`.\n",
"\n",
"**nxdl**: The specific NXDL application definition to which the converted file should conform. For XPS this should be `NXmpes`, the subdefinition `NXxps`, or any further subdefinitions of the form `NXxps_<name>`.\n",
" \n",
"**output**: The output filename of the NeXus file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e8a76ff-918e-483d-9ed1-5417613710e1",
"metadata": {},
"outputs": [],
"source": [
"convert(\n",
" input_file=[\"EX439_S718_Au_in_25_mbar_O2.sle\", \"eln_data.yaml\"],\n",
" reader='xps',\n",
" nxdl='NXmpes',\n",
" remove_align=True,\n",
" output='Au_25_mbar_O2_no_align.nxs'\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fa5fe119-0a72-4744-a84d-4d1258f55031",
"metadata": {},
"source": [
"## View the data with H5Web\n",
"\n",
"H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above.\n",
"\n",
"You can also view this data with the H5Viewer or other tools from your local filesystem."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d4ff822a-4552-49b8-ba17-9b86fd8c2ac1",
"metadata": {},
"outputs": [],
"source": [
"from jupyterlab_h5web import H5Web"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f62097ae-eb0f-4572-b8d9-bebc7266b43a",
"metadata": {},
"outputs": [],
"source": [
"H5Web(\"Au_25_mbar_O2_no_align.nxs\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.10"
},
"vscode": {
"interpreter": {
"hash": "0cbdf5d5ef28617c8bf3753ff15cd1b7b5539de5aaa68a35c3d38ca27e1ab0fa"
}
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,388 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "38f1916e-2d0f-4f81-9b00-fedc615a96aa",
"metadata": {},
"source": [
"# XPS data analysis example\n",
"\n",
"In this notebook a XPS measurement file from a SPECS detector (using the native SPECS .sle format) that has already been converted into the [NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) NeXus standard is read and some basic data analysis (a fit of one Au 4f spectrum) is done."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "fa5fe119-0a72-4744-a84d-4d1258f55031",
"metadata": {},
"source": [
"## View the data with H5Web\n",
"\n",
"H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above.\n",
"\n",
"You can also view this data with the H5Viewer or other tools from your local filesystem."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d4ff822a-4552-49b8-ba17-9b86fd8c2ac1",
"metadata": {},
"outputs": [],
"source": [
"from jupyterlab_h5web import H5Web"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f62097ae-eb0f-4572-b8d9-bebc7266b43a",
"metadata": {},
"outputs": [],
"source": [
"H5Web(\"Au_25_mbar_O2_no_align.nxs\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "359770eb-964f-48fd-97da-84038af10193",
"metadata": {},
"source": [
"## Analyze data\n",
"\n",
"First, we need to import the necessarry packages. We use h5py for reading the NeXus file, lmfit for fitting and the class XPSRegion from the provided `xps_region.py` file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ed892838-0f76-47a8-89b6-ba8bba4f9048",
"metadata": {},
"outputs": [],
"source": [
"import h5py\n",
"from xps_region import XPSRegion\n",
"\n",
"from lmfit.models import GaussianModel"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "a30f1ecb-d9f9-44eb-b9d3-f61709109d6a",
"metadata": {},
"source": [
"### Load data and plot\n",
"\n",
"We want to load the Au 4f spectrum from the Au foil from our measurement file. Feel free to adapt to different regions in the file by changing the `MEASUREMENT` variable."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "400205d5",
"metadata": {},
"outputs": [],
"source": [
"MEASUREMENT = \"Au_in_vacuum__Au4f\"\n",
"\n",
"with h5py.File(\"Au_25_mbar_O2_no_align.nxs\", \"r\") as xps_file:\n",
" binding_energy = xps_file[f\"/{MEASUREMENT}/data/energy\"][:]\n",
" cps = xps_file[f\"/{MEASUREMENT}/data/data\"][:]\n",
" cps_err = xps_file[f\"/{MEASUREMENT}/data/data_errors\"][:]"
]
},
{
"cell_type": "markdown",
"id": "5c88ac8c",
"metadata": {},
"source": [
"There is also a convenience function in XPSRegion to directly load the data: "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1f654ad7",
"metadata": {},
"outputs": [],
"source": [
"au4f = XPSRegion.load(\"Au_25_mbar_O2_no_align.nxs\", MEASUREMENT) "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "0f6ffc43-6e23-4f54-8a8a-0192a21368ca",
"metadata": {},
"source": [
"With the loaded data we create the `au4f` `XPSRegion` containing the measurement data."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "129db648-3c47-4f81-9ee8-69788c96660e",
"metadata": {},
"outputs": [],
"source": [
"au4f = XPSRegion(binding_energy=binding_energy, counts=cps, counts_err=cps_err) "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "1c500a40-b915-44ba-a8c4-a9c37f3f7f4f",
"metadata": {},
"source": [
"`XPSRegion` provides us a function to visualize the loaded data with"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "211918d9-8974-4ae2-97d7-51c6920fe6cb",
"metadata": {},
"outputs": [],
"source": [
"au4f.plot()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cf8cb26e-df35-4ef3-bea3-e9ddcc452c2e",
"metadata": {},
"source": [
"### Fit data\n",
"\n",
"From the preview plot we can detect two symmetric peaks which result from the spin-orbit splitting into the Au 4f5/2 and 4f3/2 regions. For illustration of the typical analysis routine, we construct two Gaussian peaks with the lmfit GaussianModel and initialize them with appropriate start values. Here we are just using initial good guesses for the start values. These, however, can eventually be deduced by data inside NOMAD as soon as enough data is available, e.g. similar to a peak detection in other XPS analysis programs. There are different peak shapes available in lmfit, such as Lorentz, Voigt, PseudoVoigt or skewed models. Please refer to the packages documentation for further details on these models and on how to use them."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6401d0af-7830-4093-ae2e-d1c4a719ff61",
"metadata": {},
"outputs": [],
"source": [
"peak_1 = GaussianModel(prefix=\"Au4f52_\")\n",
"peak_1.set_param_hint(\"amplitude\", value=3300)\n",
"peak_1.set_param_hint(\"sigma\", value=0.5)\n",
"peak_1.set_param_hint(\"center\", value=84.2)\n",
"\n",
"peak_2 = GaussianModel(prefix=\"Au4f32_\")\n",
"peak_2.set_param_hint(\"amplitude\", value=1600)\n",
"peak_2.set_param_hint(\"sigma\", value=0.5)\n",
"peak_2.set_param_hint(\"center\", value=87.2)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "25c2b42c-3e74-435e-88c6-64e5f04a2244",
"metadata": {},
"source": [
"We can simply add the two models together to create a composite model."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "62d6293b-f562-425d-b39a-22c8b3874676",
"metadata": {},
"outputs": [],
"source": [
"comp = peak_1 + peak_2\n",
"params = comp.make_params()"
]
},
{
"cell_type": "markdown",
"id": "1c0c03fa",
"metadata": {},
"source": [
"We also set a constraint, namely that the area of `peak_2` is exactly half the area of `peak_1` (since it is a photoemission doublet).\n",
"\n",
"To constrain the areas correctly, we need to set the expression for the amplitude of `peak_2` considering both the amplitude and sigma. The constraint should be:\n",
"\n",
"$$\\text{area of peak 2} = 0.5 \\times \\text{area of peak 1}$$\n",
"\n",
"Since the area $A$ of a Gaussian peak is given by:\n",
"\n",
"$$ A = \\text{amplitude} \\times \\sigma \\times \\sqrt{2\\pi}$$\n",
"\n",
"For `peak_2` to have half the area of `peak_1`:\n",
"\n",
"$$ \\text{amplitude}_2 \\times \\sigma_2 = 0.5 \\times (\\text{amplitude}_1 \\times \\sigma_1) $$\n",
"\n",
"So, the correct expression for the amplitude of `peak_2` should be:\n",
"\n",
"$$ \\text{amplitude}_2 = 0.5 \\times \\text{amplitude}_1 \\times \\frac{\\sigma_1}{\\sigma_2} $$\n",
"\n",
"Therefore, we can write:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b31762bd",
"metadata": {},
"outputs": [],
"source": [
"params['Au4f32_amplitude'].expr = '0.5 * Au4f52_amplitude * (Au4f52_sigma / Au4f32_sigma)'"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "2aaf4749-62d6-4cec-af67-8daced4d3ed9",
"metadata": {},
"source": [
"In the next step, we perform the actual fit. First, since the data in `Au_in_vacuum__Au4f` contains a very wide scan range, we only select the region with the Au 4f doublet (with `fit_region(...)`). Then, we calculate a Shirley baseline with `calc_baseline()`, set the fit model (`.fit_model(comp)`) and perform a fit (`.fit()`). All of this functions can also be used independently. The fit function takes the measurement uncertainties as weights to the fit function into account.\n",
"\n",
"Finally, the model is plotted with the previously used `plot()` method. Since we performed a fit the plot is now extended by the baseline and fits."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c48fbeb7-1e46-4c0b-897c-3907654d033b",
"metadata": {},
"outputs": [],
"source": [
"au4f.fit_region(start=80,stop=94).calc_baseline().fit_model(comp).fit(params).plot()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "97a42467-5bbb-4c9a-a97a-5d6ed573f5f0",
"metadata": {},
"source": [
"The fit result gets stored inside the `fit_result` parameter and is displayed to extract, e.g., the peak central energies. Please note that the fitting does not take the measurement uncertainties into account and the errors are simple fitting errors."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9345ab8f-b65f-49f2-a260-bf1c7805e4a0",
"metadata": {},
"outputs": [],
"source": [
"au4f.fit_result.params "
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "22d85cef-be55-402e-9083-c40a245fadb1",
"metadata": {},
"source": [
"We can also extract a fitting parameter shared accross different peaks, e.g. the peak central energies. This refers to the text behind the model paramters prefix, so we select `center` here to get the central energies."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b7b17c0-d781-4b62-8274-c82b00f3c267",
"metadata": {},
"outputs": [],
"source": [
"au4f.peak_property('center')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "16cbedf6-0e82-43ca-909a-e826fd8df7e7",
"metadata": {},
"source": [
"Typically, we are also interested in the peak areas which can be calculated with `peak_areas()`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8aa0759b-a920-4491-97ca-54381a85d3a8",
"metadata": {},
"outputs": [],
"source": [
"(areas := au4f.peak_areas())"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f2b57a15-e95b-4dba-a14e-63031ffa3408",
"metadata": {},
"source": [
"and their ratios"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d55e8ea6-3fc9-4049-be1b-9fcec5f27ffd",
"metadata": {},
"outputs": [],
"source": [
"areas / areas.max()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b909f5bc-446b-4c11-b5b8-1eeec79c894d",
"metadata": {},
"source": [
"To assess the quality of the fit, the fit residual can be viewed with `plot_residual()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cfa5fe97",
"metadata": {},
"outputs": [],
"source": [
"au4f.plot_residual()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.10"
},
"vscode": {
"interpreter": {
"hash": "0cbdf5d5ef28617c8bf3753ff15cd1b7b5539de5aaa68a35c3d38ca27e1ab0fa"
}
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Binary file not shown.
35 changes: 35 additions & 0 deletions src/pynxtools_xps/nomad/examples/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# XPS Reader

## Introduction

This example presents the capabilities of the NOMAD platform to store and standardize XPS data. It shows the generation of a NeXus file according to the [NXmpes](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) application definition and a successive analysis of an example data set.

## Viewing uploaded data

Below, you find an overview of your uploaded data.
Click on the `> /` button to get a list of your data or select **FILES** from the top menu of this upload.
You may add your own files to the upload or experiment with the pre-existing electronic lab notebook (ELN) example.
The ELN follows the general structure of NOMAD ELN templates and you may refer to the [documentation](https://nomad-lab.eu/prod/v1/staging/docs/archive.html) or a [YouTube tutorial](https://youtu.be/o5ETHmGmnaI) (~1h)
for further information.
When the ELN is saved a NeXus file will be generated from the provided example data.
You may also view your supplied or generated NeXus files here with the H5Web viewer.
To do so open the **FILES** tab and just select a `.nxs` file.

## Analyzing the data

The examples work through the use of NOMAD remote tools hub (NORTH) containers, i.e. besides using and dealing with the uploaded XPS data, an analysis container can be started. If you want to execute the examples locally you may also use your local python and jupyterlab installation. Please refer to the documentation of [pynxtools](https://github.com/FAIRmat-NFDI/pynxtools.git) and [h5web](https://github.com/silx-kit/h5web) on how to install it on your machine.

Note: To upload your own xps data file, you must check **"eln_data.yaml"** file to provide all the required fields. In xps some required fields or attributes do not come with xps raw data.

To start an analysis, note your upload id (which you find on top of this explanation) and select **ANALYZE** from the top menu, then **NOMAD Remote Tools Hub**.
In the appearing list you'll find the `xps` container, click on it and click **LAUNCH**.
After a few moments a new tab will open which displays a jupyter environment providing the required analysis tools.
To find the examples navigate to uploads inside the jupyter hub browser and select the folder with your noted upload id.
There you'll find the example `ipynb` notebook for data analysis (E2).
Double-clicking the notebook will open the example in the jupyter main window.

## Where to go from here?

If you're interested in using this pipeline and NOMAD in general you'll find support at [FAIRmat](https://www.fairmat-nfdi.eu/fairmat/consortium).

For questions regarding the experiment or this specific example contact [Lukas Pielsticker](https://www.fairmat-nfdi.eu/fairmat/fairmat_/fairmatteam), [Rubel Mozumder](https://www.fairmat-nfdi.eu/fairmat/fairmat_/fairmatteam), or [Florian Dobener](https://www.fairmat-nfdi.eu/fairmat/fairmat_/fairmatteam).
195 changes: 195 additions & 0 deletions src/pynxtools_xps/nomad/examples/Specs_conversion.archive.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
{
"data":{
"m_def":"../upload/raw/xps.scheme.archive.yaml#/definitions/section_definitions/0",
"reader":"xps",
"nxdl":"NXmpes",
"input_files":[
"EX439_S718_Au_in_25_mbar_O2.sle",
"eln_data.yaml"
],
"output": "Au_25_mbar_O2_no_align.nxs",
"title":"EX439_S718_Au in 25 mbar O2",
"start_time":"2022-04-08T09:47:00.000Z",
"end_time":"2022-04-08T10:32:00.000Z",
"entry_identifier":"EX439",
"experiment_institution":"Max Planck Institute for Chemical Energy Conversion",
"experiment_facility":"Surface and Interface Analysis Group",
"experiment_laboratory":"Near-Ambient Pressure XPS Lab",
"user":{
"name":"Lukas Pielsticker",
"affiliation":"MPI CEC",
"address":"Lukas Pielsticker",
"email":"lukas.pielsticker@cec.mpg.de",
"orcid":{
"service":"orcid",
"identifier":"0000-0001-9361-8333"
}
},
"instrument":{
"device_information":{
"vendor":"SPECS GmbH",
"model":"Custom NAP-XPS instrument",
"identifier":"null"
},
"energy_resolution":{
"type":"calibrated",
"resolution":0.2,
"physical_quantity":"energy"
},
"source_probe":{
"type":"Fixed Tube X-ray",
"probe":"photon",
"device_information":{
"vendor":"SPECS GmbH",
"model":"µFOCUS 500",
"identifier":"null"
},
"beam_probe":{
"distance":0.0,
"distance/@units":"mm"
},
"analyser":{
"description":"hemispherical",
"device_information":{
"vendor":"SPECS GmbH",
"model":"PHOIBOS 150 NAP",
"identifier":"null"
},
"collectioncolumn":{
"scheme":"angular dispersive",
"device_information":{
"vendor":"SPECS GmbH",
"model":"PHOIBOS 150 NAP",
"identifier":"null"
}
},
"energydispersion":{
"scheme":"hemispherical",
"diameter":{
"unit":"mm",
"value":150
},
"device_information":{
"vendor":"SPECS GmbH",
"model":"PHOIBOS 150 NAP",
"identifier":"null"
}
},
"detector":{
"amplifier_type":"channeltron",
"detector_type":"Multi-anode",
"device_information":{
"vendor":"Surface Concept GmbH",
"model":"1D-DLD detector",
"identifier":"null"
}
}
},
"manipulator":{
"device_information":{
"vendor":"SPECS GmbH",
"model":"5-axis manipulator",
"identifier":"null"
},
"temperature_sensor":{
"name":"type K thermocouple",
"measurement":"temperature",
"attached_to":"sample",
"type":"type K thermocouple",
"value":{
"unit":"K",
"value":298.0
}
},
"sample_heater":{
"name":"Coherent Compact Evolution IR Diode LASER (DILAS)",
"physical_quantity":"temperature",
"type":"IR diode laser",
"heater_power":{
"unit":"W",
"value":0.0
},
"pid":{
"setpoint":{
"unit":"K",
"value":298.0
}
}
},
"drain_current_amperemeter":{
"name":"Amperemeter 1.0",
"measurement":"current",
"type":"wire",
"heater_power":{
"unit":"nA",
"value":0.1
}
},
"sample_bias_voltmeter":{
"name":"XPS sample voltmeter",
"measurement":"voltage",
"attached_to":"sample",
"type":"oscilloscope",
"heater_power":{
"unit":"V",
"value":0.0
}
},
"sample_bias_potentiostat":{
"name":"XPS sample potentiostat",
"physical_quantity":"voltage",
"type":"potentiostat",
"pid":{
"setpoint":{
"unit":"V",
"value":0.0
}
}
}
},
"pressure_gauge":{
"name":"Atmion",
"measurement":"pressure",
"type":"hot-filament ionization gauge",
"value":{
"unit":"mbar",
"value":1e-9
}
},
"flood_gun":{
"name":"FG 22/35",
"physical_quantity":"current",
"type":"hot-filament ionization gauge",
"current":{
"unit":"A",
"value":0.0
}
}
},
"sample":{
"name":"Polycristalline Au foil",
"sample_id":"S718",
"atom_types":"Au",
"physical_form":"foil",
"situation":"vacuum",
"substance":{
"name":"Au",
"molecular_mass":{
"value":196.96657,
"unit":"g/mol"
},
"cas_number":"7440-57-5",
"molecular_formula_hill":"Au"
},
"history":{
"sample_preparation":{
"start_time":"2022-04-08T11:25:00.200Z",
"end_time":"2022-04-08T11:45:00.200Z",
"description":"sputter cleaned with Ar ions for 20 min",
"method":"Ar sputtering"
}
}
}
}
}
}
159 changes: 159 additions & 0 deletions src/pynxtools_xps/nomad/examples/eln_data.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
title: EX439_S718_Au in 25 mbar O2
start_time: 2022-04-08T11:47:02.0200Z
end_time: 2022-04-08T14:52:26.0400Z
entry_identifier: EX439
experiment_institution: Max Planck Institute for Chemical Energy Conversion
experiment_facility: Surface and Interface Analysis Group
experiment_laboratory: Near-Ambient Pressure XPS Lab
program_name: SpecsLabProdigy
user:
name: Lukas Pielsticker
affiliation: Max Planck Institute for Chemical Energy Conversion
address: Lukas Pielsticker
email: lukas.pielsticker@cec.mpg.de
orcid:
service: orcid
identifier: 0000-0001-9361-8333
instrument:
device_information:
vendor: SPECS GmbH
model: Custom NAP-XPS instrument
identifier: null
energy_resolution:
type: calibrated
resolution:
value: 0.2
unit: eV
source_xray:
type: Fixed Tube X-ray
probe: photon
device_information:
vendor: SPECS GmbH
model: µFOCUS 500
identifier: null
beam_xray:
distance:
value: 0.0
unit: mm
analyser:
description: hemispherical
device_information:
vendor: SPECS GmbH
model: PHOIBOS 150 NAP
identifier: null
collectioncolumn:
scheme: angular dispersive
device_information:
vendor: SPECS GmbH
model: PHOIBOS 150 NAP
identifier: null
energydispersion:
scheme: hemispherical
diameter:
unit: mm
value: 150
device_information:
vendor: SPECS GmbH
model: PHOIBOS 150
identifier: null
detector:
amplifier_type: channeltron
detector_type: Multi-anode
device_information:
vendor: Surface Concept GmbH
model: 1D-DLD detector
identifier: null
manipulator:
device_information:
vendor: SPECS GmbH
model: 5-axis manipulator
identifier: v1.0
temperature_sensor:
name: type K thermocouple
measurement: temperature
attached_to: sample
type: type K thermocouple
value:
value: 298.0
unit: K
sample_heater:
name: Coherent Compact Evolution IR Diode LASER (DILAS)
physical_quantity: temperature
type: IR diode laser
heater_power:
value: 0.0
unit: W
pid:
setpoint:
value: 298.0
unit: K
cryostat:
name: null
physical_quantity: null
type: null
pid:
setpoint: null
drain_current_amperemeter:
name: Amperemeter 1.0
measurement: current
type: wire
value:
value: 0.1
unit: nA
sample_bias_voltmeter:
name: XPS sample voltmeter
measurement: voltage
attached_to: sample
type: oscilloscope
value:
value: 0.0
unit: V
sample_bias_potentiostat:
name: XPS sample potentiostat
physical_quantity: voltage
type: potentiostat
pid:
setpoint:
value: 0.0
unit: V
pressure_gauge:
name: Atmion
measurement: pressure
type: hot-filament ionization gauge
value:
value: 0.000000001
unit: mbar
value_log:
value:
value: null
unit: null
flood_gun:
name: FG 22/35
physical_quantity: current
type: low energy electron source
current:
value: 0.0
unit: A
current_log:
value:
value: null
unit: null
sample:
name: Polycristalline Au foil
sample_id: S718
atom_types: Au
physical_form: foil
situation: vacuum
substance:
name: Au
molecular_mass:
value: 196.96657
unit: g/mol
cas_number: 7440-57-5
molecular_formula_hill: Au
history:
sample_preparation:
start_time: 2022-04-08T11:25:00.200Z
end_time: 2022-04-08T11:45:00.200Z
description: sputter cleaned with Ar ions for 20 min
method: Ar sputtering
887 changes: 887 additions & 0 deletions src/pynxtools_xps/nomad/examples/xps.scheme.archive.yaml

Large diffs are not rendered by default.

350 changes: 350 additions & 0 deletions src/pynxtools_xps/nomad/examples/xps_region.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,350 @@
"""Fitting functions for XPS spectra"""

from typing import Optional
from dataclasses import dataclass
import numpy as np
from numpy.linalg import norm
import plotly.graph_objects as go
import pandas as pd
from h5py import File as H5File
from lmfit import Model, CompositeModel


@dataclass
class XPSRegion:
"""An XPS region representation"""

binding_energy: np.ndarray
counts: np.ndarray
counts_err: np.ndarray
baseline: Optional[np.ndarray] = None
_fit_region: slice = slice(None, None)
_fit_mod: Optional[Model] = None
fit_result: Optional[Model] = None

@staticmethod
def load(filename: str, entry: str) -> "XPSRegion":
"""Load from a NeXus file.
Args:
filename (str): The NeXus file name to load.
entry (str):
The entry from which to load data.
Should be the name of an NXentry within the NeXus file.
Returns:
XPSRegion: The XPSRegion class with the loaded data.
"""
with H5File(filename, "r") as xps_file:
binding_energy = xps_file[f"/{entry}/data/energy"][:]
cps = xps_file[f"/{entry}/data/data"][:]
cps_err = xps_file[f"/{entry}/data/data_errors"][:]

return XPSRegion(binding_energy=binding_energy, counts=cps, counts_err=cps_err)

def fit_region(self, start: int, stop: int) -> "XPSRegion":
"""Select a fit region within this XPSregion by x-axis value.
The fit region is always selected between start and stop, regardless of their order.
Both points are included in the region,
hence the actual selected region may be a little larger.
Args:
start (int): The start ot the region.
stop (int): The end of the region.
Returns:
XPSRegion: This class
"""
region = np.argwhere(
(self.binding_energy >= start) & (self.binding_energy <= stop)
)

self._fit_region = slice(region[0, 0], region[-1, 0], 1)
return self

def fit_model(self, model: Model) -> "XPSRegion":
"""Supply a fit model to fit this xps region.
Args:
model (lmfit.Model): The lmfit model to use.
Returns:
XPSRegion: This class
"""
self._fit_mod = model
return self

def fit(self, *args, **kwargs) -> "XPSRegion":
"""Perform a fit of the data. You need to define a fit_model first and
execute a baseline correction before using this method.
Raises:
ValueError: If no fit model is provided or the baseline has not been corrected.
Returns:
XPSRegion: This class
"""
if self._fit_mod is None:
raise ValueError("You need to provide a fit model before performing a fit.")

if self.baseline is None:
raise ValueError(
"You need to perform a baseline correction before using this method."
)

self.fit_result = self._fit_mod.fit(
(
self.counts[self._fit_region] - self.baseline
if self._fit_region
else self.counts[self._fit_region]
),
*args,
x=self.binding_energy[self._fit_region],
weights=1 / self.counts_err[self._fit_region],
**kwargs,
)

return self

def calc_baseline(self, bg_type: str = "shirley") -> "XPSRegion":
"""Calculate the baseline for this xps spectrum in the given region.
Args:
bg_type (str, optional): The background type. Defaults to "shirley".
Raises:
ValueError: If the bg_type is unsupported.
Returns:
XPSRegion: This class
"""
baselines = {"shirley": shirley_baseline}
if bg_type not in baselines:
raise ValueError(f"Unsupported baseline type {bg_type}.")

self.baseline = baselines[bg_type](
self.binding_energy[self._fit_region], self.counts[self._fit_region]
)

return self

def peak_property(self, prop: str) -> pd.DataFrame:
"""Generates a dataframe with values for a property `prop` of a fitting model.
Args:
prop (str):
The name of the property to deduce for the peaks,
e.g. `center` for the peak center for gaussian or lorentzian shapes.
Raises:
ValueError: Thrown if no prior fit is performed.
Returns:
pd.DataFrame: A pandas DataFrame containing the peak property for each peak.
"""
if not self.fit_result:
raise ValueError("You need to perform a fit first.")

props = pd.DataFrame()
for prefix in map(lambda x: x.prefix, self.fit_result.components):
if f"{prefix}{prop}" not in self.fit_result.params:
continue

props = pd.concat(
[
props,
pd.DataFrame(
{prop: self.fit_result.params.get(f"{prefix}{prop}").value},
index=[prefix.rstrip("_")],
),
]
)

return props

def peak_areas(self, region_only=False) -> pd.DataFrame:
"""Calculates the peak areas of the given fit models peaks.
Args:
region_only (bool, optional):
Set true if only the area inside the set region should be consider.
Defaults to False.
Raises:
ValueError: Thrown if no prior fit is performed.
Returns:
pandas.DataFrame: A pandas DataFrame containing the peak areas.
"""
if not self.fit_result:
raise ValueError("You need to perform a fit first.")

areas = pd.DataFrame()
if region_only:
peaks = self.fit_result.eval_components(
x=self.binding_energy[self._fit_region]
)
else:
peaks = self.fit_result.eval_components(x=self.binding_energy)
for prefix in peaks:
areas = pd.concat(
[
areas,
pd.DataFrame(
{"Area": sum(peaks[prefix])},
index=[prefix.rstrip("_")],
),
]
)
return areas

def plot_residual(self):
"""Plot the fit residual"""
if not self.fit_result:
raise ValueError("You need to perform a fit first.")

fig = go.Figure(
data=go.Scatter(
name="Residual",
x=self.binding_energy[self._fit_region],
y=self.fit_result.residual,
)
)
fig.update_xaxes(title="Binding energy (eV)")
fig.update_yaxes(title="Residual")
fig.layout.xaxis.autorange = "reversed"

fig.show()

def plot(self):
"""Plot the xps region"""
fig = go.Figure(
data=go.Scatter(
name="Measurement",
x=self.binding_energy,
y=self.counts,
error_y=dict(
type="data", # value of error bar given in data coordinates
array=self.counts_err,
visible=True,
),
)
)
if self._fit_region.start is not None:
fig.add_vline(
self.binding_energy[self._fit_region.start], line_color="lightgrey"
)
if self._fit_region.stop is not None:
fig.add_vline(
self.binding_energy[self._fit_region.stop], line_color="lightgrey"
)
if self.baseline is not None:
fig.add_trace(
go.Scatter(
name="Baseline",
x=self.binding_energy[self._fit_region],
y=self.baseline,
),
)
if self.fit_result is not None:
fig.add_trace(
go.Scatter(
name="Fit",
x=self.binding_energy[self._fit_region],
y=self.fit_result.best_fit + self.baseline,
),
)
if isinstance(self._fit_mod, CompositeModel):
peaks = self.fit_result.eval_components(
x=self.binding_energy[self._fit_region]
)
for prefix in peaks:
fig.add_trace(
go.Scatter(
name=prefix.rstrip("_"),
x=self.binding_energy[self._fit_region],
y=peaks[prefix] + self.baseline,
)
)

fig.update_xaxes(title="Binding energy (eV)")
fig.update_yaxes(title="CPS")
fig.layout.xaxis.autorange = "reversed"

fig.show()


# pylint: disable=invalid-name
def shirley_baseline(
x: np.ndarray, y: np.ndarray, tol: float = 1e-5, maxit: int = 10
) -> np.ndarray:
"""Calculate the shirley background according to the Sherwood method.
Args:
x (np.ndarray): The x-axis on which to calculate the shirley baseline.
y (np.ndarray): The y-axis on which to calculate the shirley baseline.
tol (float, optional):
The convergence tolerance at which to stop the iteration.
Defaults to 1e-5.
maxit (int, optional):
The maximum iteration after which to stop the iteration.
Defaults to 10.
Raises:
ValueError:
Is thrown when the arrays have different dimensions,
are not numpy arrays or are empty.
Is also thrown when the fit does not converge.
Returns:
np.ndarray: The shirley baseline for the x, y dataset.
"""

if not isinstance(x, np.ndarray) or not isinstance(y, np.ndarray):
raise ValueError(
f"Parameters x and y must be of type numpy array, not {type(x)} and {type(y)}"
)

if len(x) != len(y):
raise ValueError("x and y arrays have different dimensions.")

if not x.any():
raise ValueError("x-array is empty.")

if len(x.shape) > 1:
raise ValueError(
f"Data arrays must be one-dimensional. Found dimension {x.shape}."
)

is_reversed = False
if x[0] < x[-1]:
is_reversed = True
x = x[::-1]
y = y[::-1]

background = np.zeros(x.shape)
background_next = background.copy()

iters = 0
while True:
k = (y[0] - y[-1]) / np.trapz(y - background, x=x)

for energy in range(len(x)):
background_next[energy] = k * np.trapz(
y[energy:] - background[energy:], x=x[energy:]
)

diff = norm(background_next - background)
background = background_next.copy()
if diff < tol:
break

iters += 1
if iters == maxit:
raise ValueError(
"Maximum number of iterations exceeded before convergence."
)

if is_reversed:
return (y[-1] + background)[::-1]
return y[-1] + background
74 changes: 74 additions & 0 deletions tests/test_nomad_examples.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#
# Copyright The NOMAD Authors.
#
# This file is part of NOMAD. See https://nomad-lab.eu for further info.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""Test for NOMAD examples in XPS reader plugin."""

import os
import pytest

try:
import nomad
except ImportError:
pytest.skip(
"Skipping NOMAD example tests because nomad is not installed",
allow_module_level=True,
)

from pynxtools.testing.nomad_example import (
get_file_parameter,
parse_nomad_examples,
example_upload_entry_point_valid,
)

from pynxtools_xps.nomad.entrypoints import xps_example


EXAMPLE_PATH = os.path.join(
os.path.dirname(__file__),
"..",
"src",
"pynxtools_xps",
"nomad",
"examples",
)


@pytest.mark.parametrize(
"mainfile",
get_file_parameter(EXAMPLE_PATH),
)
def test_parse_nomad_examples(mainfile):
"""Test if NOMAD examples work."""
archive_dict = parse_nomad_examples(mainfile)


@pytest.mark.parametrize(
("entrypoint", "example_path"),
[
pytest.param(
xps_example,
EXAMPLE_PATH,
id="xps_example",
),
],
)
def test_example_upload_entry_point_valid(entrypoint, example_path):
"""Test if NOMAD ExampleUploadEntryPoint works."""
example_upload_entry_point_valid(
entrypoint=entrypoint,
example_path=example_path,
)
1 change: 1 addition & 0 deletions tests/test_reader.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#
# Copyright The NOMAD Authors.
#
# This file is part of NOMAD. See https://nomad-lab.eu for further info.

0 comments on commit be2f10d

Please sign in to comment.