This is a NOMAD parser for NEXUS. It will read input in NEXUS format and output files and provide all information in NOMAD's unified Metainfo based Archive format.
You should create a virtual environment. This is optional, but highly recommended as the required nomad-lab pypi package requires many dependencies with specific versions that might conflict with other libraries that you have installed. This was tested with Python 3.7.
If you don't have Python 3.7 installed on your computer, follow these commands:
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.7 python3-dev libpython3.7-dev python-numpy
You can now install your virtual environment with python3.7 interpreter
mkdir <your-brand-new-folder>
cd <your-brand-new-folder>
pip install virtualenv
virtualenv --python=python3.7 .pyenv
source .pyenv/bin/activate
Simply install our pypi package with pip:
pip install --upgrade pip
pip install nomad-lab==1.0.0 --extra-index-url https://gitlab.mpcdf.mpg.de/api/v4/projects/2187/packages/pypi/simple
Clone this project (or fork and then clone the fork). Go into the cloned directory and directly run the parser from there:
git clone https://github.com/nomad-coe/nomad-parser-nexus.git --branch master --recursive parser-nexus
cd parser-nexus
git submodule sync --recursive
git submodule update --init --recursive
Install the remaining dev dependencies and install the package with:
pip install -r requirements.txt
pip install -e .[all]
There are also a basic test framework written in pytest. Run the test with:
pytest -sv tests
First go to the nexus metainfo directory:
cd nexusparser/metainfo
This directory have several modules/files as following:
| file | function |
|---|---|
| generate.py | This module contains functionality to generate metainfo source-code from metainfo definitions. |
| templates/package.j2 | A template to create/convert NEXUS classes with NOMAD schema. |
| generate_nexus.py | This module uses the generate.py module and the package.j2 template to create classes in NOMAD format, with inheritance and extension of bases classes. It will create a meta_nexus.py file after running. |
| meta_nexus.py | This file is being created every time the generate_nexus.py script runs. It contains all base classes, application definitions, and contributed classes converted from NEXUS schema to NOMAD schema. |
| test_nexus.py | After the meta_nexus.py file has been created, test_nexus.py can be run to test if it is correctly structured and working as desired. |
| nexus.py | If the meta_nexus.py passes the above test, then run the following command to create/replace the current nexus.py file: cp meta_nexus.py nexus.py. It is important that only, after testing meta_nexus.py and being sure that it is working appropriately, create nexus.py. Because nexus_py will be used by the actual parser later. |
This module contains several tools that are necessary to read user-specific experimental data and parse them into NOMAD.
1. Read nexus
First go to the nexus tools directory:
cd nexusparser/tools
The module nexus.py reads HDF5 data file (i.e it is a user-specific data file, which contains numerical and metadata about the experiment. And it is formatted in NEXUS style.)
The environmental variable called "NEXUS_DEF_PATH" should be set to the directory, which contains application definitions as XML files. If this environmental variable is not defined, then code will first clone a directory from GitHub, which contains application definitions.
To set environmental variable, you can do:
export 'NEXUS_DEF_PATH'=<folder_path_that_contains_app_defs>
Testing nexus
Following example dataset can be used to test nexus.py module tests/data/nexus_test_data/201805_WSe2_arpes.nxs. This is angular resolved photoelectron spectroscopy (ARPES) dataset and it is formatted according to the NXarpes application definition of NEXUS. Run the following command to test the nexus.py using example ARPES dataset:
python test_parser.py
You should get "Testing of nexus.py is SUCCESSFUL." message, if everything goes as expected!
Our documentation provides several resources that might be interesting:
- How to write a parser. Provides a more detailed tutorial on how to write a parser.
- Introduction to the NOMAD Metainfo. This explains how NOMAD data schema and can be extended and used within your parser.
To get you parser included in NOMAD or ask further questions, you can:
- Use our forums at matsci.org
- Open an issue on the nexus-parser GitHub project
- Write to [email protected]
Note! The rest of this README.md is the usual text that applies to all NOMAD parsers.
This is a NOMAD parser for EXAMPLE. It will read EXAMPLE input and output files and provide all information in NOMAD's unified Metainfo based Archive format.
NOMAD accepts .zip and .tar.gz archives as uploads. Each upload can contain arbitrary
files and directories. NOMAD will automatically try to choose the right parser for you files.
For each parser (i.e. for each supported code) there is one type of file that the respective
parser can recognize. We call these files mainfiles as they typically are the main
output file a code. For each mainfile that NOMAD discovers it will create an entry
in the database that users can search, view, and download. NOMAD will associate all files
in the same directory as files that also belong to that entry. Parsers
might also read information from these auxillary files. This way you can add more files
to an entry, even if the respective parser/code might not directly support it.
For EXAMPLE please provide at least the files from this table if applicable to your calculations (remember that you can provide more files if you want):
| Input Filename | Description |
|---|---|
nexus.out |
Mainfile in EXAMPLE specific plain-text |
To create an upload with all calculations in a directory structure:
zip -r <upload-file>.zip <directory>/*
Go to the NOMAD upload page to upload files or find instructions about how to upload files from the command line.
You can use NOMAD's parsers and normalizers locally on your computer. You need to install NOMAD's pypi package:
pip install nomad-lab
To parse code input/output from the command line, you can use NOMAD's command line interface (CLI) and print the processing results output to stdout:
nomad parse --show-archive <path-to-file>
To parse a file in Python, you can program something like this:
import sys
from nomad.cli.parse import parse, normalize_all
# match and run the parser
archive = parse(sys.argv[1])
# run all normalizers
normalize_all(archive)
# get the 'main section' section_run as a metainfo object
section_run = archive.section_run[0]
# get the same data as JSON serializable Python dict
python_dict = section_run.m_to_dict()Create a virtual environment to install the parser in development mode:
pip install virtualenv
virtualenv -p `which python3` .pyenv
source .pyenv/bin/activate
Install NOMAD's pypi package:
pip install nomad-lab
Clone the parser project and install it in development mode:
git clone https://github.com/nomad-coe/nomad-parser-nexus.git parser-nexus
pip install -e parser-nexus
Running the parser now, will use the parser's Python code from the clone project.