diff --git a/README.md b/README.md new file mode 100644 index 0000000..41a2802 --- /dev/null +++ b/README.md @@ -0,0 +1,73 @@ +# Pyxem template matching workflows + +This repository contains all the Jupyter notebook files and other files necessary to reproduce the results and figures in the paper *"Free, flexible and fast: orientation mapping using the multi-core and GPU-accelerated template matching capabilities in the python-based open source 4D-STEM analysis toolbox Pyxem"* which was recently submitted to the journal *Ultramicroscopy*; arXiv preprint details are pending. + +What these notebooks show: +* how to use diffsims, orix, and pyxem to perform template matching on a precession electron diffraction (PED) or nanobeam electron diffraction (NBED) dataset. +* how template matching works under the hood, and which kinds of image pre-processing steps are helpful for getting good results. +* how to build custom workflows like a two-step indexation for finding the orientation of embedded precipitates + +## Viewing the workflows + +For static views of the notebooks visit: + +* +* +* +* + +## Downloading the necessary data + +A lot of data is already included in the `data` directory. +However, this repository does not include the large raw 4D-STEM datasets, they can be downloaded from here: +* [Cu-Ag dataset](https://doi.org/10.5281/zenodo.5595292) +* [G-phase dataset](https://doi.org/10.5281/zenodo.5597738) + +Inside the respective notebooks, there are cells that download the correct file in case it is not available. + +## Running the notebooks on your computer + +### Option 1 (easy): getting the Docker image +Everything in this repository, the data and outputs, as well as all the necessary dependencies were put into a [docker container image](https://docs.docker.com/get-started/overview/). +With this option you don't need to download or configure anything else, everything should be self contained. +The steps to run the notebooks are: + +1. [Download and install Docker Engine for your system](https://docs.docker.com/engine/install/). On Linux, ensure the docker daemon is loaded. +2. Pull the docker image with the command `$ docker pull nielscautaerts/pyxem_template_matching_workflows`. +3. Run the docker image with `$ docker run -p 7000:8888 din14970/pyxem_template_matching_workflows`. You can change the 7000 port on local host to any port number you like. +4. Now visit . Enter and submit the token that you may find in the terminal when you ran the `docker run` command (it's a long string of numbers and letters after `token=`). You should now see all the necessary files. + +#### Caveats +* I could not get Mayavi to work properly inside the Docker container in the `Generating figures` notebook. Running some of these cells may crash the kernel. This only pertains to a single subfigure of Figure 1 in the paper. +* Even if you have a CUDA enabled GPU, GPU code will not work inside the docker container. You will have to run template matching code on the CPU. I have already included outputs of most of the notebooks in the docker image so you aren't forced to rerun these processing steps in order to play with the data. + +If you know how to solve these issues, feel free to modify the Dockerfile in this repository and make a pull request. + +### Option 2 (harder): installing all dependencies in a virtual environment +To run the notebooks directly on your computer and get them working properly you may have to do a bit more work. +In order to save you some time I have already included an `environment.yml` file to install the right versions of dependencies. + +1. Ensure you have `conda` installed, either with [Anaconda](https://www.anaconda.com/products/individual) or [miniconda](https://docs.conda.io/en/latest/miniconda.html). Miniconda is much more light weight and recommended. +2. Ensure you have [git](https://git-scm.com/downloads) installed. +3. Clone this repository to a destionation of your choosing with `$ git clone https://github.com/din14970/pyxem_template_matching_workflows.git` +4. Go inside the repository folder with `cd` and create a new virtual environment from the `environment.yml` file using `conda env create -f environment.yml` +5. Go out of this folder and clone the repositories for pyxem and diffsims `$ git clone https://github.com/din14970/diffsims.git` and `$ git clone https://github.com/pyxem/pyxem.git` +6. Go into these respective folders and check out specific commits. Then install the versions in your virtual environment. + ``` + $ conda activate pyxenv + $ cd diffsims + $ git checkout 29b0715 + $ python3 -m pip install -e . + $ cd ../pyxem + $ git checkout ca4f58de + $ python3 -m pip install -e . + ``` +These steps are necessary because the notebooks use specific features which have not yet been packaged into an official version of the software. +7. You should now be able to launch a Jupyter notebook server with `$ jupyter notebook` and open and run the notebooks. Always ensure the virtual environment is active. + +If you want GPU acceleration to work you will need to also install `cupy` with a version >9.0.0. +Additionally you may have to install the appropriate NVIDIA drivers. +If you can import `cupy` in a notebook and create an array with `cupy.array([1, 2, 3])` without errors, the GPU acceleration should work. + +If Mayavi doesn't work on your system you may have to do some digging on required system packages. +Pyxem does not rely on Mayavi, only one image uses this package. diff --git a/environment.yml b/environment.yml index 28bd625..20fe14d 100644 --- a/environment.yml +++ b/environment.yml @@ -27,4 +27,3 @@ dependencies: - pywget=3.2 - vtk=9.0.3 - mayavi=4.7.2 -prefix: /home/din14970/miniconda3/envs/pyxenv