Merge pull request #641 from hakonanes/fix-edax-binary-reader

Fix EDAX binary reader and prepare for 0.8.6 patch release

CHANGELOG.rst

@@ -13,6 +13,24 @@ its best to adhere to `Semantic Versioning <https://semver.org/spec/v2.0.0.html>
 List entries are sorted in descending chronological order. Contributors to each release
 were listed in alphabetical order by first name until version 0.7.0.
 
+0.8.6 (2023-05-29)
+==================
+
+Changed
+-------
+- Use memory mapping (``numpy.memmap()``) instead of reading into memory
+  (``numpy.fromfile()``) for non-lazy reading of EBSD patterns from EDAX binary .up1/2
+  files. (`#641 <https://github.com/pyxem/kikuchipy/pull/641>`_)
+
+Fixed
+-----
+- EBSD patterns from some EDAX binary .up1/2 files were incorrectly read due to an
+  incorrect file offset, making the patterns appear shifted horizontally.
+  (`#641 <https://github.com/pyxem/kikuchipy/pull/641>`_)
+- Reading of EBSD patterns from H5OINA files with the "Camera Binning Mode" dataset not
+  containing the detector binning.
+  (`#641 <https://github.com/pyxem/kikuchipy/pull/641>`_)
+
 0.8.5 (2023-05-21)
 ==================
 

doc/reference/index.rst

@@ -27,7 +27,7 @@ this:
     >>> import kikuchipy as kp
     >>> s = kp.data.nickel_ebsd_small()
     >>> s
-    <EBSD, title: patterns My awes0m4 ..., dimensions: (3, 3|60, 60)>
+    <EBSD, title: patterns Scan 1, dimensions: (3, 3|60, 60)>
 
 .. currentmodule:: kikuchipy
 

doc/tutorials/feature_maps.ipynb

@@ -398,13 +398,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "s.xmap.plot(\n",
+    "fig = s.xmap.plot(\n",
     "    adp3.ravel(),\n",
     "    cmap=\"gray\",\n",
     "    colorbar=True,\n",
     "    colorbar_label=\"Average dot product\",\n",
     "    remove_padding=True,\n",
-    ")"
+    "    return_figure=True\n",
+    ")\n",
+    "fig.axes[0].scatter(x, y, c=\"r\");"
    ]
   }
  ],
@@ -424,7 +426,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.10.11"
   }
  },
  "nbformat": 4,

doc/tutorials/load_save_data.ipynb

@@ -493,21 +493,21 @@
     "\n",
     "Currently, kikuchipy has readers and writers for the following formats:\n",
     "\n",
-    "| Format                                                                  | Read | Write |\n",
-    "| ------------------------------------------------------------------------| ---- | ----- |\n",
-    "| [EMsoft simulated EBSD HDF5](#EMsoft-simulated-EBSD-HDF5)               | Yes  | No    |\n",
-    "| [EMsoft EBSD master pattern HDF5](#EMsoft-EBSD-master-pattern-HDF5)     | Yes  | No    |\n",
-    "| [EMsoft ECP master pattern HDF5](#EMsoft-ECP-master-pattern-HDF5)       | Yes  | No    |\n",
-    "| [EMsoft TKD master pattern HDF5](#EMsoft-TKD-master-pattern-HDF5)       | Yes  | No    |\n",
-    "| [kikuchipy h5ebsd](#kikuchipy-h5ebsd)                                   | Yes  | Yes   |\n",
-    "| [Bruker h5ebsd](#Bruker-h5ebsd)                                         | Yes  | No    |\n",
-    "| [EDAX h5ebsd](#EDAX-h5ebsd)                                             | Yes  | No    |\n",
-    "| [EDAX binary](#EDAX-binary)                                             | Yes  | No    |\n",
-    "| [NORDIF binary](#NORDIF-binary)                                         | Yes  | Yes   |\n",
-    "| [NORDIF calibration patterns](#NORDIF-calibration-patterns)             | Yes  | No    |\n",
-    "| [Oxford Instruments binary](#Oxford-Instruments-binary)                 | Yes  | No    |\n",
-    "| [Oxford Instruments h5ebsd (H5OINA)](#Oxford-Instruments-h5ebsd-H5OINA) | Yes  | No    |\n",
-    "| [Directory of EBSD patterns](#Directory-of-EBSD-patterns)               | Yes  | No    |\n",
+    "| Format                                                                    | Read | Write |\n",
+    "| ------------------------------------------------------------------------- | ---- | ----- |\n",
+    "| [EMsoft simulated EBSD HDF5](#EMsoft-simulated-EBSD-HDF5)                 | Yes  | No    |\n",
+    "| [EMsoft EBSD master pattern HDF5](#EMsoft-EBSD-master-pattern-HDF5)       | Yes  | No    |\n",
+    "| [EMsoft ECP master pattern HDF5](#EMsoft-ECP-master-pattern-HDF5)         | Yes  | No    |\n",
+    "| [EMsoft TKD master pattern HDF5](#EMsoft-TKD-master-pattern-HDF5)         | Yes  | No    |\n",
+    "| [kikuchipy h5ebsd](#kikuchipy-h5ebsd)                                     | Yes  | Yes   |\n",
+    "| [Bruker h5ebsd](#Bruker-h5ebsd)                                           | Yes  | No    |\n",
+    "| [EDAX h5ebsd](#EDAX-h5ebsd)                                               | Yes  | No    |\n",
+    "| [EDAX binary](#EDAX-binary)                                               | Yes  | No    |\n",
+    "| [NORDIF binary](#NORDIF-binary)                                           | Yes  | Yes   |\n",
+    "| [NORDIF calibration patterns](#NORDIF-calibration-patterns)               | Yes  | No    |\n",
+    "| [Oxford Instruments binary](#Oxford-Instruments-binary)                   | Yes  | No    |\n",
+    "| [Oxford Instruments h5ebsd (H5OINA)](#Oxford-Instruments-h5ebsd-(H5OINA)) | Yes  | No    |\n",
+    "| [Directory of EBSD patterns](#Directory-of-EBSD-patterns)                 | Yes  | No    |\n",
     "\n",
     "<div class=\"alert alert-info\">\n",
     "\n",
@@ -1251,7 +1251,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.10.11"
   },
   "widgets": {
    "application/vnd.jupyter.widget-state+json": {

doc/user/applications.rst

@@ -14,6 +14,14 @@ Most of these works are also listed when searching for ``"kikuchipy"`` `on Googl
 
 2023
 ====
+- A. V. Bugten, L. Michels, R. B. Brurok, C. Hartung, E. Ott, L. Vines, Y. Li,
+  L. Arnberg and M. Di Sabatino, "The Role of Boron in Low Copper Spheroidal Graphite
+  Irons," *Metallurgical and Materials Transactions A* (2023).
+  https://doi.org/10.1007/s11661-023-07014-y.
+- O. W. Sandvik, A. M. Müller, H. W. Ånes, M. Zahn, J. He, M. Fiebig, T. Lottermoser,
+  Th. Rojac, D. Meier and J. Schultheiß, "Pressure-control of non-ferroelastic
+  ferroelectric domains in ErMnO3," *arXiv* (2023).
+  https://doi.org/10.48550/arXiv.2304.08423.
 - H. W. Ånes, A. T. J. van Helvoort and K. Marthinsen, "Orientation dependent pinning
   of (sub)grains by dispersoids during recovery and recrystallization in an Al-Mn
   alloy," *Acta Materialia* **248** (2023).

doc/user/related_projects.rst

@@ -6,18 +6,23 @@ This is a non-exhaustive list of related, open-source projects for analysis of
 EBSD data that users of kikuchipy might find useful:
 
 - :doc:`HyperSpy <hyperspy:index>`: Python library with tools for multi-dimensional data
-  analysis, which kikuchipy builds upon.
+  analysis. kikuchipy extends this library for EBSD analysis.
 - `EMsoft <https://github.com/EMsoft-org/EMsoft>`__: Series of Fortran programs which,
   among numerous other tasks, can dynamically simulate EBSD, Electron Channeling Pattern
   (ECP) and Transmission Kikuchi Diffraction (TKD) master patterns.
 - :doc:`orix <orix:index>`: Python library for handling crystal orientation mapping
   data.
-  kikuchipy depends on orix for all operations with vectors, rotations and crystal
-  symmetry.
+  kikuchipy depends on this library for all operations with vectors, rotations and
+  crystal symmetry.
 - :doc:`diffsims <diffsims:index>`: Python library for simulating diffraction.
-  kikuchipy depends on diffsims for handling of reciprocal lattice vectors.
+  kikuchipy depends on this library for handling of reciprocal lattice vectors.
 - :doc:`pyxem <pyxem:index>`: Python library for multi-dimensional diffraction
   microscopy.
+- :doc:`PyEBSDIndex <pyebsdindex:index>`: Python library for Hough/Radon based EBSD
+  indexing.
+  kikuchipy depends on this library for Hough indexing.
+- `OpenECCI <https://github.com/gorliks/OpenECCI>`__: GUI-based software for controlled
+  Electron Channelling Contrast Imaging (ECCI) analysis of crystal defects in an SEM.
 - `MTEX <https://mtex-toolbox.github.io/>`__: Matlab toolbox for analyzing and modeling
   crystallographic textures by means of EBSD or pole figure data.
 - `AstroEBSD <https://github.com/benjaminbritton/AstroEBSD>`__: Matlab package with

kikuchipy/data/_data.py

@@ -644,7 +644,7 @@ def ebsd_master_pattern(
 
     See Also
     --------
-    nickel_ebsd_master_pattern_small, si_ebsd_master_pattern
+    nickel_ebsd_master_pattern_small, ebsd_master_pattern
 
     Notes
     -----

kikuchipy/data/edax_binary/create_dummy_edax_binary_file.py

@@ -19,7 +19,6 @@
 in the documentation.
 """
 
-import os
 from pathlib import Path
 
 import numpy as np
@@ -35,11 +34,11 @@
 n_patterns = ny * nx
 step_size = s.axes_manager["x"].scale
 
-dir_data = Path(os.path.dirname(__file__))
+DIR_DATA = Path(__file__).parent
 
 # UP1, version 1
 # --------------
-with open(dir_data / "edax_binary.up1", mode="w") as file1:
+with open(DIR_DATA / "edax_binary.up1", mode="w") as file1:
     # File header: 16 bytes
     # 4 bytes with the file version
     np.array([1], "uint32").tofile(file1)
@@ -57,7 +56,7 @@
 # x x x x #
 #  x x x  #
 # ------- #
-with open(dir_data / "edax_binary.up2", mode="w") as file2:
+with open(DIR_DATA / "edax_binary.up2", mode="w") as file2:
     # File header: 42 bytes
     # 4 bytes with the file version
     np.array([3], "uint32").tofile(file2)

kikuchipy/data/oxford_h5ebsd/create_oxford_h5ebsd_file.py

@@ -21,7 +21,7 @@
 Only the groups and datasets read in the reader are included.
 """
 
-import os
+from pathlib import Path
 
 from h5py import File
 import numpy as np
@@ -42,10 +42,10 @@
 sy, sx = s._signal_shape_rc
 dx = s.axes_manager["x"].scale
 
-dir_data = os.path.abspath(os.path.dirname(__file__))
+dir_data = Path(__file__).parent
 
 
-f = File(os.path.join(dir_data, "patterns.h5oina"), mode="w")
+f = File(dir_data / "patterns.h5oina", mode="w")
 
 # Top group
 f.create_dataset("Format Version", data=b"5.0")

kikuchipy/io/plugins/_h5ebsd.py

@@ -45,8 +45,7 @@ def _hdf5group2dict(
     dictionary
         To fill dataset values into.
     recursive
-        Whether to add subgroups to ``dictionary`` (default is
-        ``False``).
+        Whether to add subgroups to ``dictionary`` (default is False).
     data_dset_names
         List of names of HDF5 data sets with data to not read.
 
@@ -171,10 +170,10 @@ def check_file(self):
         ------
         IOError
             If there are no datasets in the top group named
-            ``"manufacturer"`` and ``"version"``, or if there are no
-            groups in the top group containing the datasets
-            ``"EBSD/Data"`` and ``"EBSD/Header"``, or if there is no
-            reader for the file ``"manufacturer"``.
+            "manufacturer" and "version", or if there are no groups in
+            the top group containing the datasets
+            "EBSD/Data" and "EBSD/Header", or if there is no reader for
+            the file "manufacturer".
         """
         error = None
         if self.manufacturer is None or self.version is None:
@@ -291,7 +290,7 @@ def get_data(
             Output shape of pattern array, (ny, nx, sy, sx) = (
             map rows, map columns, pattern rows, pattern columns).
         lazy
-            Whether to read dataset lazily (default is ``False``).
+            Whether to read dataset lazily (default is False).
         indices
             Mapping from pattern entry in the file to the 2D map, only
             used in the Bruker Nano reader.
@@ -361,10 +360,10 @@ def get_axes_list(data_shape: tuple, data_scale: tuple) -> List[dict]:
         Parameters
         ----------
         data_shape
-            4D shape of pattern array, ``(ny, nx, sy, sx)`` = (
-            map rows, map columns, pattern rows, pattern columns).
+            4D shape of pattern array, (ny, nx, sy, sx) = ( map rows,
+            map columns, pattern rows, pattern columns).
         data_scale
-            Map scale and detector pixel size, ``(dy, dx, px_size)``.
+            Map scale and detector pixel size, (dy, dx, px_size).
 
         Returns
         -------
@@ -454,15 +453,15 @@ def read(
             Name or a list of names of the desired top HDF5 group(s). If
             not given, the first scan group is returned.
         lazy
-            Read dataset lazily (default is ``False``). If ``False``,
-            the file is closed after reading.
+            Read dataset lazily (default is False). If False, the file
+            is closed after reading.
 
         Returns
         -------
         scan_list
-            List of dictionaries with keys ``"axes"``, ``"data"``,
-            ``"metadata"``, ``"original_metadata"``, ``"detector"``,
-            (possibly) ``"static_background"``, and ``"xmap"``.
+            List of dictionaries with keys "axes", "data", "metadata",
+            "original_metadata", "detector", (possibly)
+            "static_background", and "xmap".
         """
         scan_dict_list = []
         for scan in self.get_desired_scan_groups(group_names):
@@ -482,15 +481,14 @@ def scan2dict(self, group: h5py.Group, lazy: bool = False) -> dict:
         group
             HDF5 group with patterns.
         lazy
-            Read dataset lazily (default is ``False``).
+            Read dataset lazily (default is False).
 
         Returns
         -------
         scan_dict
-            Dictionary with keys ``"axes"``, ``"data"``, ``"metadata"``,
-            ``"original_metadata"``, ``"detector"``,
-            ``"static_background"``, and ``"xmap"``. This dictionary can
-             be passed as keyword arguments to create an
-             :class:`~kikuchipy.signals.EBSD` signal.
+            Dictionary with keys "axes", "data", "metadata",
+            "original_metadata", "detector", "static_background", and
+            "xmap". This dictionary can be passed as keyword arguments
+            to create an :class:`~kikuchipy.signals.EBSD` signal.
         """
         return NotImplemented  # pragma: no cover