Improve docs.

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## Unreleased
 
+### Features
+- Initial implementation of multi-source tile source ([#764](../../pull/764))
+
 ### Improvements
 - Add more opacity support for image overlays ([#761](../../pull/761))
 - Make annotation schema more uniform ([#763](../../pull/763))

docs/index.rst

@@ -21,6 +21,7 @@
    _build/large_image_source_gdal/modules
    _build/large_image_source_mapnik/modules
    _build/large_image_source_multi/modules
+   multi_source_specification
    _build/large_image_source_nd2/modules
    _build/large_image_source_ometiff/modules
    _build/large_image_source_openjpeg/modules

docs/make_docs.sh

@@ -15,6 +15,7 @@ ln -s ../build/docs-work _build
 
 large_image_converter --help > _build/large_image_converter.txt
 python -c 'from girder_large_image_annotation.models import annotation;import json;print(json.dumps(annotation.AnnotationSchema.annotationSchema, indent=2))' > _build/annotation_schema.json
+python -c 'import large_image_source_multi, json;print(json.dumps(large_image_source_multi.MultiSourceSchema, indent=2))' > _build/multi_source_schema.json
 
 sphinx-apidoc -f -o _build/large_image ../large_image
 sphinx-apidoc -f -o _build/large_image_source_bioformats ../sources/bioformats/large_image_source_bioformats

docs/multi_source_specification.rst

@@ -0,0 +1,6 @@
+.. include:: ../sources/multi/docs/specification.rst
+
+This returns the following:
+
+.. include:: ../build/docs-work/multi_source_schema.json
+   :literal:

sources/multi/docs/specification.rst

@@ -0,0 +1,136 @@
+Multi Source Schema
+===================
+
+A multi-source tile source is used to composite multiple other sources into a
+single conceptual tile source.  It is specified by a yaml or json file that
+conforms to the appropriate schema.
+
+Examples
+--------
+
+All of the examples presented here are in yaml; json works just as well.
+
+Multi Z-position
+~~~~~~~~~~~~~~~~
+
+For example, if you have a set of individual files that you wish to treat as
+multiple z slices in a single file, you can do something like:
+
+::
+
+    ---
+    sources:
+      - path: ./test_orient1.tif
+        z: 0
+      - path: ./test_orient2.tif
+        z: 1
+      - path: ./test_orient3.tif
+        z: 2
+      - path: ./test_orient4.tif
+        z: 3
+      - path: ./test_orient5.tif
+        z: 4
+      - path: ./test_orient6.tif
+        z: 5
+      - path: ./test_orient7.tif
+        z: 6
+      - path: ./test_orient8.tif
+        z: 7
+
+Here, each of the files is explicitly listed with a specific ``z`` value.
+Since these files are ordered, this could equivalently be done in a simpler
+manner using a ``pathPattern``, which is a regular expression that can match
+multiple files.
+
+::
+
+    ---
+    sources:
+      - path: .
+        pathPattern: 'test_orient[1-8]\.tif'
+        zStep: 1
+
+Since the ``z`` value will default to 0, this works.  The files are sorted in
+C-sort order (lexically using the ASCII or UTF code points).  This sorting will
+break down if you have files with variable length numbers (e.g., ``file10.tif``
+will appear before ``file9.tiff``.  You can instead assign values from the
+file name using named expressions:
+
+::
+
+    ---
+    sources:
+      - path: .
+        pathPattern: 'test_orient(?P<z1>[1-8])\.tif'
+
+Note that the name in the expression (``z1`` in this example) is the name of
+the value in the schema.  If a ``1`` is added, then it is assumed to be 1-based
+indexed.  Without the ``1``, it is assumed to be zero-indexed.
+
+Composite To A Single Frame
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Multiple sources can be made to appear as a single frame.  For instance:
+
+::
+
+    ---
+    width: 360
+    height: 360
+    sources:
+      - path: ./test_orient1.tif
+        position:
+          x: 0
+          y: 0
+      - path: ./test_orient2.tif
+        position:
+          x: 180
+          y: 0
+      - path: ./test_orient3.tif
+        position:
+          x: 0
+          y: 180
+      - path: ./test_orient4.tif
+        position:
+          x: 180
+          y: 180
+
+Here, the total width and height of the final image is specified, along with
+the upper-left position of each image in the frame.
+
+Composite With Scaling
+~~~~~~~~~~~~~~~~~~~~~~
+
+Transforms can be applied to scale the individual sources:
+
+::
+
+    ---
+    width: 720
+    height: 720
+    sources:
+      - path: ./test_orient1.tif
+        position:
+          scale: 2
+      - path: ./test_orient2.tif
+        position:
+          scale: 2
+          x: 360
+      - path: ./test_orient3.tif
+        position:
+          scale: 2
+          y: 360
+      - path: ./test_orient4.tif
+        position:
+          scale: 360
+          x: 180
+          y: 180
+
+Note that the zero values from the previous example have been omitted as they
+are unnecessary.
+
+Full Schema
+-----------
+
+The full schema (jsonschema Draft6 standard) can be obtained by referencing the
+Python at ``large_image_source_multi.MultiSourceSchema``.

sources/multi/large_image_source_multi/__init__.py

@@ -195,6 +195,13 @@
                         'items': {'type': 'number'},
                         'minItems': 1,
                     },
+                    'channel': {
+                        'description':
+                            'A channel name to correspond with the main '
+                            'image.  Ignored if c, cValues, or channels is '
+                            'specified.',
+                        'type': 'string',
+                    },
                     'channels': {
                         'description':
                             'A list of channel names used to correspond '
@@ -514,6 +521,8 @@ def _addSourceToFrames(self, tsMeta, source, sourceIdx, frameDict):
         channels = tsMeta.get('channels', [])
         if source.get('channels'):
             channels[:len(source['channels'])] = source['channels']
+        elif source.get('channel'):
+            channels[:1] = [source['channel']]
         if len(channels) > len(self._channels):
             self._channels += channels[len(self._channels):]
         if not any(key in source for key in {
@@ -537,6 +546,9 @@ def _addSourceToFrames(self, tsMeta, source, sourceIdx, frameDict):
                     self._axisKey(source, tIdx, 't'),
                     self._axisKey(source, xyIdx, 'xy'))
             channel = channels[cIdx] if cIdx < len(channels) else None
+            if channel and channel not in self._channels and (
+                    'channel' in source or 'channels' in source):
+                self._channels.append(channel)
             if (channel and channel in self._channels and
                     'c' not in source and 'cValues' not in source):
                 aKey = (self._channels.index(channel), aKey[1], aKey[2], aKey[3])
@@ -673,6 +685,14 @@ def _collectFrames(self, checkAll=False):
         self.levels = int(max(1, math.ceil(math.log(
             max(self.sizeX / self.tileWidth, self.sizeY / self.tileHeight)) / math.log(2)) + 1))
 
+    def getNativeMagnification(self):
+        """
+        Get the magnification at a particular level.
+
+        :return: magnification, width of a pixel in mm, height of a pixel in mm.
+        """
+        return self._nativeMagnification.copy()
+
     def getAssociatedImage(self, imageKey, *args, **kwargs):
         """
         Return an associated image.
@@ -835,10 +855,10 @@ def _addSourceToTile(self, tile, sourceEntry, corners, scale):
         # Otherwise, get an area twice as big as needed and use
         #  scipy.ndimage.affine_transform to transform it
         else:
-            # ##DWM::
+            # TODO
             raise TileSourceError('Not implemented')
         # Crop
-        # ##DWM::
+        # TODO
         tile = self._mergeTiles(tile, sourceTile, x, y)
         return tile
 
@@ -885,7 +905,7 @@ def getTile(self, x, y, z, pilImageAllowed=False, numpyAllowed=False, **kwargs):
         for sourceEntry in sourceList:
             tile = self._addSourceToTile(tile, sourceEntry, corners, scale)
         if tile is None:
-            # ##DWM:: number of channels?
+            # TODO number of channels?
             colors = self._info.get('backgroundColor', [0])
             if colors:
                 tile = numpy.full((self.tileWidth, self.tileHeight, len(colors)), colors)

test/test_files/multi_channels.yml

@@ -0,0 +1,27 @@
+---
+name: Multi orientationa
+sources:
+  - path: ./test_orient1.tif
+    channel: CY3
+    z: 0
+  - path: ./test_orient2.tif
+    channel: A594
+    z: 0
+  - path: ./test_orient3.tif
+    channel: CY5
+    z: 0
+  - path: ./test_orient4.tif
+    channel: DAPI
+    z: 0
+  - path: ./test_orient5.tif
+    channel: CY3
+    z: 1
+  - path: ./test_orient6.tif
+    channel: A594
+    z: 1
+  - path: ./test_orient7.tif
+    channel: CY5
+    z: 1
+  - path: ./test_orient8.tif
+    channel: DAPI
+    z: 1

test/test_source_multi.py

@@ -23,6 +23,7 @@ def multiSourceImagePath():
     'multi1.yml',
     'multi2.yml',
     'multi3.yml',
+    'multi_channels.yml',
 ])
 def testTilesFromMulti(filename):
     testDir = os.path.dirname(os.path.realpath(__file__))