Tile compositing not working

[arjunrajlab]

@manthey not sure if the file reader was set to do this, but it seems that currently, the software doesn't do compositing. For instance, here is an ND2 file with a bunch of tiles, but the software doesn't composite it together into a big stitched image. Is that a feature that is easy to enable?

https://www.dropbox.com/sh/aagt5bf24t0o7zm/AACPG12_mndS0pqMXgqG4aada?dl=0

[finetjul]

On 2022/07/29: @manthey does not reproduce

@zjniu can you reproduce and send a screenshot

[manthey]

This will use the internal metadata from the nd2 files that contains the location for each XY tile. We then can generate the multi-source yaml file with the appropriate coordinates to composite it. I'll make a sample to demonstrate.

[manthey]

This need a small tweak to to the multi source to properly composite tiles if the first one wasn't at the origin (girder/large_image#993).

This can work for any file where we have coordinates for the XY frames. This is probably only for nd2 files unless we examine and determine the location of such data (OME tiffs also have it, at least).

The process that needs to be done (in python pesudo code):

ts = large_image.open(<nd2 path>)
# We need the scale and frame list from the metadata
metadata = ts.getMetadata()
# We need the xy positions from the internal metadata
internal = ts.getInternalMetadata()
# Extract the x, y coordinates in pixels.  If we throw an exception here, then we can't do composite
# There will be one set of coordinates for each unique combination of z, t, xy (not c)
coordinates = [[
    f['position']['stagePositionUm'][0] / metadata['mm_x'] / 1000, 
    f['position']['stagePositionUm'][1] / metadata['mm_y'] / 1000
] for f in internal['nd2_frame_metadata']]
# Find the min values
minc = [min(xy[0] for cc in coordinates), min(xy[1] for xy in coordinates)]
# Convert to integer pixels where the left and top are 0 and 0
coordinates = [[int(xy[0] - minc[0]), int(xy[1] - minc[1])] for cc in coordinates]
# General a yaml file (this assumes number of frames and a bunch of other stuff -- it was taken from a 2 channel example)
multi_source = {
    'channels': ['GFP', 'DAPI'],  # or whatever the channels are
    'sources': [{
        'path': <nd2 path>, 
        'c': 0,   # We keep the existing channel stack
        # We probable need to specify t and z appropriately here
        'frames': [xy * 2, xy * 2 + 1],  # list of frames with this exact z, t, xy (i.e., all the channels for this frame)
        'xySet': 0,  # we are discarding the xy information
        'position': {'x': xy[0], 'y': xy[1]}
    } for xy in coordinates]
}
# Save a yaml file
open('sample.yaml', 'w').write(yaml.safe_dump(multi_source))

[manthey]

I think the UI will be added in src/views/dataset/MultiSourceConfiguration.vue. 20210723_155544_798__Plate20210723_STR_gfp_Well03_ChannelGFP,DAPI_Seq0000.nd2 is a good example.

[arjunrajlab]

@bruyeret @manthey Ping

[arjunrajlab]

Need to apply a matrix transform FYI.