Augmentation strategy for mantis data

[ziw-liu]

Label-free datasets acquired on the mantis have similar in-plane pixel size as hummingbird@63x, where a lot of the training data was acquired. To apply 2.5D virtual staining across microscopes, however, axial spatial augmentation is needed to match training data distribution (250 nm) to the wide range of existing (570 nm, czbiohub-sf/shrimPy#69) and future (205 nm) Z-sampling on mantis.

@talonchandler suggests that trilinear is a good interpolation to start with (0.4x to 1.2x scaling in Z). We will also investigate the unit of voxel intensity values in the reconstructed phase images, since mantis has a different illumination wavelength (450 nm) than hummingbird (532 nm).

Pinging @Soorya19Pradeep and @edyoshikun in case these numbers are not accurate.

[ziw-liu]

This is related to #16 but more specific to mantis data.

[ziw-liu]

For this scaling range and a 5-slice output, the input stack needs to be 11-slice.

import torch
import matplotlib.pyplot as plt
from monai.transforms import RandAffine

# C, Z, Y, X
x = torch.ones((1, 11, 12, 3))

t = RandAffine(
    prob=1,
    rotate_range=(3.14, 0, 0),
    shear_range=(0, (0.05), (0.05)),
    scale_range=((-0.2, 1.4), 0.3, 0.3),
    padding_mode="zeros",
    mode="trilinear",
)

f, ax = plt.subplots(1, 10, figsize=(20, 4), sharey=True)

for i in range(10):
    y = t(x)
    ax[i].imshow(y[0].numpy())

plt.tight_layout()

Figure: Z-Y projection of output of random affine transform, white pixels are resampled solely from in-grid data (not from padding).

Z-only:

This augmentation will use more I/O bandwidth (2x).

[mattersoflight]

@ziw-liu thanks for starting this discussion on axial spatial augmentation.

We will also investigate the unit of voxel intensity values in the reconstructed phase images, since mantis has a different illumination wavelength (450 nm) than hummingbird (532 nm).

Examining the XY and XZ cross sections of the phase and fluorescence volumes from both microscopes will be very informative. I look forward to that data.

For this scaling range and a 5-slice output, the input stack needs to be 11-slice.

I don't see images in your last comment, so I ran the snippet above, and see this:

It does make sense that we read more z-slices to make sure that we don't have boundary artifacts.

Some other comments:

• The training data acquired on hummingbird and test data from mantis has sufficient spatial resolution and sampling in z to capture the shape and texture of nuclei and membrane. In other words, 205nm, 250nm, and 570nm are all sufficient sampling intervals to resolve nuclei and cell contours. 570nm sampling is likely insufficient for the membrane.
• We will collect a lot of data on membrane with mantis.

[mattersoflight]

The solution to this issue is in progress.

#27 and #29 were merged into the main to make it easy to install viscy from the main during the DL@MBL course.

@ziw-liu can you elaborate on #28? Should it be merged while testing the 2D UNet?

[ziw-liu]

@ziw-liu can you elaborate on #28? Should it be merged while testing the 2D UNet?

Yes, that PR has 2 bug fixes.