Augmentation strategy for generalization across magnification

[ziw-liu]

Different magnification of the microscope alters the sampling of all 3 spatial dimensions. And the changes in Z is different from that in XY. If we want to train a model on a single dataset that generalizes across magnifications, we need to employ augmentation strategies that simulate the changes in spatial sampling.

@mattersoflight pointed out that scaling down is not a good approximate of reducing magnification. Blurring (e.g. Gaussian filtering) before rescaling can simulate the integration of information along the light path and reduce artifacts.

Another question is that how do we determine the training time Z sampling for better utilization of defocus information. This can potentially be estimated from magnification, Z step size, and the NA of illumination and detection.

[mattersoflight]

@ziw-liu

Another question is that how do we determine the training time Z sampling for better utilization of defocus information. This can potentially be estimated from magnification, Z step size, and the NA of illumination and detection.

A meta-comment: we should capture the most information in the training data, and we can always augment the data with CV filters or optical filters to mimic the loss of information that occurs in test data.

Specifically, I think the following are good settings for the types of virtual staining models we need:

• 63x1.3 NA objective and Nyquist sampling in Brightfield and all fluorescence channels. The high spatial resolution and corresponding sampling will help us see small structures such as nucleolar compartments and lipid droplets.
• Ensure that the data is acquired to capture all of the sample + substantial blur (~10 slices above and below the edge of the sample).