Is it possible to quantize to FP8 W8A16 without calibration data

[us58]

I want to quantize a model to FP8 W8A16 (since I am on Ampere). In the quantization_w8a8_fp8 example, it says no calibration is needed for FP8 W8A8. Is this also possible for FP8 W8A16? I did not find any information on this.

Also, if possible, can you give me an example on how to do this (like the FP8 W8A8 example)? Thanks in advance.

[okwinds]

A single sample should work

[dsikka]

Hi @us58:

If you specify an FP8 recipe targeting weights only for quantization, you should not need any calibration data.

Example:

quant_stage:
    quant_modifiers:
        QuantizationModifier:
            ignore: ["lm_head"]
            config_groups:
                group_0:
                    weights:
                        num_bits: 8
                        type: float
                        strategy: tensor
                        dynamic: false
                        symmetric: true