This repository has been archived by the owner on Oct 23, 2023. It is now read-only.
Resolve fixed-point datatypes for Quant nodes #53
Comments
2 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
When querying the output datatype of
Quantnodes, the current implementation only supports identifying quantization cases the corespond to integers. Everything else is set toFLOAT32.In cases where the scale is global and is a negative power of two, we can resolve the datatype as
FIXED<>instead.See also relevant discussion: #52 (comment)_
The text was updated successfully, but these errors were encountered: