Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve float representation accuracy by increasing precision from 8 to 9 decimal digits## Description #226

Merged
merged 1 commit into from
Jan 14, 2025

Conversation

ZhichuCen
Copy link
Contributor

Using 8 digits to represent a float may lead to accuracy loss during decimal-to-binary conversion.

Problem

When floats are represented with only 8 decimal digits, the precision can be insufficient, potentially causing inaccuracies during decimal-to-binary conversions.

Solution

Increase the precision to 9 decimal digits, ensuring more accurate float representation and minimizing the risk of conversion errors.

Impact

This change improves numerical accuracy in scenarios requiring high precision, without introducing significant computational overhead.

@christophe0606 christophe0606 added review Under review enhancement New feature or request good first issue Good for newcomers and removed review Under review labels Jan 7, 2025
@christophe0606
Copy link
Contributor

I'll merge it as soon as the issues with the github actions have been corrected (issues not related to this PR but preventing this PR from being checked)

@christophe0606 christophe0606 merged commit fde38c3 into ARM-software:main Jan 14, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants