Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lost of precision due to the use of float as type for degrees #72

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jmfermun
Copy link

Function minmea_tocoord now converts positions from type minmea_float to type double.

Conversion from latitude and longitude to meters:

  • delta_latitude_in_meters = delta_latitude_in_degrees * 40008000 / 360
  • delta_longitude_in_meters = delta_longitude_in_degrees * 40075160 * cos(latitude_in_degrees) / 360
  • The below examples are centered in the conversions of longitude to meters in its worst case (latitude 0º).

The use of the type minmea_float for storing the positions seems to be convenient:

  • Mapping of digits in 32 bit integer range:
    2**31 21474 83648
    Latitude XDDMM.MMMMM
    Longitude DDDMM.MMMMM
  • It allows to store 5 decimals, so we will obtain an error of 0.00001 / 60 * 40075160 * cos(0) / 360 = 0.01855 m.
  • In the readme it is specified an error of 2 cm. It matches with the above value.

Use of float for storing the positions is not convenient:

  • The precision of float is between 6 and 7 digits.
  • The precission of double is between 15 and 16 digits.
  • The worst case is with a latitude 0.0 º (equator) and a longitude with 3 digits used by the degrees part.
  • If we use the type float, 4 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0001 * 40075160 * cos(0) / 360 = 11.13 m.
  • If we use the type double, 13 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0000000000001 * 40075160 * cos(0) / 360 = 0.00000001113 m.
  • We need at least 7 digits for the decimal part of the longitude to obtain an error of 0.0000001 * 40075160 * cos(0) / 360 = 0.01113 m.

References:

… to type double.

Conversion from latitude and longitude to meters:
- delta_latitude_in_meters = delta_latitude_in_degrees * 40008000 / 360
- delta_longitude_in_meters = delta_longitude_in_degrees * 40075160 * cos(latitude_in_degrees) / 360
- The below examples are centered in the conversions of longitude to meters in its worst case (latitude 0º).

The use of the type minmea_float for storing the positions seems to be convenient:
- Mapping of digits in 32 bit integer range:
    2**31       21474 83648
    Latitude    XDDMM.MMMMM
    Longitude   DDDMM.MMMMM
- It allows to store 5 decimals, so we will obtain an error of 0.00001 / 60 * 40075160 * cos(0) / 360 = 0.01855 m.
- In the readme it is specified an error of 2 cm. It matches with the above value.

Use of float for storing the positions is not convenient:
- The precision of float is between 6 and 7 digits.
- The precission of double is between 15 and 16 digits.
- The worst case is with a latitude 0.0 º (equator) and a longitude with 3 digits used by the degrees part.
- If we use the type float, 4 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0001 * 40075160 * cos(0) / 360 = 11.13 m.
- If we use the type double, 13 digits are available for the decimal part of the longitude, so we will obtain an error of 0.0000000000001 * 40075160 * cos(0) / 360 = 0.00000001113 m.
- We need at least 7 digits for the decimal part of the longitude to obtain an error of 0.0000001 * 40075160 * cos(0) / 360 = 0.01113 m.

References:
- https://blog.demofox.org/2017/11/21/floating-point-precision/.
- https://stackoverflow.com/questions/3024404/transform-longitude-latitude-into-meters
@jmfermun jmfermun changed the title Lost of precission due to the use of float as type for degrees Lost of precision due to the use of float as type for degrees Jan 29, 2023
@matthieu-c-tagheuer
Copy link

I also would like this modification.

@KJ7LNW
Copy link

KJ7LNW commented Feb 20, 2024

+1, looks like a simple change. If there are MCUs out there that really can't or shouldn't be using doubles, then maybe provide a #ifdef MINMEA_USE_FLOAT for users that really need 32-bit floats?

@johlim
Copy link
Contributor

johlim commented Feb 20, 2024

I also would like this modification. there is experience of experiencing problems with float-type GPS coordinate errors.

@lucid-at-dream
Copy link

The tests are not building. You should change the use of fabsf in tests.c to fabs to follow the type change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants