Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The root of most evil #1063

Merged
merged 3 commits into from
Feb 13, 2023
Merged

The root of most evil #1063

merged 3 commits into from
Feb 13, 2023

Conversation

dabreegster
Copy link
Collaborator

Long long ago, #686 started taking advantage of the fact that all the numeric quantities in geom are rounded to a few decimal places, by serializing smaller integers instead of full f64s. But that PR had a bug, one which @michaelkirk pointed out. This PR improves the unit tests to catch this problem, then fixes it.

The problem is this:

  1. We construct a bunch of geometry in-memory. Every time we do Pt2D::new, we trim the floats to 4 decimal places.
  2. We serialize the f64's as ints
  3. We deserialize the ints, multiplying back to f64's
  4. The in-memory geometry and the round-tripped saved-then-loaded geometry differ

This has extremely scary implications -- #689 hit one in the traffic simulation that I never bothered to dig into. But there are so many more, mentioned in a-b-street/geom#1. When we construct a PolyLine most of the time, we guarantee that no adjacent points are equal (up to the epsilon implied by this rounding behavior). That means later we can get individual Line segments that have nonzero distance. The problem is, this guarantee held when we originally constructed the geometry, but sometimes after deserializing, we lost that guarantee, because the points changed very slightly. This has caused so much havoc.

An example of the problem in the unit test, before the fix:

tests::f64_trimming' panicked at 'Round-tripping mismatch!
input=            Pt2D { x: -138849.6769, y: 196157.1725 }
json_roundtrip=   Pt2D { x: -138849.6769, y: 196157.1724 }
bincode_roundtrip=Pt2D { x: -138849.6769, y: 196157.1724 }', geom/src/lib.rs:198:17

196157.1725 in-memory gets turned into 196157.1724.

Next steps

If this fix is truly correct, it has very far-reaching implications. For example,

// TODO Hack around geometry problems generating these, usually at intersections
actually just catches panics that I'm pretty sure were root-caused by this. I want to start digging out these hacks.

More short-term, I want to simplify a few things within geom. We have a custom PartialEq implementation for Pt2D here. I don't think this should be necessary, I'm going to just derive it. I also want to consider removing HashablePt2D and just making Pt2D internally use ordered_float, but that's less urgent. (And there's some current bug here where we violate the hash(x) == hash(y) implies x == y invariant; I need to dig into this...)

CC @BudgieInWA

@dabreegster
Copy link
Collaborator Author

dabreegster commented Feb 13, 2023

I've tested a bit more, and am going to merge this. The next steps of cleaning up HashablePt2D and EPSILON_DIST are actually tricky. The problem is that Pt2D equality currently uses EPSILON_DIST (0.1 meters), but HashablePt2D hashes on more detailed f64's. I'll dig out of that hole separately. I think we usually just want to do raw comparison the trimmed f64's and stop doing approx_eq checks in many places.

Edit: the new end-to-end test will keep failing until I regenerate all maps. Working on that separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant