Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V3 casl 663 twkb srid fix #380

Open
wants to merge 2 commits into
base: v3
Choose a base branch
from
Open

V3 casl 663 twkb srid fix #380

wants to merge 2 commits into from

Conversation

Amaneusz
Copy link
Collaborator

No description provided.

Signed-off-by: Jakub Amanowicz <[email protected]>

class SridTest : PgTestBase() {

data class SridTestConfig(val encodingName: String, val flags: Flags, val collectionId: String)
Copy link
Collaborator Author

@Amaneusz Amaneusz Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might look odd at first glance but there's a reason to have test configs as separate being.
Currently in Naksha, when we want to write features with specific encoding we need to specify this as a collection property (via NakshaCollection.defaultFlags). Then, whenever we write features to collection, this field is read and proper encoding is enforced on geometry, tags etc.

In the tests below, we are validating correct encoding of geometry (more specifically, we check whether naksha_geometry function, when run against persisted feature, returns geometry with SRID set to 4326 which is our default and only supported SRID).

To test all encodings we have, we need separate collection for each encoding - we need features to be properly encoded before persisted so we need proper collection.flags to be set (see 1st paragraph above).

Why not use single collection and modify it with defferent flags for each test?

  • Because collections and their properties are cached, so if we had shared collection for all test cases, the application wouldn't fetch collection from DB, rather it will use what it has in cache (with previous encoding) and we would end up with encoding not matching our expectation
  • To avoid the above, the collection cache would have to be cleared, and since it is not public (rightly so), the only way to do that is dropping the collection - this already makes the whole process more complicated and slowe than simply having N collections for N encodings

elsif (encoding = 2) then
RETURN ST_GeomFromWKB(geo);
elsif (encoding = 4) then
RETURN ST_GeomFromEWKB(geo);
elsif (encoding = 6) then
RETURN ST_GeomFromGeoJSON(geo::text);
RETURN ST_GeomFromGeoJSON(convert_from(geo, 'UTF8'));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had a bug here, casting bytea::text won't work as expected, I was getting SQL Error [XX000]: ERROR: unexpected character (at offset 0). The correct way to get text represntation of JSON previously encoded as byte array is to use convert_from with UTF8 encoding specified (this is also the encoding we use in lib-psql).

Some snippets to prove it step by step:

  1. Hex representation of simple Line String:
// Kotlin:
LineStringCoord(
    PointCoord(longitude = 25.0, latitude = 25.0),
    PointCoord(longitude = 25.0, latitude = 26.0),
)

// bytes (hex):
'\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d'
  1. This is what happened before - confirming that we passed something invalid to ST_GeomFromGeoJSON:
> SELECT ST_GeomFromGeoJSON((bytea '\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d')::text)

SQL Error [XX000]: ERROR: unexpected character (at offset 0)
  1. Checking the cast - we can see that we did not get json, rather raw bytes as text
SELECT (bytea '\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d')::text

\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d
  1. Passing these byte representation is not suitable for ST_GeomFromGeoJSON - we get the same error as above
SELECT ST_GeomFromGeoJSON('\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d')

SQL Error [XX000]: ERROR: unexpected character (at offset 0)
  1. Switching from cast to convert_from, we get proper JSON
SELECT convert_from(bytea '\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d', 'UTF8')

{"type":"LineString","coordinates":[[25.0,25.0],[25.0,26.0]]}
  1. Passing proper JSON to ST_GeomFromGeoJSON we get what wanted - geometry instance
SELECT ST_GeomFromGeoJSON(convert_from(bytea '\x7b2274797065223a224c696e65537472696e67222c22636f6f7264696e61746573223a5b5b32352e302c32352e305d2c5b32352e302c32362e305d5d7d', 'UTF8'))

LINESTRING (25 25, 25 26)

Copy link

Code Coverage

Overall Project 31.1% 🍏

There is no coverage information present for the Files changed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant