[Feat] Support WKB geometry column in CSV #2312
Conversation
Signed-off-by: Xun Li <[email protected]>
| if (!parsedGeo) { | ||
| try { | ||
| const buffer = Buffer.from(geoString, 'hex'); | ||
| const binaryGeo = parseSync(buffer, WKBLoader); |
There was a problem hiding this comment.
This looks great. We should probably just use the WKTLoader from loaders.gl/wkt for the wkt parsing on line 138. It is based on wellknown so it should be equivalent but keeping consistent API.
There was a problem hiding this comment.
Agreed. Will try to use WKTLoader instead of wktParser from wellknown package. Thanks!
| // try parse as wkb using loaders.gl WKBLoader | ||
| if (!parsedGeo) { | ||
| try { | ||
| const buffer = Buffer.from(geoString, 'hex'); |
There was a problem hiding this comment.
Nit: Wish we had a loader for this so we didn't need to mess with Node.js Buffer class. I will give it some thought.
There was a problem hiding this comment.
Thanks! Ib will address this in visgl/loaders.gl#2652
| * @param str input string | ||
| * @returns true if string is a valid WKB in HEX format | ||
| */ | ||
| export function isHexWkb(str: string | null): boolean { |
There was a problem hiding this comment.
This is interesting. I wonder how to handle HexWKB in loaders.gl. Maybe just add HexWKBLoader / HexWKBWriter?
There was a problem hiding this comment.
HexWKBLoader/Writer sounds great 👍
| @@ -21,7 +21,7 @@ | |||
| "strict": true, | |||
| "resolveJsonModule": true, | |||
| "isolatedModules": true, | |||
| "baseUrl": "./src", | |||
| "baseUrl": ".", | |||
There was a problem hiding this comment.
What does this do? Looks unrelated?
There was a problem hiding this comment.
This is not related to WKB, but it seems not been set correctly and causing the alias "@kepler.gl/*" not mapping to correct source code. (see below @kepler.gl/*": ["src/*/src"])
| if (!parsedGeo) { | ||
| try { | ||
| const buffer = Buffer.from(geoString, 'hex'); | ||
| const binaryGeo = parseSync(buffer, WKBLoader); |
There was a problem hiding this comment.
Agreed. Will try to use WKTLoader instead of wktParser from wellknown package. Thanks!
| * @param str input string | ||
| * @returns true if string is a valid WKB in HEX format | ||
| */ | ||
| export function isHexWkb(str: string | null): boolean { |
There was a problem hiding this comment.
HexWKBLoader/Writer sounds great 👍
|
@ibgreen added new functions e.g. Let me know if merging this PR is OK, or if we should wait for the patch in loaders.gl v3. Thanks! |
|
Yes its fine to merge this now. It will take a while to issue a loaders 3.4 patch. |
In Kepler.gl, geometries (Polygons, Points, LindStrings etc) can be embedded into CSV as a GeoJSON or WKT formatted string (see doc here). This PR is to support WKB geometry column in CSV file.
For example, the following CSV row contains a WKT geometry:
the following CSV row contains a WKB geometry in hex format:
(In PostGIS, the WKB in hex is the default format of the output of the geometry column, unless using e.g. ST_AsWKT)
For Arrow/GeoArrow, there is another wip PR to support Arrow/GeoArrow in Kepler.gl: GeoDaCenter#2