Skip to content

Commit

Permalink
[feat] experimental support for ARROW:extension:point; support for ar…
Browse files Browse the repository at this point in the history
…rrow:wkb for geojson layer (#2716)

- adds support for ARROW:extension:name EXTENSION_NAME.POINT position columns in Point, Line, Arc, Text layers.
- can pick geoarrow point column as source for Heatmap layer.
- interleaved xy geoarrow point vector is passed directly as a position data attribute for rendering.
- separate lon / lat columns are converted to geoarrow point vectors (only when source file is an .arrow file).
- this PR doesn't affect existing loading and support of non-geo columns.
- support for arrow:wkb in geojson layer. Atm features are transformed from wkb to features to binary features as getBinaryGeometriesFromArrow doesn't support arrow:wkb.

Known issues:
- text layer supplied with ARROW:extension:point column can only render string arrow columns (no values offsets).

Signed-off-by: Ihor Dykhta <[email protected]>
  • Loading branch information
igorDykhta authored Nov 3, 2024
1 parent 2668757 commit d783b43
Show file tree
Hide file tree
Showing 42 changed files with 1,988 additions and 461 deletions.
9 changes: 5 additions & 4 deletions examples/demo-app/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,11 @@
"@kepler.gl/schemas": "^3.0.0-alpha.1",
"@kepler.gl/styles": "^3.0.0-alpha.1",
"@kepler.gl/utils": "^3.0.0-alpha.1",
"@loaders.gl/arrow": "^4.1.1",
"@loaders.gl/core": "^4.1.1",
"@loaders.gl/csv": "^4.1.1",
"@loaders.gl/json": "^4.1.1",
"@loaders.gl/arrow": "^4.3.2",
"@loaders.gl/core": "^4.3.2",
"@loaders.gl/csv": "^4.3.2",
"@loaders.gl/json": "^4.3.2",
"@loaders.gl/parquet": "^4.3.2",
"@types/classnames": "^2.3.1",
"@types/keymirror": "^0.1.1",
"classnames": "^2.2.1",
Expand Down
12 changes: 8 additions & 4 deletions examples/demo-app/src/actions.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ import {fetch} from 'global';
import {loadFiles, toggleModal} from '@kepler.gl/actions';
import {load} from '@loaders.gl/core';
import {CSVLoader} from '@loaders.gl/csv';
import {ArrowLoader} from '@loaders.gl/arrow';
import {GeoArrowLoader} from '@loaders.gl/arrow';
import {_GeoJSONLoader as GeoJSONLoader} from '@loaders.gl/json';
import {ParquetWasmLoader} from '@loaders.gl/parquet';

import {
LOADING_SAMPLE_ERROR_MESSAGE,
Expand Down Expand Up @@ -251,13 +252,16 @@ function loadRemoteData(url) {

// Load data
return new Promise(resolve => {
const loaders = [CSVLoader, ArrowLoader, GeoJSONLoader];
const loaders = [CSVLoader, GeoArrowLoader, ParquetWasmLoader, GeoJSONLoader];
const loadOptions = {
csv: {
shape: 'object-row-table'
},
arrow: {
shape: 'arrow-table'
},
csv: {
shape: 'object-row-table'
parquet: {
shape: 'arrow-table'
},
metadata: true
};
Expand Down
Loading

0 comments on commit d783b43

Please sign in to comment.