Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support H3 spatial indexed data via the h3j extension? #68

Open
cboettig opened this issue Jan 28, 2025 · 4 comments
Open

support H3 spatial indexed data via the h3j extension? #68

cboettig opened this issue Jan 28, 2025 · 4 comments

Comments

@cboettig
Copy link
Contributor

Would you be open to a PR adding support for H3-indexed data?

I think the simplest approach would use h3j, as linked from the maplibre plugin page.

Motivation: H3 is a powerful hierachical spatial index. Because it is supported through a duckdb extension, R users can leverage dplyr to efficiently calculate and manipulate h3-indices, e.g. in group-by/summarise aggregations and joins.

@cboettig
Copy link
Contributor Author

I sketched this out in my fork by adding a add_h3j_sources() method that plugs into the existing layer interface like so

My approach adds a new function add_h3j_source mirrors the way the javascript extension provides addH3JSource() (unlike the PMTiles extension which took a different route but introducing a custom "protocol" for pmtiles:// layerd over the add_vector_source().

url = "https://inspide.github.io/h3j-h3t/examples/h3j/sample.h3j"
maplibre(center=c(-3.704, 40.417), zoom=15, pitch=30) |>
add_h3j_source("h3j_testsource",
              url = url
)  |>
  add_fill_extrusion_layer(
    id = "h3j_testlayer",
    source = "h3j_testsource",
    fill_extrusion_color = interpolate(
      column = "value",
      values = c(0, 21.864),
      stops = c("#430254", "#f83c70")
    ),
    fill_extrusion_height = list(
      "interpolate",
      list("linear"),
      list("zoom"),
      14,
      0,
      15.05,
      list("*", 10, list("get", "value"))
    ),
    fill_extrusion_opacity = 0.7
  )

Image

@cboettig cboettig changed the title support h3j extension? support H3 spatial indexed data via the h3j extension? Jan 31, 2025
@walkerke
Copy link
Owner

This looks awesome! Yes I'd be interested in a PR.

Motivation: H3 is a powerful hierachical spatial index. Because it is supported through a duckdb extension, R users can leverage dplyr to efficiently calculate and manipulate h3-indices, e.g. in group-by/summarise aggregations and joins.

I'm wondering, given your comment: would this be a method we could use for efficient plotting of high-density point data, e.g. bin the data and visualize without having to send a hefty GeoJSON over to be rendered?

@walkerke
Copy link
Owner

@cboettig one thing I notice is that the repo does not appear to be maintained (although it's good to know that it still works); perhaps we fork it and modify as needed - I saw your open issue. I also note that they are using a very old version of maplibre on their side, so we'd want to make sure it works with anything new.

@cboettig
Copy link
Contributor Author

@walkerke Thanks Kyle! These are good questions.

Yeah I hear you on the maintenance side, that's a good question. It have it working fine in my fork, so at least so far seems good.

would this be a method we could use for efficient plotting of high-density point data, e.g. bin the data and visualize without having to send a hefty GeoJSON over to be rendered?

yes, this is precisely my use case. e.g. GBIF is a biodiversity occurrence dataset of some 3 billion rows, about half a terrabyte as parquet point data. Here is a demo app that bins the data by H3 hexes at various resolutions. You can aggregate data for the country or the world using low zooms (h4 or say) quite quickly, or zoom into a state or national park and aggregate at much finer resolutions: https://streamlit.carlboettiger.info (this one python/streamlit based, but we can do the same in R with your package now, and probably make it look much nicer :-) )

I should note that deck.gl also has support for h3, but that would not integrate into the existing maplibre syntax for add_fill_layer, add_extrusion_layer and all the power that comes with those, so a user would have to use a different function to tell deck.gl how to handle data color, fill, etc.

I don't know javascript but yeah I think we could fork it if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants