Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speed of GeoInterface.extent could be improved for large collections #112

Closed
alex-s-gardner opened this issue Sep 30, 2023 · 3 comments
Closed

Comments

@alex-s-gardner
Copy link
Contributor

alex-s-gardner commented Sep 30, 2023

Load packages

using GeoInterface
using GeometryOps

Download file

# for convenience I've temporarily put the files here
fn = "02_rgi60_WesternCanadaUS"
url2shp =  "https://its-live-data.s3.amazonaws.com/test/$fn.zip"

Downloads.download(url2shp, "$fn.zip")
run(`unzip $fn`)

Load polygons

glacier = Shapefile.Handle("$fn/$fn.shp")

Find extents

here's where it gets interesting

calculate extents on raw file

@time GeoInterface.extent(glacier)
0.000008 seconds (1 allocation: 64 bytes)
Extent(X = (-133.73235307, -105.60821024699995), Y = (36.38625038500004, 65.15664046000005), Z = (0.0, 0.0))

I guess this is just grabbing the extents from the header... that's the only place I can see extent info in glacier .

now retroject and get extents [requires https://github.com/JuliaGeo/Proj.jl/pull/97]

epsg = GeoFormatTypes.EPSG("EPSG:32609")
glacier_rep = GeometryOps.reproject(glacier; target_crs=epsg);

@time GeoInterface.extent(glacier_rep)
  3.575146 seconds (14.00 M allocations: 714.567 MiB, 11.20% gc time)
Extent(X = (273581.3643637521, 2.5071260739734517e6), Y = (4.0779523613324095e6, 7.23421013737856e6))

3.5s seems like a very long time for this operation. I suspect that the Extents if being calculated on each feature then the union of extents is being taken.. if this is the case it would probably be better to load all feature nodes in as points then take the extrema once.

@rafaqz
Copy link
Member

rafaqz commented Sep 30, 2023

Yeah, theres something wrong with that time.

Try ProfileView on it and see its all type instability, and what function its in.

As an aside, reproject should probably update the extent as it runs.

@rafaqz
Copy link
Member

rafaqz commented Oct 1, 2023

Turns out this is already fixed on master we just forgot to register it.

Should be 50x faster but it looks like it still isn't completely type stable.

@rafaqz
Copy link
Member

rafaqz commented Oct 4, 2023

Fixed in the latest release.

@rafaqz rafaqz closed this as completed Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants