Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add actinia notebook #573

Merged
merged 17 commits into from
Dec 8, 2023
Merged

Add actinia notebook #573

merged 17 commits into from
Dec 8, 2023

Conversation

neteler
Copy link
Contributor

@neteler neteler commented Oct 13, 2023

The cloud based geoprocessing platform actinia is able to ingest and analyse large volumes of geodata. This PR proposes a notebook showcasing the processing of a hillshade map from an external data source (SRTM map).

To interface with actinia, the https://actinia-org.github.io/actinia-python-client/ is used.

Fixes #348

Requires actinia-org/actinia-core#482

Copy link

@mmacata mmacata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to add extent=region to r.import ? Then only the needed parts are imported (Not tested yet)

@neteler neteler marked this pull request as ready for review October 20, 2023 15:32
@neteler neteler requested a review from mmacata October 20, 2023 15:32
@neteler
Copy link
Contributor Author

neteler commented Oct 20, 2023

Would it be possible to add extent=region to r.import ? Then only the needed parts are imported (Not tested yet)

I've been using extent=import as we don't know the computational region a priori.

@giswqs
Copy link
Member

giswqs commented Oct 25, 2023

@neteler Thank you for working on this. Is this PR ready to reviewed and merged? Or are you still working on it?

@neteler
Copy link
Contributor Author

neteler commented Oct 25, 2023

The current blockers are actinia-org/actinia-core#482 and actinia-org/actinia-core#491 which we are working on. This PR should be fine but cannot be used yet.

That said, we also face a problem when testing this PR against a locally patched actinia server:

>>> leafmap.cog_bounds(url)
>>> leafmap.cog_center(url)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ mneteler/test/venv/lib/python3.8/site-packages/leafmap/stac.py", line 400, in cog_center
    center = ((bounds[0] + bounds[2]) / 2, (bounds[1] + bounds[3]) / 2)  # (lat, lon)
TypeError: 'NoneType' object is not subscriptable
>>> leafmap.cog_bands(url)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ mneteler/test/venv/lib/python3.8/site-packages/leafmap/stac.py", line 426, in cog_bands
    bands = [b[0] for b in r["band_descriptions"]]
KeyError: 'band_descriptions'
>>> leafmap.cog_tile(url)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ mneteler/test/venv/lib/python3.8/site-packages/leafmap/stac.py", line 154, in cog_tile
    band_names = cog_bands(url, titiler_endpoint)
  File "/home/ mneteler/test/venv/lib/python3.8/site-packages/leafmap/stac.py", line 426, in cog_bands
    bands = [b[0] for b in r["band_descriptions"]]
KeyError: 'band_descriptions'

The reason for this keyerror seems to be tests used in leafmap which use

https://titiler.xyz/cog/info?url=http://demouser:[email protected]:8088/api/v3/resources/demouser/resource_id-0c2014b7-3264-4419-85a9-812325ed40b4/hillshade.tif

and fail because https://titiler.xyz can not work with a local server (seems to be used around here). But that's certainly somewhat unrelated.

We are working on the completion of the /vsicurl/ support in actinia-core and will then deploy the new version in order to support the approach used in this PR.

@neteler
Copy link
Contributor Author

neteler commented Oct 25, 2023

Btw: we can increment the number of this notebook if you need 83 for another one!

@giswqs
Copy link
Member

giswqs commented Oct 25, 2023

No worries. I don't have new notebooks to add at the moment

The cloud based geoprocessing platform actinia is able to ingest and analyse large volumes of geodata.
This PR proposes a notebook showcasing the processing of a hillshade map from an external data source (SRTM map).

We use the https://actinia-org.github.io/actinia-python-client/.
@neteler
Copy link
Contributor Author

neteler commented Nov 27, 2023

Still a few things to tweak, but after redeployment of actinia.mundialis.de which now supports /vsicurl/ we now have the workflow from fetching data stored somewhere online, processing it in actinia and visualising the results in leafmap:

image

@neteler
Copy link
Contributor Author

neteler commented Nov 27, 2023

Nonetheless the definition of OpenTopoMap in f952854 I get the following error (still Esri.WorldTopoMap is loaded):

Help welcome!

------------------
m.add_basemap("Esri.WorldTopoMap")
------------------

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 m.add_basemap("Esri.WorldTopoMap")

File /usr/share/miniconda3/envs/test/lib/python3.11/site-packages/leafmap/plotlymap.py:200, in Map.add_basemap(self, basemap)
    194 """Adds a basemap to the map.
    195 
    196 Args:
    197     basemap (str, optional): Can be one of string from basemaps. Defaults to 'ROADMAP'.
    198 """
    199 if basemap not in basemaps:
--> 200     raise ValueError(
    201         f"Basemap {basemap} not found. Choose from {','.join(basemaps.keys())}"
    202     )
    204 if basemap in self.get_tile_layers():
    205     self.remove_basemap(basemap)

ValueError: Basemap Esri.WorldTopoMap not found. Choose from OpenStreetMap,ROADMAP,SATELLITE,TERRAIN,HYBRID,
...

@giswqs
Copy link
Member

giswqs commented Nov 28, 2023

The notebook error is a werid one. I can't reproduce it locally. You can comment it out temporarily. m.add_basemap("Esri.WorldTopoMap")

For the colorbar, I would suggest using the m.add_colormap method. It looks better than the branca colorbar.
https://leafmap.org/notebooks/23_colormaps/

m = leafmap.Map()
m.add_basemap("OpenTopoMap")
m.add_colormap(
    'terrain',
    label="Elevation",
    width=4,
    height=0.3,
    orientation='horizontal',
    vmin=0,
    vmax=4000,
)
m

image

@neteler
Copy link
Contributor Author

neteler commented Nov 29, 2023

Thanks for the suggestion, the error disappeared the color bar looks better.
Still the weird error remains (see CI).

Now the result looks like this:

image

TODO: fix an actinia-python-client error in this notebook, then this PR should be ready.

@giswqs
Copy link
Member

giswqs commented Nov 29, 2023

I just committed changes to ignore the notebook that causes the werid error. It should pass the tests now.

@neteler
Copy link
Contributor Author

neteler commented Nov 29, 2023

Thanks for the doc fix/workaround.

TODO to self: sort out remaining actinia glitch.

@neteler
Copy link
Contributor Author

neteler commented Dec 8, 2023

TODO to self: sort out remaining actinia glitch.

This has now been addressed in
https://github.com/actinia-org/actinia-python-client/releases/tag/0.3.1

From my point of view ready to merge. Thanks for your patience.

@giswqs
Copy link
Member

giswqs commented Dec 8, 2023

It looks great! I made some minor changes, including improving the installation method because wget does not work on Windows, and clean up the notebook.

Thank you very much for your time and efforts on this! Excited to see this happen.

@neteler
Copy link
Contributor Author

neteler commented Dec 8, 2023

It looks great! I made some minor changes, including improving the installation method because wget does not work on Windows,

Right, good point and thanks for the fix.

and clean up the notebook.
Thank you very much for your time and efforts on this! Excited to see this happen.

Thank you for inviting us to contribute this notebook and for all your support.

Also thanks to @mmacata, @anikaweinmann and @linakrisztian for the needed updates in actinia-core and the actinia-python-client!

@giswqs giswqs merged commit cbbb32c into opengeos:master Dec 8, 2023
12 checks passed
@giswqs
Copy link
Member

giswqs commented Dec 8, 2023

The notebook is available on the website now: https://leafmap.org/notebooks/87_actinia

@neteler neteler deleted the actinia_notebook branch January 30, 2024 22:54
sthagen pushed a commit to sthagen/giswqs-leafmap that referenced this pull request Jul 10, 2024
kuzja111 pushed a commit to kuzja111/leafmap that referenced this pull request Jul 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Actinia
3 participants