Docker/Panel fixes and improvements

.github/workflows/auto-publish.yml

@@ -1,14 +1,17 @@
 # This workflow will upload a Python Package using Twine when a release is created
 # For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
 
-name: Upload Package to PyPI
+name: Upload Package to PyPI and Publish NWBPanel Docker Image
 
 on:
+  # push:
+  #   branches:
+  #     - panel-fixes-and-improvements
   release:
     types: [published]
 
 jobs:
-  deploy:
+  publish-pypi:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2
@@ -29,3 +32,28 @@ jobs:
           verbose: true
           user: __token__
           password: ${{ secrets.PYPI_API_TOKEN }}
+
+  release-nwbpanel-image:
+    name: NWB Panel Image
+    runs-on: ubuntu-latest
+    needs: publish-pypi
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ secrets.REPO_GITHUB_USERNAME }}
+          password: ${{ secrets.REPO_GITHUB_TOKEN }}
+      - name: Build and push
+        uses: docker/build-push-action@v3
+        with:
+          push: true # Push is a shorthand for --output=type=registry
+          tags: ghcr.io/neurodatawithoutborders/nwbwidgets-panel:latest
+          context: docker
+          file: docker/Dockerfile_publish

CHANGELOG.md

@@ -1,6 +1,12 @@
 # Upcoming
 
-### New features
+
+# v0.11.0
+
+### New Features
+* Improvements on Panel Docker file, including path for read-only mount to access local files [PR #299](https://github.com/NeurodataWithoutBorders/nwbwidgets/pull/299)
+* Panel error handling with message output [PR #299](https://github.com/NeurodataWithoutBorders/nwbwidgets/pull/299)
+* New flag argument on Panel to enable/disable warnings [PR #299](https://github.com/NeurodataWithoutBorders/nwbwidgets/pull/299)
 * Improved browsing and reading local NWB files with Panel, using ipyfilechooser [PR #300](https://github.com/NeurodataWithoutBorders/nwbwidgets/pull/300)
 * Improve readibility of dandisets and files dropdown lists [PR #301](https://github.com/NeurodataWithoutBorders/nwbwidgets/pull/301)
 

README.md

@@ -41,6 +41,19 @@ you will see a list of extra modules needed for that specific widget.
 All other widgets in the file will still work.
 
 ## Usage
+
+### Using `Panel`
+The easiest way to use NWB widgets is with the interactive `Panel`:
+
+```python
+from nwbwidgets.panel import Panel
+
+Panel()
+```
+
+### Using `nwb2widget`
+If you’re working directly with a NWB file object in your Jupyter notebook, you can also explore it with NWB Widgets using
+
 ```python
 from pynwb import NWBHDF5IO
 from nwbwidgets import nwb2widget
@@ -51,6 +64,13 @@ nwb = io.read()
 nwb2widget(nwb)
 ```
 
+### Using Docker
+You can also run the NWB Widgets Panel using Docker:
+
+```bash
+$ docker run -p 8866:8866 ghcr.io/NeurodataWithoutBorders/nwbwidgets-panel:latest
+```
+
 ## Demo
 ![](https://drive.google.com/uc?export=download&id=1JtI2KtT8MielIMvvtgxRzFfBTdc41LiE)
 

docker/Dockerfile

@@ -1,16 +1,25 @@
-FROM python:3.10
+FROM python:3.10.12-slim
+LABEL org.opencontainers.image.source=https://github.com/NeurodataWithoutBorders/nwbwidgets
 
-RUN cd ..
 COPY requirements.txt .
 RUN pip install --upgrade pip \
     && pip install -r requirements.txt \
-    && pip install git+https://github.com/NeurodataWithoutBorders/nwbwidgets.git
+    && pip install nwbwidgets
 
-COPY notebook.ipynb .
+# python slim image does not have git, leaving this here only for reference for now
+# && pip install git+https://github.com/NeurodataWithoutBorders/nwbwidgets.git
+
+WORKDIR /app
+RUN mkdir local_files
 RUN mkdir nwb-cache
 
+COPY favicon.ico .
+COPY NWB_Panel.ipynb .
+COPY run.sh .
+
 EXPOSE 8866
-CMD ["voila", "notebook.ipynb", "--port=8866", "--Voila.ip=0.0.0.0", "--enable_nbextensions=True", "--autoreload=True", "--no-browser"]
+CMD ["./run.sh"]
+# CMD ["voila", "notebook.ipynb", "--port=8866", "--Voila.ip=0.0.0.0", "--enable_nbextensions=True", "--autoreload=True", "--no-browser"]
 
 # EXPOSE 8888
 # CMD ["jupyter", "notebook", "notebook.ipynb", "--port=8888", "--ip=0.0.0.0", "--no-browser", "--allow-root", "--NotebookApp.token=''"]

docker/Dockerfile_publish

@@ -0,0 +1,18 @@
+FROM python:3.10.12-slim
+LABEL org.opencontainers.image.source=https://github.com/NeurodataWithoutBorders/nwbwidgets
+
+COPY requirements.txt .
+RUN pip install --upgrade pip \
+    && pip install -r requirements.txt \
+    && pip install nwbwidgets
+
+WORKDIR /app
+RUN mkdir local_files
+RUN mkdir nwb-cache
+
+COPY favicon.ico .
+COPY NWB_Panel.ipynb .
+COPY run.sh .
+
+EXPOSE 8866
+CMD ["./run.sh"]

docker/README.md

@@ -2,22 +2,22 @@
 
 From `/docker` directory, run:
 ```bash
-$ docker build -t nwbwidgets-voila .
+$ DOCKER_BUILDKIT=1 docker build -t nwbwidgets-panel .
 ```
 
 ## Run Docker Image
 
 To run with `Panel`:
 ```bash
-$ docker run -p 8866:8866 nwbwidgets-voila
+$ docker run -p 8866:8866 nwbwidgets-panel
 ```
 
-By default, the Panel will run with `enable_local_source=False` (see [Panel](https://nwb-widgets.readthedocs.io/en/latest/contents/quickstart.html#basic-usage-with-panel)). To allow for local files browsering, run the container with:
+By default, the container will run to access remote files only, using `enable_local_source=False` (see [Panel](https://nwb-widgets.readthedocs.io/en/latest/contents/running_on_jupyter.html#using-panel)). To allow for local files browsering, run the container with:
 ```bash
-$ docker run -p 8866:8866 -e ENABLE_LOCAL_SOURCE=True nwbwidgets-voila
+$ docker run -p 8866:8866 -e ENABLE_LOCAL_SOURCE=True -v "$(pwd):/app/local_files:ro" nwbwidgets-panel
 ```
 
-To run for a specific NWB file, pass the s3 url as an ENV variable to the container:
+To run for a specific remote NWB file, pass the s3 url as an ENV variable to the container:
 ```bash
-$ docker run -p 8866:8866 -e S3_URL_NWBFILE=<s3_url_to_nwb_file> nwbwidgets-voila
+$ docker run -p 8866:8866 -e S3_URL_NWBFILE=<s3_url_to_nwb_file> nwbwidgets-panel
 ```

docker/requirements.txt

@@ -1,35 +1 @@
-pynwb>=2.2.0
-dandi>=0.46.6
-nwbwidgets>=0.10.0
-ndx-grayscalevolume==0.0.2
-ndx-icephys-meta==0.1.0
-ndx-spectrum==0.2.2
-voila==0.3.6
-numpy==1.23.3
-matplotlib==3.6.1
-scikit-image==0.19.3
-plotly==5.10.0
-ipydatagrid==1.1.13
-ipydatawidgets==4.3.2
-ipytree==0.2.2
-ipyvolume==0.6.0a10
-ipyvue==1.8.0
-ipyvuetify==1.8.4
-ipywebrtc==0.6.0
-ipywidgets==7.7.2
-ipykernel==6.16.0
-ipympl==0.9.2
-jupyter_client==7.4.2
-jupyter_core==4.11.1
-jupyter-server==1.21.0
-jupyterlab==3.4.8
-jupyterlab-pygments==0.2.2
-jupyterlab_server==2.15.2
-jupyterlab-widgets==1.1.1
-tqdm>=4.36.0
-zarr==2.13.3
-aiohttp==3.8.3
-tifffile==2022.10.10
-trimesh==3.15.4
-importlib-metadata<5.0
-fsspec==2022.8.2
+voila==0.4.1

docker/run.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+voila NWB_Panel.ipynb \
+  --port=8866 \
+  --Voila.ip=0.0.0.0 \
+  --enable_nbextensions=True \
+  --autoreload=True \
+  --no-browser \
+  --VoilaConfiguration.file_whitelist="['.*\.(png|jpg|gif|svg|mp4|avi|ico)']"

docs/source/contents/running_on_jupyter.md

@@ -1,5 +1,5 @@
 
-# Running NWB widgets on Jupyter
+# Running on Jupyter
 
 ## Using `Panel`
 

docs/source/contents/running_with_docker.md

@@ -0,0 +1,34 @@
+# Running with Docker
+
+## Use the pre-built image
+You can easily run NWB widgets Panel using Docker.
+
+```bash
+$ docker pull ghcr.io/NeurodataWithoutBorders/nwbwidgets-panel:latest
+$ docker run -p 8866:8866 nwbwidgets-panel
+```
+then navigate to `http://localhost:8866/` in your browser.
+
+By default, the container will run to access remote files only, using `enable_local_source=False` (see [Panel](https://nwb-widgets.readthedocs.io/en/latest/contents/quickstart.html#basic-usage-with-panel)). To allow for local files browsering, run the container with:
+```bash
+$ docker run -p 8866:8866 -e ENABLE_LOCAL_SOURCE=True -v "$(pwd):/app/local_files:ro" nwbwidgets-panel
+```
+where `$(pwd)` is the current working directly. Change it if needed to the path containing the NWB files you want to visualize.
+
+If you want to skip the default Panel and instead run it just for a specific remote NWB file, pass the s3 url as an ENV variable to the container:
+```bash
+$ docker run -p 8866:8866 -e S3_URL_NWBFILE=<s3_url_to_nwb_file> nwbwidgets-panel
+```
+
+
+## Build from source
+Start by cloning the repository:
+```bash
+$ git clone https://github.com/NeurodataWithoutBorders/nwbwidgets.git
+$ cd nwbwidgets/docker
+```
+
+Build the Docker image:
+```bash
+$ DOCKER_BUILDKIT=1 docker build -t nwbwidgets-panel .
+```

docs/source/index.rst

@@ -29,7 +29,7 @@ NWB Widgets is a library of widgets for visualization NWB data in a Jupyter note
 
    contents/installation
    contents/running_on_jupyter
-   contents/running_remotely
+   contents/running_with_docker
    contents/developer_guide
 
 .. toctree::

nwbwidgets/panel.py

@@ -1,4 +1,5 @@
 import concurrent.futures
+import warnings
 from pathlib import Path
 
 import fsspec
@@ -26,6 +27,7 @@
         enable_dandi_source: bool = True,
         enable_s3_source: bool = True,
         enable_local_source: bool = True,
+        show_warnings: bool = False,
         **kwargs,
     ):
         """
@@ -38,15 +40,22 @@
                 created under the current working directory. Defaults to None.
             enable_dandi_source : bool, default: True
                 Enable DANDI source option.
+            enable_s3_source : bool, default: True
+                Enable S3 source option.
             enable_local_source : bool, default: True
                 Enable local source option.
+            show_warnings : bool, default: False
+                Show warnings.
         """
         super().__init__(children=[], **kwargs)
 
         self.stream_mode = stream_mode
         self.io = None
         self.nwbfile = None
 
+        if not show_warnings:
+            warnings.filterwarnings("ignore")
+
         self.source_options_names = list()
         if enable_local_source:
             self.source_options_names.append("Local file")
@@ -207,14 +216,20 @@
         self.widgets_panel.children = [widgets.Label("loading...")]
         dandiset_id = self.source_dandi_id.value.split("-")[0].strip()
         file_path = self.source_dandi_file_dropdown.value
-        s3_url = get_file_url(dandiset_id=dandiset_id, file_path=file_path)
-        self._stream_s3_file(s3_url)
+        try:
+            s3_url = get_file_url(dandiset_id=dandiset_id, file_path=file_path)
+            self._stream_s3_file(s3_url)
+        except Exception as e:
+            self.widgets_panel.children = [widgets.Label(str(e))]
 
     def stream_s3_file(self, args=None):
         """Stream NWB file from S3 url"""
-        self.widgets_panel.children = [widgets.Label("loading...")]
+        self.widgets_panel.children = [widgets.Label("loading file...")]
         s3_url = self.source_s3_file_url.value
-        self._stream_s3_file(s3_url)
+        try:
+            self._stream_s3_file(s3_url)
+        except Exception as e:
+            self.widgets_panel.children = [widgets.Label(str(e))]
 
     def _stream_s3_file(self, s3_url):
         if self.stream_mode == "ros3":

nwbwidgets/version.py

@@ -1 +1 @@
-version = "0.10.2"
+version = "0.11.1"

requirements.txt

@@ -6,7 +6,7 @@ ipympl
 ipydatagrid
 ipyvolume>=0.6.0a10
 ipywidgets>=8.0.0
-plotly==5.15.0
+plotly==5.13.1
 tqdm>=4.36.0
 zarr
 tifffile