Allow to run jupyter-gallery server in a sidecar container

[krassowski]

Reference Issues or PRs

Closes #2514 - this is the proposed solution (a)

What does this implement/fix?

Put a x in the boxes that apply

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds a feature)
• Breaking change (fix or feature that would cause existing features not to work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Build related changes
• Other (please describe):

Testing

• Did you test the pull request locally?
• Did you add new tests?

Any other comments?

To test this feature:

1. Shut down your Jupyter server
2. Deploy with the following config in your nebari-config.yaml:

   jupyterlab:
     gallery_settings:
       title: Example repositories
       destination: examples
       run_in_separate_container: true
       exhibits:
       - title: Nebari
         git: https://github.com/nebari-dev/nebari.git
         homepage: https://github.com/nebari-dev/nebari
         description: 🪴 Nebari - your open source data science platform
   default_images:
     jupyterlab: quay.io/nebari/nebari-jupyterlab:jupyterlab-gallery-v0.5.0-72d14e3-20240727

3. Open a notebook/terminal and execute ls /etc/jupyter
4. See that jupyyter_gallery_config.json is no longer present (when run_in_separate_container is set to true)

Still to be done

• update docker image main branch (unless we want a dedicated, smaller docker image):
  • with a new version of jupyter-server-proxy one the PR with SSE streaming support is merged
  • with a new version of jupyterhub once the PR patching **kwargs handling in initalize is merged
  • remove the monkeypatch for the latter from this PR once we can
• implement (and test) fallback volume mounting for run_in_separate_container=false
• open a documentation PR against nebari docs
• add resource limits for the container
• rename variables and functions
  • preserve_extra_files is too technical, and now it does more than that;
  • etc-jupyter-secret-config-data is correct if we want to treat more config files like that; but then we probably would want to use a more generic name for this container (currently "gallery-sidecar")
  • run_in_separate_container - this is the admin-facing config option; other ideas welcome, e.g. isolate_runtime_and_config
• reconsider if we can re-enable xsrf token validation (I will write down a separate comment on this one later)

[dcmcand]

@krassowski what is the status here? Is it still being worked on?

[krassowski]

I have some work locally. I would need more time to rebase it on top of the current main. There are no blockers. I am just unable to get enough focus time given the time it takes to deploy/iterate.

[dcmcand]

Ok, I am going to close this, but leave the branch, when you have time to work on it, we can open a new PR.

[krassowski]

FYI, GitHub has an annoying UX when PR gets closed and then rebased:

[krassowski]

To clarify I just pushed the local changes to make sure they are up, but they will not show up here and the conversation will not be updated. No need to open a new one as I will not get to finalizing it this week.