Implement support for jupyterlab-gallery config

[krassowski]

Reference Issues or PRs

• [ENH] - Extension to present and pull a set of examples/gallery #2456

What does this implement/fix?

Put a x in the boxes that apply

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds a feature)
• Breaking change (fix or feature that would cause existing features not to work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Build related changes
• Other (please describe):

Testing

• Did you test the pull request locally?
• Did you add new tests?

Any other comments?

[kenafoster]

Any consideration yet into how we could handle authenticating to a Github/Gitlab server with this workflow?

As a workaround to date, we've put a PAT in the HTTPS URL. But that would mean committing a PAT Into source code. Might not be a HUGE problem if it's scoped to read-only on the needed repo and the nebari config is protected... still not ideal

[krassowski]

@kenafoster One possibility would be to source the PATs from environment variables and substitute in the URLs; let's continue the discussion in nebari-dev/jupyterlab-gallery#2

[krassowski]

Tested locally:

jupyterlab:
  gallery_settings:
    title: Example repositories
    destination: examples
    exhibits:
      - title: Nebari
        git: https://github.com/nebari-dev/nebari.git
        homepage: https://github.com/nebari-dev/nebari
        description: 🪴 Nebari - your open source data science platform
        icon: https://raw.githubusercontent.com/nebari-dev/nebari-design/main/logo-mark/horizontal/Nebari-Logo-Horizontal-Lockup-White-text.svg
      - title: PyTorch Tutorial
        git: https://github.com/yunjey/pytorch-tutorial.git
        homepage: https://github.com/yunjey/pytorch-tutorial
        description: PyTorch Tutorial for Deep Learning Researchers
        icon: https://github.com/yunjey/pytorch-tutorial/raw/master/logo/pytorch_logo_2018.svg
default_images:
  jupyterlab: quay.io/nebari/nebari-jupyterlab:jupyterlab-gallery-db9c173-20240604

[krassowski]

A limitation for now is that users can see contents /etc/jupyter/. If we want to put PATs in we should configure, at least for jupyter_gallery config this file to be not readable for users. Alternatively, we need to use a Python file (rather than JSON) and pass environment variables.

[krassowski]

I attempted passing the file_permission: "0400" to the local_sensitive_file resource, but it is not used as a file at all, but just as a temporary variable.

Currently, all the config files get passed on to the of JupyterHub Helm chart via custom.extra-mounts as kubernetes config maps:

nebari/src/_nebari/stages/kubernetes_services/template/modules/kubernetes/services/jupyterhub/main.tf

Lines 85 to 104 in 749a1f1

	extra-mounts = merge(
	var.extra-mounts,
	{
	"/etc/ipython" = {
	name = kubernetes_config_map.etc-ipython.metadata.0.name
	namespace = kubernetes_config_map.etc-ipython.metadata.0.namespace
	kind = "configmap"
	}
	"/etc/jupyter" = {
	name = kubernetes_config_map.etc-jupyter.metadata.0.name
	namespace = kubernetes_config_map.etc-jupyter.metadata.0.namespace
	kind = "configmap"
	}
	"/opt/conda/envs/default/share/jupyter/lab/settings" = {
	name = kubernetes_config_map.jupyterlab-settings.metadata.0.name
	namespace = kubernetes_config_map.jupyterlab-settings.metadata.0.namespace
	kind = "configmap"
	}

which then get populated by the custom logic during the spawn sequence. This logic also only works with k8s but this is not a problem for nebari.

The problem is that Kubernetes config maps are designed to store non-confidential data as per the documentation.

A possibly better solution would be to use singleuser.extraFiles as these allow to set the file mode, and are recommended for providing configuration.

[krassowski]

I managed to use the dedicated k8 secrets via JupyterHub's singleuser.extraFiles but:

• there is a bug in k8s which means that read permissions from group cannot be removed: it has been open for years now and got lots of upvotes and two PRs but neither was merged, which is disappointing (Setting defaultMode is not Fully Respected When Pod.spec.securityContext.runAsUser is Set kubernetes/kubernetes#57923)
• the entire secretes mode handling in k8s seems to have many gotchas see Secrets with specific permissions (defaultMode or mode) not being applied in Kubernetes 1.4.0 kubernetes/kubernetes#34982

[krassowski]

The latest commit, b3a5bf9, includes a PoC implementation for setting the mode of config file to 0400. Unfortunately, this removes the access from server too, so the configs are not applied. We would need to spawn the server with elevated privilege as compared to the user who is accessing it, and later reduce them once creating kernels or terminals. This is possible on Linux by using capabilities with FOWNER or SETGID but will need extensive testing to ensure that the elevated permissions are cleared for any process that user may spawn, and it might not be secure if we cannot scope it appropriately.

An alternative might be to either:

• pass the settings via an environmental variable and unset it immediately once consumed, or
• pass the settings as a config file and remove it immediately once the gallery extension starts.

For now we can instead revert to using the configuration files as before, and document that the PAT need to be scoped to the specific repositories that are used and to be read-only (which is a good practice anyways) and that it might be accessed by the user.

[Adam-D-Lewis]

@krassowski It seems like Nebari would benefit from having docs on this. Could you open a docs PR that explain how to use it or at a minimum point to the docs in https://github.com/nebari-dev/jupyterlab-gallery?

[krassowski]

nebari-dev/nebari-docs#478 is waiting for review over a week now.

[Adam-D-Lewis]

Actually, is there a way to update the example repos? If so and it's not obvious how to do so or if there is not a way to update the example repo could you add that info to the docs? It sounds like there is an update button that shows up. If that's the case, docs may not be needed.

[krassowski]

Actually, is there a way to update the example repos? If so and it's not obvious how to do so or if there is not a way to update the example repo could you add that info to the docs? It sounds like there is an update button that shows up. If that's the case, docs may not be needed.

Thank you, this is actually a good point - I opened nebari-dev/jupyterlab-gallery#23 to track this on the jupyterlab-gallery side.