More flexible pinning of JupyterLab, Jupyter Notebook, IPython, and lower level components

[astrojuanlu]

Description

We're pinning Jupyter in various ways in the development requirements for projects:

kedro/kedro/templates/project/{{ cookiecutter.repo_name }}/src/requirements.txt

Lines 4 to 8 in c48a846

	ipython~=8.10; python_version >= '3.8'
	isort~=5.0
	jupyter~=1.0
	jupyterlab_server>=2.11.1, <2.16.0
	jupyterlab~=3.0, <3.6.0

and we're pulling more dependencies than needed. For example, jupyter in itself is basically a metapackage:

https://pypi.org/project/jupyter/1.0.0/#files

and it ships a bunch of stuff:

    install_requires    = [
        'notebook',
        'qtconsole',
        'jupyter-console',
        'nbconvert',
        'ipykernel',
        'ipywidgets',
    ],

I don't think we want to necessarily pull things like qtconsole or even ipywidgets.

This is related to #2519 in the sense that these should not be project dependencies anyway, but even when we address gh-2519, we should make these dependencies slimmer (see also (see #2569 (comment) for extra discussion). At the same time, our kedro.ipython extension isn't doing anything fancy, so apart from incompatibilities with other packages, we should keep upper version caps to a minimum I'd say.

Context

I'm happy to report that %load_ext kedro.ipython works nicely in Jupyter (Classic) Notebook 7.0.0rc0 🎉

And it would be cool if our users could benefit from the new interface as soon as it's out.

Note that Notebook is now based on JupyterLab, so it has more dependencies.

Anecdotally, I've found that many users still stick to the classic notebook interface and not all of them have migrated, nor intend to, to JupyterLab.

Possible Implementation

Make our pinnings something like

# requirements.txt
ipython>=8.10  # runtime dependency of kedro.ipython
notebook>=7.0  # favor modern version for better usability
# jupyterlab  # it's a dependency of notebook already, so not even needed

This would be blocked on gh-2158.

Possible Alternatives

Alternatively, we could drop the classic notebook and have users install only JupyterLab:

# requirements.txt
ipython>=8.10  # runtime dependency of kedro.ipython
jupyterlab>=3.0  # whatever version, old or new
# notebook  # not included, fewer dependencies are installed

[deepyaman]

Would this PR include evaluation of Jupiter Lab 4.x? I don't think anything should break, but it's probably worth checking in relaxing it. Noting, as discussed in backlog grooming, so it's not forgotten.

[astrojuanlu]

I did evaluate JupyterLab 4.x and Jupyter classic Notebook 7.x and everything seemed to work https://www.linen.dev/s/kedro/t/12356404/i-m-happy-to-report-that-load-ext-kedro-ipython-works-nicely#41f24afc-78e6-4d40-837a-1ba9c323e4e2

But yes, to your point, a bit of manual QA would be needed before relaxing the pinning.

[astrojuanlu]

Jupyter Notebook 7.0.0 is out https://github.com/jupyter/notebook/releases/tag/v7.0.0

[astrojuanlu]

Closing this in favour of #2276.