Update docs to remove mentions of pkg_resources

[astrojuanlu]

Description

Kedro uses pkg_resources to register plugins:

kedro/docs/source/extend_kedro/plugins.md

Lines 5 to 7 in 200ebdb

	## Overview
	Kedro uses [`setuptools`](https://setuptools.readthedocs.io/en/latest/setuptools.html), which is a collection of enhancements to the Python `distutils` to allow developers to build and distribute Python packages. Kedro uses various entry points in [`pkg_resources`](https://setuptools.readthedocs.io/en/latest/setuptools.html) to provide plugin functionality.

However, setuptools has just officially deprecated pkg_resources (see pypa/setuptools#3843):

Although pkg_resources has been discouraged for use, some projects still consider pkg_resources viable for usage. This change makes it clear that pkg_resources should not be used, emitting a DeprecationWarning when imported.

and:

Use of pkg_resources is deprecated in favor of importlib.resources, importlib.metadata, and their backports (importlib_resources, importlib_metadata). Users should refrain from new usage of pkg_resources and should work to port to importlib-based solutions.

The time has come to drop pkg_resources.

Tangentially related: #2152 and #2280.

Context

With the long support windows of Kedro, relying on deprecated functionality is a liability. The sooner we move away from it, the better.

In addition, importlib is much faster than pkg_resources (see discussion from 2017 pypa/setuptools#926).

Possible Implementation

Entry points are fully standardized in PEP 621, therefore setuptools is not a privileged build backend in this regard anymore (if you're curious, this is the spec).

The way to go is to use importlib.metadata: https://docs.python.org/3/library/importlib.metadata.html#entry-points

Possible Alternatives

• Don't do anything. Bad idea.
• Use a third party library like pluggy. Benefits seem unclear, plus other projects are considering leaving it behind too prepare a migration from pluggy hooks to entrypoints pypa/hatch#752.

[antonymilne]

Just to add a bit more information: we already use pluggy for hooks (both CLI and runtime). Actually our current handling of what uses entry points vs. what uses pluggy is somewhat haphazard and historical (e.g. the secret undocumented kedro.init entrypoint) and probably there's something we should improve, but that's out of the scope for just this issue. Related: #1041 #2384

Either way, migration away from pkg_resources is a no-brainer I think.

@astrojuanlu do you have any other examples of people dropping pluggy?

[antonymilne]

Wait, actually now that I look at the code again I think @noklam may have already done this migration to importlib when we made the kedro.starters entrypoint in #1592. We should look at our use of pkg_resources in kedro but I think it might just be that the docs are wrong here.

[astrojuanlu]

You're right @AntonyMilneQB, we're only using pkg_resources to parse requirements in micropackaging:

kedro/kedro/framework/cli/micropkg.py

Line 575 in 63fb4ca

requirements = pkg_resources.parse_requirements(requirements_txt.read_text())

I'm not a fan of feeding requirements.txt to the project metadata anyway (and I asked for help to find specific cases in which this might break), but in any case I agree that for now we can go ahead and adjust our docs.

[antonymilne]

I am also not a fan of this at all (or other parts of the micropackaging workflow) and there are definitely cases where it breaks: Fix inconsistent requirements handling in kedro pipeline package vs. kedro package. Using requirements.txt to provide install_requires has always been wrong.

[noklam]

The doc should be fixed as we switch to importlib for a while. It is worth to mention pkg_resources is the major source of slowing down Kedro CLI (for a relatively small project). I remember it's a bit tricky to parse the requirement so I couldn't remove the dependency for micropkg. I would love to see pkg_resources disappear from the codebase otherwise. :)

• [spike] Improve Kedro CLI startup time #1476

So I think this issue should be split into two part

• fixing the docs (minor doc change)
• (Not sure if we need a new issue or we can try to address this in Fix inconsistent requirements handling in kedro pipeline package vs. kedro package #1536