Checking that a set of requirements is available and compatible (ALA pip check)

[FFY00]

As discussed on IRC, I propose adding a requirement validator to packaging. There's is a real need for it, I need it for at least some of my projects.

The validator should find the metadata (using importlib.metadata) and validate the specified requirement string.

[pradyunsg]

Are you asking for #147?

[FFY00]

No, sorry. I wasn't clear enough. I am asking for a way to validate if the requirements are met. This can be used for eg. in python-build to validate the build dependencies are met, or in a wheel installer, to make sure the dependencies are available, and are compatible, before installing the wheel.

[FFY00]

I have a minimal implementation, but it does not validate the dependencies of extras. It would be good to have a strong implementation here, since it turns out it is not trivial.

https://github.com/FFY00/python-build/blob/master/build/__init__.py#L39

[pradyunsg]

I see, something like pip check but independent of pip?

[FFY00]

Yes.

[brettcannon]

One thing to consider about this proposal is I think this would be the first bit of functionality that actually queries installed packages. Everything else in 'packaging' is mostly parsing. packaging.tags is the only thing that queries the live interpreter running, but we don't have anything that goes into the environment to accomplish something.

Now I'm not saying expanding out is bad, I just want to be upfront it does open up the scope of this project a bit (and bring in a dependency on importlib-metadata).

[FFY00]

I agree, we should consider if we should use importlib.metadata or something else.

I would say the importlib-metadata dependency is just a temporary nuisance, importlib.metdata is available from Python 3.8 and up. Given time, older versions a compatible Python version will start to get adopted. If the importlib-metadata dependency is still a problem, we can declare an extra that pulls it.

[pradyunsg]

FWIW, I'm certainly not 100% sure if packaging is the correct place to put it. It might be that importlib should grow a helper function to do this or something.

I do think decoupling this from pip is a good idea, but IDK if packaging is the correct place.

[FFY00]

But it would need to use the functionality provided by packaging. As importlib is part of the standard library, this is not possible.

I agree packaging might not be the most optimal place, but given the context, I believe it is where it makes most sense. And the functionality proposed falls somewhat in scope of the project.

Do you have any place in mind other than importlib?

[brettcannon]

I agree that I don't think importlib is a good fit for this.

Would this big a good fit for pradyunsg/installer?

[FFY00]

I think it makes more sense to have it here. This library aims to provide "Core utilities for Python packages", I think verifying if the requirements are met is one of those workflows. pradyunsg/installer provides a library with the install logic, I do not really believe this is install logic, IMO "packaging logic" better describes it.

[layday]

Regardless of what happens here I think it would be useful to expose individual markers in markers.Marker so we can e.g. filter requirements based on their marker value for extras. A related issue is that Marker.evaluate will raise if its own extra is missing from the environment, so requirements cannot be evaluated if they contain an extra (i.e. if they've been extracted from wheel metadata).

[FFY00]

We have a working implementation in pypa/build. How do we want to proceed?

[FFY00]

Sorry to bother, but we have working code for this. Would a PR be welcome?

[brettcannon]

I'm still on the fence for this since it just feel like it quite fits into 'packaging' (don't ask me why; gut feel), but then I don't know where it would belong in PyPA either.

[FFY00]

Yeah, and I agree that pypa/packaging might not be the best place to put it, but I think it is close enough to be relevant and from all other projects I think pypa/packaging is what makes the most sense.

I would really like to figure this out, otherwise it will be indefinitely stalled 😕

This simply integrates the pypa/packaging functionality with stdlib's importlib.metadata.

It feels silly to create a new package just for this, but that could be done if we don't arrive to any consensus. I mean, maybe we could bundle a few package introspection routines, but I struggle to make a strong case for this new package.

[layday]

I suppose we could use a callback in place of importlib.metadata if that makes it more palatable for inclusion in packaging. I've adapted the implementation from build with this in mind: https://github.com/layday/build/blob/prospective-packaging-implementation/check_dependencies.py.