Explain how to use uv with airflow virtualenv and make it works

[potiuk]

Since we are switching to uv as our main development tooling, we should explain how to use uv with airflow and explain some basic commands that should be used to have a workign uv-manaaged venv.

This documentation explains some why's and initial hows with uv, also it fixes uv to work on macos with some default extras - such as devel, devel-tests and --all-extras, so that it works on a wider range of systems (includin MacOS). This includes making plyvel not installed on MacOS, because it's next to impossible to compile levelDB on a modern MacOS Operating system and it is anyway an optional component of google provider.

Fixes: #43200

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

Just to copy a bit more context on this change:

My - very basic and long term assumption for our local dev environment is that while uv makes it (and it will only get better) to maintain, upgrade, sync and setup the environment very easily and the UX is great and complexity is low, we do not want to EXCLUSIVELY have setup that only works with UV

I think everything that works with uv should also be achievable (even if in a much more convoluted way) with the standard tooling that is managed by PSF-managed tools (pip , hatch etc.) .

While I love Astral, they are VC-backed company that will expect some returns of investment, and while I believe they are great community member and have no evil goals, they might get acquired, or whatever at any time, and we do not want to have a “strong” dependency on their tooling being available forever.

This goes in line with ASF’s “plan for 50 years” tools dependency approach - use what’s available and makes you more productive, but don’t rely too much on anything you cannot exchange easily. Sooner or later EVERY vendor we work with will disappear, and we must be there.

That’s why this doc has two chapters “recommended with UV” and “here is how you can do it with pip”

I also removed hatch - for simplicity - we do not want to describe 3 tools, also I removed plyvel as dependency as it is almost impossible to install it MacOs and it blocked --all-extras and --devel extras to be installed for MacOS

All the rest might fail when installing - and you might need to have some system dependencies but they should be “easy” to install with brew and others

Plyvel has a long time history of breaking MacOS and 10.15 is famously broken now wbolster/plyvel#114

Also uv.lock is excluded for now until we figure out if /how to keep it updated acrooss all ~700 dependencies of ours and how to join it with the constraints we produce.

[amoghrajesh]

Tried reviewing it thoroughly and tested it step by step too, content looks fine, just a few general comments

[amoghrajesh]

Thanks for making the changes, LGTM +1