BLD: Initial linux wheel build setup

[mtsokol]

This PR introduces linux wheel build process. In the CI there are two jobs:

1. build-wheel:

• builds LLVM in a build dir
• installs LLVM in an install dir
• builds Finch dialect in a build dir in a standalone mode
• installs Finch dialect to setuptools-install dir
• uploads build artifact

2. release-wheel:

• downloads build artifact
• creates a release

[mtsokol]

@willow-ahrens I did my homework yesterday and building Finch dialect in a standalone mode yields a wheel of size 200MB.

The CI run first builds LLVM and then Finch dialect in an out-of-tree mode. Only the Finch part build is placed in a wheel and uploaded as a release artifact: https://github.com/nullplay/Finch-mlir/releases

Then the wheel can be directly installed via pip, here are example shell commands:

conda create --name temp-env python=3.10 --yes

conda activate temp-env

pip install PyYAML numpy

pip install https://github.com/nullplay/Finch-mlir/releases/download/latest/mlir_finch-0.0.1-cp310-cp310-linux_x86_64.whl

python -c 'from mlir_finch.dialects import finch, arith, func, linalg, sparse_tensor, tensor'

conda env remove --name temp-env --yes

I think mlir_c_runner_utils lib is missing from the CMake setup - it should be included in the build - it's required to run ExecutionEngine from the Python layer (but I think it just a few MBs only).

This build setup will be run periodically. Now it's only python 3.10, linux, x86_64 but target build matrix could be [3.10, 3.11, 3.12] x [linux, macos, win64] x [x86_64, arm64].

CC @nullplay @hameerabbasi

[mtsokol]

It's for a later discussion but I see that libFinchCAPITestLib.so is also shipped in the install dir (150MB file size). Assuming by its name it might not be necessary to include it.

[hameerabbasi]

A couple of quick comments:

• If this is down to 10m, that's AWESOME -- I'd support uploading a wheel every commit!
• Let's have a job that builds Finch-mlir first and uploads that to artifacts, then another that "just" builds wheel/bindings using the already-built artifact; we save on CI resources this way
• We should test the wheel before publishing it
• Let's have a tag rolling in addition to the x.y.z tags.

[mtsokol]

* Let's have a job that builds Finch-mlir first and uploads that to artifacts, then another that "just" builds wheel/bindings using the already-built artifact; we save on CI resources this way

Currently here there are two jobs: one that builds artifact and separate for publishing the wheel

* We should test the wheel before publishing it

Do you mean a separate job that installs the wheel and imports something? Or a more sophisticated mechanism?

[mtsokol]

Ok, I think I will go and merge what we have here - I will continue in a follow-up PR.
@hameerabbasi Let me know if you have any more comments.

[hameerabbasi]

Just Windows and 3.13 support but we’re all set.

[mtsokol]

@hameerabbasi Sure! Can you approve it? It's needed for merging.

The follow-up TODO:

• Windows support (I think this will take some time. https://github.com/makslevental/mlir-wheels has a lot of windows specific ifs in CI)
• enable 3.13

And the rest is lower priority:

• ubuntu arm64 build (qemu)
• build_local.sh script
• inspect ccache cache size, final CI yaml cleanup

[mtsokol]

@hameerabbasi It looks that Numba doesn't support Python 3.13 - pip install numba fails. So I can't install sparse in the CI, as Numba is a dependency, therefore I can't test built wheel.

I think sparse should first support Python 3.13 and then we can add 3.13 here (So we need for Numba to support 3.13).

[hameerabbasi]

@hameerabbasi It looks that Numba doesn't support Python 3.13 - pip install numba fails. So I can't install sparse in the CI, as Numba is a dependency, therefore I can't test built wheel.

I think sparse should first support Python 3.13 and then we can add 3.13 here (So we need for Numba to support 3.13).

Since we don't use import Numba in the MLIR backend, we can use it, we just need to install sparse with the --no-deps flag.