PDM, Nox and Meson

[paugier]

Oh finally it was much more changes that I expected...

This is a draft, but it gives a good idea of what it could give.

There are few interesting commands in the Makefile, in particular make should install the package in a dedicated local virtual env (.venv).

I used Nox instead of Tox because it is simpler for me.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.38%. Comparing base (604483b) to head (c51cb7f).
Report is 6 commits behind head on main.

❗ Current head c51cb7f differs from pull request most recent head 9f42693

Please upload reports for the commit 9f42693 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #51      +/-   ##
==========================================
- Coverage   94.40%   94.38%   -0.03%     
==========================================
  Files           6        5       -1     
  Lines        1037     1033       -4     
==========================================
- Hits          979      975       -4     
  Misses         58       58              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[martibosch]

Thank you so much for doing this @paugier ! I have no experience using pdm and meson but I understand that pylandstats is now compiled using pythran in windows too, right?

Regarding nox: I am okay with switching, but if I understand this correctly, we still need to keep tox.ini? How about the tests in github actions? shouldn't tests.yml be changed to use nox?

Thank you again!

[paugier]

but I understand that pylandstats is now compiled using pythran in windows too, right?

Yes, exactly! And as a bonus there are also the files necessary to use Numba in the wheels. So Numba can be used if TRANSONIC_BACKEND is set to "numba".

How about the tests in github actions? shouldn't tests.yml be changed to use nox?

Yes, in practice, it is not necessary to have tox.ini and noxfile.py. I didn't touch the Github Actions configuration so I didn't remove tox.ini. Do you feel you can try to adapt tests.yml to use Nox? You can find an example here https://foss.heptapod.net/fluiddyn/fluidimage/-/blob/branch/default/.github/workflows/ci-linux.yml Do not hesitate to tell me if it does not work!

[martibosch]

well for some reason some pythran build errors seem to occur in github actions 😿

[paugier]

I agree that it looks reasonable. It looks a bit like a Nox bug. I'm going to have a look at that.

[martibosch]

Hello @paugier,

sorry to ping you again. Builds in conda-forge are failing due to a missing setup.py, which I suppose it is more an issue of the setuptools version. However, I would like to ask whether the current host and run requirements (see https://github.com/conda-forge/pylandstats-feedstock/blob/ff2cdf96342d4516c6c9a3869952a57210dc5efa/recipe/meta.yaml) make sense after the changes of this PR.

I ran grayskull locally and suggested:

requirements:
  host:
    - python >=3.9
    - meson-python
    - numpy
    - pythran >=0.16.1
    - transonic >=0.4.0
    - pip
  run:
    - python >=3.9
    - black
    - dask-core
    - geopandas
    - matplotlib-base >=2.2
    - numba  # [win]
    - numpy >=1.15
    - pandas >=0.23
    - rasterio >=1.0.0
    - scipy >=1.0.0
    - transonic >=0.6.4

does this make more sense? I am especially curious of whether we still need black and about the # [win] for numba.

Thank you again. Best,
Martí

[martibosch]

I have applied the changes as suggested by grayskull and a few other tweaks (see https://github.com/conda-forge/pylandstats-feedstock/blob/089d09ecaebe0f2a7931abc9927181c468fc40c5/recipe/meta.yaml), however now I get the following error in windows only (see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1064001&view=results):

..\meson.build:1:0: ERROR: Compiler clang++ cannot compile programs.

I wonder if this has something to do with the fact that I removed the build requirements (see https://github.com/conda-forge/pylandstats-feedstock/blob/ff2cdf96342d4516c6c9a3869952a57210dc5efa/recipe/meta.yaml#L19-L21).

Thank you.

[paugier]

I guess you need

    - {{ compiler('cxx') }}
    - {{ stdlib("c") }}

to get a working clang.

I think you don't need numba (and black) anymore.

[martibosch]

well, for one it was safe to remove the black requirement (https://github.com/martibosch/pylandstats/actions/runs/11519189852), but I am still struggling with the conda-forge builds in windows. More precisely, now I am getting errors of the form cannot overload a member function without ref-qualifier with a member function with ref-qualifier (see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1064175&view=results).

Following some similar issues that I found and based on the scipy and scikit-image conda forge recipes, I added the following to the build requirements (besides the two lines in the snippet of the previous comment):

    - clang  # [win]    

However I am still getting the same error. Any ideas on how to fix this @serge-sans-paille (sorry to ping you here too).

Thank you. Best,
Martí

[serge-sans-paille]

You should probably for the clang-cl compiler on windows if that's not the case?

[martibosch]

Hello! I have tried mimicking what is done in scikit-image-feedstock by copying their bld.bat (which did not solve the issue) and then I also added {{ compiler('c')}} as build requirement (again, based on scikit-image's conda recipe) and instead got a different error (see https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=1065040&view=results)

conda_libmamba_solver.conda_build_exceptions.ExplainedDependencyNeedsBuildingError: Unsatisfiable dependencies for platform win-64: {MatchSpec("vs_win-64==2017.9=hddac466_12"), MatchSpec("vs2017_win-64=19.16.27033")}

Then I also tried adding clangxx # [win] to the build requirements (the current version of the PR) but then got the original error again.

any ideas? I am sorry to have to ping you guys but this is really outside my expertise.

[martibosch]

Hello!

I am sorry to ping you again, but would you have any hint on how to get this fixed? I just tried re-running the failed workflows (to see if some updated dependencies magically fixed the issue) without success.

Here is the:

• PR: pylandstats v3.0.0 conda-forge/pylandstats-feedstock#30
• recipe: https://github.com/regro-cf-autotick-bot/pylandstats-feedstock/blob/3.0.0_h1bebcf/recipe/meta.yaml
• bld.bat: https://github.com/regro-cf-autotick-bot/pylandstats-feedstock/blob/3.0.0_h1bebcf/recipe/bld.bat

Any help will be greatly appreciated. Best,
Martí