Add support for python version 3.12 (#1068)

* Update python version in toml file. * Remove python version marker from CZI reader. * Fix typecheck errors. * Update poetry lock files. * Update poetry from 1.6.1 to 1.8.2. * upgrades a bunch of dependencies * upgrade deps in cluster_tools * Update poetry.lock. * Remove own fixes for fixed bugs in vcr. * update fastremap * format * cassettes * Update cassettes. * Run formatter, linter and typechecker. * Update cassettes. * Downgrade universal-pathlib to v0.1.4 * update upath * fix upath bug, maybe? * Update changelog. * Remove official python 3.8 support. * Update cluster_tools readme. * Update changelog. * Update ci. * Update ci.yml * Update ci.yml --------- Co-authored-by: Norman Rzepka <[email protected]>
scalableminds · Aug 2, 2024 · 7e85e0c · 7e85e0c
1 parent a166aea
commit 7e85e0c
Show file tree

Hide file tree

Showing 53 changed files with 2,889 additions and 2,610 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -38,7 +38,7 @@ jobs:
       max-parallel: 4
       matrix:
         executors: [multiprocessing, slurm, kubernetes, dask]
-        python-version: ["3.11", "3.10", "3.9", "3.8"]
+        python-version: ["3.12", "3.11", "3.10", "3.9"]
     defaults:
       run:
         working-directory: cluster_tools
@@ -54,8 +54,8 @@ jobs:
         run: |
           cd ./dockered-slurm
 
-          echo docker-compose up
-          docker-compose up -d
+          echo docker compose up
+          docker compose up -d
 
           # Register cluster (with retry)
           for i in {1..5}; do
@@ -138,7 +138,7 @@ jobs:
           PYTEST_EXECUTORS=kubernetes poetry run python -m pytest -sv test_all.py test_kubernetes.py
 
       - name: Run dask tests
-        if: ${{ matrix.executors == 'dask' && matrix.python-version != '3.8' }}
+        if: ${{ matrix.executors == 'dask' }}
         run: |
           cd tests
           PYTEST_EXECUTORS=dask poetry run python -m pytest -sv test_all.py test_dask.py
@@ -152,7 +152,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: ["3.11", "3.10", "3.9", "3.8"]
+        python-version: ["3.12", "3.11", "3.10", "3.9"]
         group: [1, 2, 3]
       fail-fast: false
     defaults:

diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
@@ -11,7 +11,7 @@ jobs:
     strategy:
       max-parallel: 4
       matrix:
-        python-version: ["3.8", "3.9", "3.10", "3.11"]
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
         group: [1, 2, 3]
       fail-fast: false
     defaults:

diff --git a/cluster_tools/README.md b/cluster_tools/README.md
@@ -22,7 +22,7 @@ if __name__ == '__main__':
 ```
 
 ## Installation
-The `cluster_tools` package requires at least Python 3.8.
+The `cluster_tools` package requires at least Python 3.9.
 
 You can install it from [pypi](https://pypi.org/project/cluster_tools/), e.g. via pip:
 

diff --git a/cluster_tools/poetry.lock b/cluster_tools/poetry.lock
diff --git a/cluster_tools/pyproject.toml b/cluster_tools/pyproject.toml
@@ -13,7 +13,7 @@ dask = ["distributed"]
 all = ["kubernetes", "distributed"]
 
 [tool.poetry.dependencies]
-python = ">=3.8"
+python = ">=3.9"
 distributed = { version = "^2023.9.1", python = ">=3.9", optional = true }
 kubernetes = {version = "^27.2.0", optional = true }
 typing-extensions = "^4.4.0"
@@ -22,7 +22,7 @@ typing-extensions = "^4.4.0"
 icecream = "^2.1.1"
 mypy = "^1.0.0"
 pytest = "^7.2.1"
-ruff = "^0.4.0"
+ruff = "^0.5.0"
 
 [build-system]
 requires = ["poetry-core>=1.0.0"]

diff --git a/docs/src/cli/index.md b/docs/src/cli/index.md
@@ -27,7 +27,7 @@ The WEBKNOSSOS CLI offers many useful commands to work with WEBKNOSSOS datasets:
 
 ### Python 3 with pip from PyPi
 
-- `webknossos` requires at least Python 3.8
+- `webknossos` requires at least Python 3.9
 
 ```bash
 pip install "webknossos[all]"

diff --git a/docs/src/webknossos-py/installation.md b/docs/src/webknossos-py/installation.md
@@ -3,7 +3,7 @@
 [![PyPI version](https://img.shields.io/pypi/v/webknossos)](https://pypi.python.org/pypi/webknossos)
 [![Supported Python Versions](https://img.shields.io/pypi/pyversions/webknossos.svg)](https://pypi.python.org/pypi/webknossos)
 
-The `webknossos` package requires at least Python 3.8.
+The `webknossos` package requires at least Python 3.9.
 
 You can install it from [pypi](https://pypi.org/project/webknossos/), e.g. via pip:
 

diff --git a/webknossos/Changelog.md b/webknossos/Changelog.md
@@ -13,6 +13,7 @@ For upgrade instructions, please check the respective _Breaking Changes_ section
 [Commits](https://github.com/scalableminds/webknossos-libs/compare/v0.14.26...HEAD)
 
 ### Breaking Changes
+- Python version 3.8 is no longer officially supported. [#1068](https://github.com/scalableminds/webknossos-libs/pull/1068)
 
 ### Added
 - Added example for scaling a skeleton. [#1147](https://github.com/scalableminds/webknossos-libs/pull/1147)
@@ -21,6 +22,7 @@ For upgrade instructions, please check the respective _Breaking Changes_ section
 - Added options `--layer-name` and `--mag` for compress command of the CLI. [#1141](https://github.com/scalableminds/webknossos-libs/pull/1141)
 - Added options `--chunk-shape` and `--chunks-per-shard` for convert command of the CLI. [#1150](https://github.com/scalableminds/webknossos-libs/pull/1150)
 - The `from_images` method of the `Dataset` supports directories and single files as `input_path` now. [#1152](https://github.com/scalableminds/webknossos-libs/pull/1152)
+- Added support for python version 3.12. [#1068](https://github.com/scalableminds/webknossos-libs/pull/1068)
 - The number of pixel limit for JPG conversion is disabled now. [#1154](https://github.com/scalableminds/webknossos-libs/pull/1154)
 - Added option `--batch-size` to the convert command of the CLI. [#1158](https://github.com/scalableminds/webknossos-libs/pull/1158)
 

diff --git a/webknossos/poetry.lock b/webknossos/poetry.lock
diff --git a/webknossos/pyproject.toml b/webknossos/pyproject.toml
@@ -28,39 +28,39 @@ classifiers = [
 include = ["webknossos/version.py"]  # included explicitly since it's in .gitignore
 
 [tool.poetry.dependencies]
-python = ">=3.8,<3.12"
+python = ">=3.9,<3.13"
 aiohttp = "^3.8.1"
 attrs = ">=22.0.0"
-boltons = "~21.0.0"
+boltons = ">=21.0.0"
 cattrs = ">=22.0.0"
 cluster_tools = { path = "../cluster_tools/", develop = true }
-fsspec = "^2023.9.0"
-httpx = ">=0.20.0,<=0.24.1"
+fsspec = "^2024.6.0"
+httpx = "^0.27.0"
 loxun = "^2.0"
-natsort = "^6.2.0"
-networkx = "^2.6.2"
-numcodecs = "^0.11"
-numpy = "^1.21.0"  # see https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table
-pims = "^0.6.0"
-psutil = "^5.6.7"
+natsort = "^8.4.0"
+networkx = "^3"
+numcodecs = "^0.12"
+numpy = "^2"
+pims = "^0.7.0"
+psutil = "^6"
 python-dateutil = "^2.8.0"
-python-dotenv = "^0.19.0"
-rich = "^10.9.0"
+python-dotenv = "^1"
+rich = "^13"
 scipy = "^1.9.0"
-typer = "0.12.3"
+typer = "^0.12"
 typing-extensions = "^4.0"
-universal-pathlib = "0.1.3"
-wkw = "1.1.22"
-zarr = "^2.16.0"
-zarrita = "0.2.2"
+universal-pathlib = "^0.2"
+wkw = "1.1.24"
+zarr = "^2.18.0"
+zarrita = "0.2.5"
 zipp = "^3.5.0"
 
 # A list of all of the optional dependencies, some of which are included in the
 # below `extras`. They can be opted into by apps.
 imagecodecs = { version = ">=2021.11.20", optional = true }
 JPype1 = { version = "^1.3.0", optional = true }
 tifffile = { version = ">=2021.11.2", optional = true }
-pylibCZIrw = { version = "3.5.2", source = "scm", optional = true }
+pylibCZIrw = { version = "4.1.1", source = "scm", optional = true}
 
 [tool.poetry.extras]
 tifffile = ["tifffile"]
@@ -78,29 +78,29 @@ hypothesis = "^6.35.0"
 icecream = "^2.1.1"
 inducoapi = "^2.0.2"
 jsonschema = "^4.6.0"
-mypy = "^1.6.0"
-pytest = "^7.2.1"
+mypy = "^1.10.0"
+pytest = "^8"
 pytest-custom-exit-code = "^0.3.0"
 pytest-recording = "^0.13.0"
-pytest-split = "^0.8.0"
-pytest-sugar = "^0.9.4"
-pytest-timeout = "^2.1.0"
-pytest-timestamper = "^0.0.9"
-types-python-dateutil = "^0.1.6"
-ruff = "^0.4.0"
+pytest-split = "^0.9.0"
+pytest-sugar = "^1"
+pytest-timeout = "^2.3.0"
+pytest-timestamper = "^0.0.10"
+types-python-dateutil = "^2.9"
+ruff = "^0.5"
 
 [tool.poetry.group.examples]
 optional = true
 
 [tool.poetry.group.examples.dependencies]
 # packages for examples:
-fastremap = "^1.13.3"
-pandas = "^1.3.4"
+fastremap = "^1.15.0"
+pandas = "^2"
 pooch = "^1.5.2"
-s3fs = "^2023.9.0"
-tabulate = "^0.8.9"
+s3fs = ">=2023.9.0"
+tabulate = "^0.9"
 scikit-learn = "^1.0.1"
-scikit-image = "^0.21.0"
+scikit-image = "^0.24.0"
 
 [[tool.poetry.source]]
 name = "PyPI"

diff --git a/webknossos/stubs/upath/__init__.py b/webknossos/stubs/upath/__init__.py
diff --git a/...s/tests/cassettes/test_annotation/test_annotation_from_url[61c20205010000cc004a6356].yaml b/...s/tests/cassettes/test_annotation/test_annotation_from_url[61c20205010000cc004a6356].yaml
@@ -11,16 +11,16 @@ interactions:
       host:
       - webknossos.org
       user-agent:
-      - python-httpx/0.24.1
+      - python-httpx/0.27.0
     method: GET
     uri: https://webknossos.org/api/v7/annotations/61c20205010000cc004a6356/download?skipVolumeData=true
   response:
     content:
       zip:
         l4dense_motta_et_al_demo_v2__explorational__potto__4a6356.nml: "<things>\n
           \ <meta name=\"writer\" content=\"NmlWriter.scala\"/>\n  <meta name=\"writerGitCommit\"
-          content=\"e8d5ea3dd097af5a4a7a3bb834d9da56d072ba4b\"/>\n  <meta name=\"timestamp\"
-          content=\"1721042692224\"/>\n  <meta name=\"annotationId\" content=\"61c20205010000cc004a6356\"/>\n
+          content=\"1333f7e57e93b9f906ec4ed8d9c1ec4b5d70aeeb\"/>\n  <meta name=\"timestamp\"
+          content=\"1722350945254\"/>\n  <meta name=\"annotationId\" content=\"61c20205010000cc004a6356\"/>\n
           \ <meta name=\"username\" content=\"Philipp Otto\"/>\n  <parameters>\n    <experiment
           name=\"l4dense_motta_et_al_demo_v2\" organization=\"scalable_minds\" description=\"\"
           wkUrl=\"https://webknossos.org\"/>\n    <scale x=\"11.239999771118164\"
@@ -53,7 +53,7 @@ interactions:
       content-disposition:
       - attachment;filename="l4dense_motta_et_al_demo_v2__explorational__potto__4a6356.zip"
       content-length:
-      - '1063'
+      - '1061'
       content-type:
       - application/zip
       date: Mon, 01 Jan 2000 00:00:00 GMT
@@ -77,7 +77,7 @@ interactions:
       host:
       - webknossos.org
       user-agent:
-      - python-httpx/0.24.1
+      - python-httpx/0.27.0
     method: GET
     uri: https://webknossos.org/api/v7/datasets/scalable_minds/l4dense_motta_et_al_demo_v2
   response: