Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use docker to pin dependencies #829

Merged
merged 8 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 28 additions & 26 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
name: Continuous Integration

on: [pull_request]

jobs:
Expand All @@ -14,8 +13,8 @@ jobs:
with:
python-version: '3.10'
architecture: 'x64'
- name: Install yapf
run: pip install -r dev_tools/requirements/format.env.txt
- name: Install requirements
run: pip install -r dev_tools/requirements/envs/format.env.txt
- name: Format
run: check/format-incremental
mypy:
Expand All @@ -27,11 +26,11 @@ jobs:
with:
python-version: '3.10'
architecture: 'x64'
- name: Install mypy
run: pip install -r dev_tools/requirements/mypy.env.txt
- name: Install requirements
run: pip install -r dev_tools/requirements/envs/mypy.env.txt
- name: Type check
run: check/mypy
lint:
pylint:
name: Lint check
runs-on: ubuntu-latest
steps:
Expand All @@ -40,54 +39,58 @@ jobs:
with:
python-version: '3.10'
architecture: 'x64'
- name: Install pylint
run: pip install -r dev_tools/requirements/pylint.env.txt
- name: Install requirements
run: pip install -r dev_tools/requirements/envs/pylint.env.txt
- name: Lint
run: check/pylint
pytest-max-compat:
name: Pytest max compat
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install requirements
run: |
pip install -r dev_tools/requirements/max_compat/pytest-max-compat.env.txt
- name: Pytest check
run: check/pytest
shell: bash
pytest:
name: Pytest
runs-on: ${{ matrix.os }}
strategy:
matrix:
# On each operating system, check latest version of python and cirq
os: [ ubuntu-latest, macos-latest, windows-latest ]
python-version: [ '3.10' ]
cirq-version: [ '~=1.0' ]
# Also check least-supported versions (linux only)
include:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the minimum python version we support now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of abusing the strategy matrix, I moved the "least supported" test to a new job pytest-minimum. There, I bumped the minimum python version for shady reasons:

  • The "minimum" environment uses cirq 0.15, which supports python 3.8 so we could run the job on python 3.8
  • however, the way the job works is first you install the full, modern, pinned environment which requires python 3.9 because cirq 1.2 requires python 3.9
  • After the full environment, we install the older version of cirq. But that step would fail with the old python

The "correct" fix would be to pin a minimal environment. The correct way to do this would be to have another Dockerfile that uses python 3.8

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have unbumped the minimum python version by doing a correct fix

- os: ubuntu-latest
python-version: 3.8
cirq-version: '~=0.15.0'
fail-fast: false
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
python-version: '3.10'
- name: Install requirements
run: |
pip install -r dev_tools/requirements/pytest.env.txt
pip install -r dev_tools/requirements/envs/pytest.env.txt
pip install cirq-core${{matrix.cirq-version}}
- name: Pytest check
run: check/pytest
shell: bash
pytest_resource_estimates:
name: Pytest Resource Estimates
pytest-extra:
name: Pytest extra
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [ '3.10' ]
cirq-version: [ '~=1.0' ]
fail-fast: false
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
python-version: '3.10'
- name: Install requirements
run: |
pip install -r dev_tools/requirements/resource_estimates.env.txt
pip install -r dev_tools/requirements/envs/pytest-extra.env.txt
pip install cirq-core${{matrix.cirq-version}}
- name: Pytest check resources
run: check/pytest -m 'not slow' src/openfermion/resource_estimates
Expand All @@ -102,8 +105,7 @@ jobs:
- uses: actions/setup-python@v1
with:
python-version: '3.10'
architecture: 'x64'
- name: Install requirements
run: pip install -r dev_tools/requirements/pytest.env.txt
run: pip install -r dev_tools/requirements/envs/pytest.env.txt
- name: Coverage check
run: check/pytest-and-incremental-coverage
4 changes: 2 additions & 2 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Cirq Pre-release
name: Nightly

on:
schedule:
Expand All @@ -20,7 +20,7 @@ jobs:
python-version: ${{ matrix.python-version }}
- name: Install requirements
run: |
pip install -r dev_tools/requirements/pytest.env.txt
pip install -r dev_tools/requirements/envs/pytest.env.txt
pip install -U cirq-core --pre
- name: Pytest check
run: check/pytest
Expand Down
29 changes: 29 additions & 0 deletions dev_tools/requirements/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

ARG PYTHON_VERSION=3.10
FROM python:${PYTHON_VERSION}

WORKDIR /pip-compile

# Step 0: install pip-tools
COPY envs/pip-tools.env.txt ./
RUN pip install -r pip-tools.env.txt

# Step 1: compile a complete & consistent environment with all dependencies
COPY deps/ ./deps/
COPY run-pip-compiles.py ./

ARG PLATFORM="default"
RUN python run-pip-compiles.py --platform $PLATFORM
7 changes: 0 additions & 7 deletions dev_tools/requirements/deps/dev-tools.txt

This file was deleted.

3 changes: 3 additions & 0 deletions dev_tools/requirements/deps/oldest-versions.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Used as a constraint file for pytest-max-compat
cirq-core~=1.0.0
cirq-google~=1.0.0
1 change: 1 addition & 0 deletions dev_tools/requirements/deps/pip-tools.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pip-tools>=7.0
Loading
Loading