Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] [GHA] Introduce additional Python (3.9-3.12) API tests on Windows #27304

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
2e60f85
build additional wheels on win
akashchi Oct 29, 2024
462a17c
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Oct 29, 2024
95febdf
get py version
akashchi Oct 29, 2024
298d251
expand correctly
akashchi Oct 29, 2024
85bbeea
use the py launcher
akashchi Oct 29, 2024
7f81439
use pybuilddir
akashchi Oct 29, 2024
6ec23b0
add missing dependencies
akashchi Oct 30, 2024
ff48a8c
Update .github/workflows/job_build_windows.yml
akashchi Oct 30, 2024
fc4e692
merge
akashchi Oct 30, 2024
836ccad
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Oct 30, 2024
c96669a
collect wheels from install dir
akashchi Oct 30, 2024
cfdb4d2
correct path
akashchi Oct 30, 2024
94e0349
use install wheels action
akashchi Oct 31, 2024
ef39455
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Oct 31, 2024
797a22c
skip to see other stages
akashchi Oct 31, 2024
16c6ea0
do not build for 3.11
akashchi Oct 31, 2024
1938717
extract api tests
akashchi Nov 1, 2024
ec3be60
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 1, 2024
ebcefcb
check dirs, do not fail-fast API tests
akashchi Nov 1, 2024
89e588a
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 1, 2024
a570fe9
rm unused
akashchi Nov 4, 2024
db3eb0b
set PATH too
akashchi Nov 4, 2024
88e82c5
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 4, 2024
e62830d
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 5, 2024
4fefe20
add missing dir
akashchi Nov 5, 2024
fd949d3
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 7, 2024
a61ff4e
correct name
akashchi Nov 7, 2024
f5bc263
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 8, 2024
d208cac
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 11, 2024
8ac65d4
rm installation of the dev wheell
akashchi Nov 11, 2024
64b811c
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 12, 2024
320187d
use simpler inputs for install action
akashchi Nov 12, 2024
fa5e1af
try with index and cache
akashchi Nov 12, 2024
0fa0034
revert the simplification
akashchi Nov 13, 2024
895af7a
Merge remote-tracking branch 'upstream/master' into ci/gha/python-api…
akashchi Nov 13, 2024
4afb060
use simpler inputs again
akashchi Nov 13, 2024
8a39a78
fix name
akashchi Nov 13, 2024
1a92d95
correct tokenizers wheel name
akashchi Nov 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/actions/install_ov_wheels/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: 'Find and install OpenVINO Python wheels'
description: 'Finds the OpenVINO Python wheels (core, dev and tokenizers) suitable for the "python3" executable and installs them'
inputs:
wheels-dir-path:
description: 'Path to the directory in which wheels are located'
required: true
install-core-wheel:
description: 'Whether to install the OpenVINO Core wheel (openvino-*.whl)'
required: true
install-dev-wheel:
description: 'Whether to install the OpenVINO Dev wheel (openvino_dev*.whl)'
required: true
extras-to-install:
description: 'Extras to install with the OpenVINO Dev wheel (openvino_dev.whl[caffe,kaldi,onnx,tensorflow2,pytorch])'
required: false
default: ''
install-tokenizers-wheel:
description: 'Whether to install the OpenVINO Dev wheel (openvino_tokenizes*.whl)'
required: true
runs:
using: 'composite'
steps:
- name: Install OpenVINO Python wheels (Windows)
shell: pwsh
if: runner.os == 'Windows'
run: |
if ( "${{ inputs.install-core-wheel }}" -eq "true" )
{
$pyVersion = python3 -c "import sys; print(f'{sys.version_info.major}{sys.version_info.minor}')"
$ovCoreWheelPath = Get-ChildItem -Path ${{ inputs.wheels-dir-path }} -Filter "openvino-*cp$pyVersion*.whl" | % { $_.FullName }
python3 -m pip install $ovCoreWheelPath
}

if ( "${{ inputs.install-tokenizers-wheel }}" -eq "true" )
{
$ovTokenizersWheelPath = Get-ChildItem -Path ${{ inputs.wheels-dir-path }} -Filter openvino_tokenizers-*.whl | % { $_.FullName }
python3 -m pip install "$ovTokenizersWheelPath"
}

if ( "${{ inputs.install-core-wheel }}" -eq "true" )
{
$ovDevWheelPath = Get-ChildItem -Path "${{ inputs.wheels-dir-path }}" -Filter openvino_dev*.whl | % { $_.FullName }
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
if ( "${{ inputs.extras-to-install }}" )
{
python3 -m pip install "$ovDevWheelPath[${{ inputs.extras-to-install }}]"
}
else
{
python3 -m pip install "$ovDevWheelPath"
}
}
- name: Install OpenVINO Python wheels (Linux and macOS)
shell: bash
if: runner.os != 'Windows'
run: |
if [[ "${{ inputs.install-core-wheel }}" == "true" ]]; then
py_version=$(python3 -c "import sys; print(f'{sys.version_info.major}{sys.version_info.minor}')")
ov_wheel_path=$(find ${{ inputs.wheels-dir-path }} -name "openvino-*cp$py_version*.whl")
python3 -m pip install $ov_wheel_path
fi

if [[ "${{ inputs.install-tokenizers-wheel }}" == "true" ]]; then
python3 -m pip install ${{ inputs.wheels-dir-path }}/openvino_tokenizers-*.whl
fi

if [[ "${{ inputs.install-dev-wheel }}" == "true" ]]; then
ov_dev_wheel_path=$(find ${{ inputs.wheels-dir-path }} -name 'openvino_dev*.whl')
if [[ "${{ inputs.extras-to-install }}" ]]; then
python3 -m pip install $ov_dev_wheel_path[${{ inputs.extras-to-install }}]
else
python3 -m pip install $ov_dev_wheel_path
fi
fi
58 changes: 55 additions & 3 deletions .github/workflows/job_build_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@ on:
description: 'A string of options passed to CMake'
type: string
required: true
build-additional-python-wheels:
description: 'Whether to build additional, i.e., non-system Python wheels. Should have Python 3.9-3.12 installed'
type: boolean
required: false
default: false

permissions: read-all

Expand Down Expand Up @@ -157,8 +162,7 @@ jobs:
run: echo SSL_CERT_FILE=$(python3 -m certifi) >> $env:GITHUB_ENV

- name: CMake configure
run: |
cmake -S ${{ env.OPENVINO_REPO }} -B ${{ env.BUILD_DIR }} ${{ inputs.cmake-options }}
run: cmake -S ${{ env.OPENVINO_REPO }} -B ${{ env.BUILD_DIR }} ${{ inputs.cmake-options }}

- name: Clean ccache stats
run: '& ccache --zero-stats'
Expand All @@ -176,6 +180,54 @@ jobs:
cmake --install . --config ${{ env.CMAKE_BUILD_TYPE }} --prefix ${{ env.INSTALL_TEST_DIR }} --component tests
working-directory: ${{ env.BUILD_DIR }}

# Setup additional Python versions for wheels building
- name: Setup Python 3.9
mryzhov marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ inputs.build-additional-python-wheels }}
uses: ./openvino/.github/actions/setup_python
with:
version: '3.9'
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'

# Setup additional Python versions for wheels building
- name: Setup Python 3.10
if: ${{ inputs.build-additional-python-wheels }}
uses: ./openvino/.github/actions/setup_python
with:
version: '3.10'
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'

# Setup additional Python versions for wheels building
- name: Setup Python 3.12
if: ${{ inputs.build-additional-python-wheels }}
uses: ./openvino/.github/actions/setup_python
with:
version: '3.12'
pip-cache-path: ${{ env.PIP_CACHE_PATH }}
should-setup-pip-paths: 'true'
self-hosted-runner: 'true'

- name: Build additional Python wheels
if: ${{ inputs.build-additional-python-wheels }}
run: |
$pyVersions = '3.9', '3.10', '3.12'
foreach ($pyVersion in $pyVersions) {
$pyBuildDir = "${{ github.workspace }}/py$pyVersion"
New-Item -ItemType Directory -Path "$pyBuildDir" -Force

$pythonCommand = "py -$pyVersion -c `"import sys; print(f'{sys.executable}')`""
$pythonExecutablePath = & cmd /c $pythonCommand

& $pythonExecutablePath -m pip install -r ${{ env.OPENVINO_REPO }}/src/bindings/python/wheel/requirements-dev.txt

cmake -DPython3_EXECUTABLE="$pythonExecutablePath" -DOpenVINODeveloperPackage_DIR=${{ env.BUILD_DIR }} -S ${{ env.OPENVINO_REPO }}/src/bindings/python -B "$pyBuildDir"
cmake --build "$pyBuildDir" --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
cmake --install "$pyBuildDir" --config ${{ env.CMAKE_BUILD_TYPE }} --prefix ${{ env.INSTALL_WHEELS_DIR }} --component python_wheels
}

- name: Pack Artifacts
run: |
$file = Get-ChildItem -Path "${{ env.INSTALL_DIR }}"
Expand Down Expand Up @@ -220,7 +272,7 @@ jobs:
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: openvino_wheels
path: ${{ env.BUILD_DIR }}/wheels/*.whl
path: ${{ env.INSTALL_WHEELS_DIR }}/wheels/*.whl
if-no-files-found: 'error'

- name: Upload openvino tests package
Expand Down
25 changes: 10 additions & 15 deletions .github/workflows/job_pytorch_layer_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ jobs:
echo "OPENVINO_REPO=$GITHUB_WORKSPACE/openvino" >> "$GITHUB_ENV"
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "INSTALL_WHEELS_DIR=$GITHUB_WORKSPACE/install/wheels" >> "$GITHUB_ENV"
echo "LAYER_TESTS_INSTALL_DIR=$GITHUB_WORKSPACE/install/tests/layer_tests" >> "$GITHUB_ENV"

- name: Install OpenVINO dependencies (mac)
Expand All @@ -82,11 +83,12 @@ jobs:
Expand-Archive openvino_tests.zip -DestinationPath ${{ env.INSTALL_DIR }}
working-directory: ${{ env.INSTALL_DIR }}

- name: Fetch setup_python action
- name: Fetch setup_python and install wheels actions
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
.github/actions/install_ov_wheels/action.yml
sparse-checkout-cone-mode: false
path: 'openvino'

Expand All @@ -98,20 +100,13 @@ jobs:
should-setup-pip-paths: ${{ runner.os == 'Linux' }}
self-hosted-runner: ${{ runner.os == 'Linux' }}

- name: Install OpenVINO Python wheels (Linux and macOS)
if: runner.os != 'Windows'
run: |
# Install the core OV wheel
python3 -m pip install ./openvino-*.whl
working-directory: ${{ env.INSTALL_WHEELS_DIR }}

- name: Install OpenVINO Python wheels (Windows)
if: runner.os == 'Windows'
run: |
# Find and install the core OV wheel
$ovCoreWheelPath=Get-ChildItem -Path . -Filter openvino-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"
working-directory: ${{ env.INSTALL_WHEELS_DIR }}
- name: Install OpenVINO Python wheels
uses: ./openvino/.github/actions/install_ov_wheels
with:
wheels-dir-path: ${{ env.INSTALL_WHEELS_DIR }}
install-core-wheel: true
install-dev-wheel: false
install-tokenizers-wheel: false

- name: Install Pytorch Layer tests dependencies
run: |
Expand Down
32 changes: 10 additions & 22 deletions .github/workflows/job_tensorflow_layer_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ jobs:
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "INSTALL_TEST_DIR=$GITHUB_WORKSPACE/install/tests" >> "$GITHUB_ENV"
echo "LAYER_TESTS_INSTALL_DIR=$GITHUB_WORKSPACE/install/tests/layer_tests" >> "$GITHUB_ENV"
echo "INSTALL_WHEELS_DIR=$GITHUB_WORKSPACE/install/wheels" >> "$GITHUB_ENV"

- name: Install OpenVINO dependencies (mac)
if: runner.os == 'macOS'
Expand All @@ -82,11 +83,12 @@ jobs:
Expand-Archive openvino_tests.zip -DestinationPath ${{ env.INSTALL_DIR }}
working-directory: ${{ env.INSTALL_DIR }}

- name: Fetch setup_python action
- name: Fetch setup_python and install wheels actions
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
sparse-checkout: |
.github/actions/setup_python/action.yml
.github/actions/install_ov_wheels/action.yml
sparse-checkout-cone-mode: false
path: 'openvino'

Expand All @@ -98,27 +100,13 @@ jobs:
should-setup-pip-paths: ${{ runner.os == 'Linux' }}
self-hosted-runner: ${{ runner.os == 'Linux' }}

- name: Install OpenVINO Python wheels (Linux and macOS)
if: runner.os != 'Windows'
run: |
# Install the core OV wheel
python3 -m pip install ./openvino-*.whl

# Install the core OV Tokenizers wheel
python3 -m pip install ./openvino_tokenizers-*.whl
working-directory: ${{ env.INSTALL_WHEELS_DIR }}

- name: Install OpenVINO Python wheels (Windows)
if: runner.os == 'Windows'
run: |
# Find and install the core OV wheel
$ovCoreWheelPath=Get-ChildItem -Path . -Filter openvino-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"

# Find and install the core OV Tokenizers wheel
$ovCoreWheelPath=Get-ChildItem -Path . -Filter openvino_tokenizers-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"
working-directory: ${{ env.INSTALL_WHEELS_DIR }}
- name: Install OpenVINO Python wheels
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest a bit different approach to find and install wheels, to simplify support in future:

For example how to call this action:
# Install Python benchmark_app by installing openvino-*.whl
- name: Install OpenVINO Python wheels
uses: ./openvino/.github/actions/install_ov_wheels
with:
wheels-dir-path: ${{ env.INSTALL_WHEELS_DIR }}
install-wheels: |
- openvino[extra]
- openvino-dev
- openvino-tokenizers
- openvino-genai

And inside the action we could use the following installation aproach:

pip install openvino[extra] openvino-dev openvino-tokenizers --find-links ${{ env.INSTALL_WHEELS_DIR }} --no-index --no-cache

@ilya-lavrenov , @akladiev what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip will find the suitable weel by himself and do not pickup it from PyPI

Copy link
Contributor Author

@akashchi akashchi Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a problem with this approach: since the idea is to use --no-index so that pip does not try to install any local wheels from PyPI instead, it cannot install the dependencies because it is forbidden from trying to access PyPI like in https://github.com/openvinotoolkit/openvino/actions/runs/11794716928/job/32855012693#step:7:43. The wheels have many dependencies that are hosted on the package index hence we cannot restrict the pip from accessing the index.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, but the 1st comment is still actual, I think it is better to make action more universal (just provide the wheel pattern names as argument)

Copy link
Collaborator

@akladiev akladiev Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, to make sure that a local wheel is installed, internally we pass the expected package version like this pip install openvino-dev==2025.0.0.dev20241113 --find-links.... So there is no need for --no-index. Package version is saved to our manifest file before build starts (field wheel_product_version).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we should proceed with the current approach: using the openvino openvino_tokenizers ... as the input and iterating over the given wheel names for installation.
Additionally, the openvino_tokenizers wheel has a different version for some reason:
image
so we would not be able to use it for local installation. Also, it would be rather cumbersome to pass the version from the build job to other jobs in which the wheels are used.

uses: ./openvino/.github/actions/install_ov_wheels
with:
wheels-dir-path: ${{ env.INSTALL_WHEELS_DIR }}
install-core-wheel: true
install-dev-wheel: false
install-tokenizers-wheel: true

- name: Install Python Layer tests dependencies
run: |
Expand Down
27 changes: 10 additions & 17 deletions .github/workflows/job_tokenizers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,13 +53,15 @@ jobs:
echo "INSTALL_DIR=$GITHUB_WORKSPACE/install" >> "$GITHUB_ENV"
echo "OPENVINO_TOKENIZERS_REPO=$GITHUB_WORKSPACE/openvino_tokenizers" >> "$GITHUB_ENV"
echo "EXTENSION_BUILD_DIR=$GITHUB_WORKSPACE/build" >> "$GITHUB_ENV"
echo "INSTALL_WHEELS_DIR=$GITHUB_WORKSPACE/install/wheels" >> "$GITHUB_ENV"

- name: checkout action
- name: checkout actions
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
sparse-checkout: |
.github/actions/setup_python
.github/actions/cache
.github/actions/install_ov_wheels/action.yml
install_build_dependencies.sh

- name: Install OpenVINO dependencies (mac)
Expand Down Expand Up @@ -92,22 +94,13 @@ jobs:
# Dependencies
#

- name: Install OpenVINO Python wheel (Linux and macOS)
if: runner.os != 'Windows'
run: |
# Find and install wheel
wheel_name=$(find . -name 'openvino-*.whl')
python3 -m pip install $wheel_name
working-directory: ${{ env.INSTALL_WHEELS_DIR }}


- name: Install OpenVINO Python wheel (Windows)
if: runner.os == 'Windows'
run: |
# Find and install wheel
$ovCoreWheelPath=Get-ChildItem -Path . -Filter openvino-*.whl | % { $_.FullName }
python3 -m pip install "$ovCoreWheelPath"
working-directory: ${{ env.INSTALL_WHEELS_DIR }}
- name: Install OpenVINO Python wheels
uses: ./.github/actions/install_ov_wheels
with:
wheels-dir-path: ${{ env.INSTALL_WHEELS_DIR }}
install-core-wheel: true
install-dev-wheel: false
install-tokenizers-wheel: false

#
# Build
Expand Down
Loading
Loading