[Feature] GitHub Actions CI/CD Enablement and Expansion

[markurtz]

Summary

This feature focuses on finalizing and expanding the GitHub Actions CI/CD pipelines to automate builds, testing, releases, and quality assurance for the GuideLLM repository. Success is defined by seamless automated testing and deployment processes across all branches, ensuring high code quality, full test coverage, and timely release management.

Scope and Constraints

Scope

• Implement comprehensive GitHub Actions workflows for the following:
  • Development flows based on dev branch PRs or main branch PRs
  • Development flows for main branch commits
  • Nightly and weekly flows
  • Release flows for staging and production
• Automate tests at all levels: unit, integration, and end-to-end.
• Publish nightly, dev, and release candidate builds to the internal package store and public PyPi repository.
• Ensure comprehensive logging and reporting across workflows.

User Stories

N/A

Requirements and Acceptance Criteria

Workflows

Development Branch Workflow (development.yml)

• Conditions:
  • Runs on PR open or PR commit for any non-draft PR on any branch
• Jobs:
  • Run quality checks on {min - max} supported Python versions on ubuntu-latest: tox {quality, types, pre-commit}
  • Run unit tests on {min, max} supported Python versions on ubuntu-latest: levels {smoke, sanity}
  • Run integration tests on {min, max} supported Python versions on ubuntu-latest: levels {smoke}
  • Always publish a dev build {wheel, tar} for {min - max} Python versions: neural magic pypi, guidellm-dev package name, #.#.#.{PR#} package version
  • Update PR state with code coverage results comment and pip install instructions for dev package
• Outputs:
  • build.wheel - from publish job
  • build.tar - from publish job
  • pypi.url - from publish job
  • pypi.command - from publish job

Main Branch Workflow (main.yml)

• Conditions:
  • Runs on any commit pushed to the main branch.
• Jobs:
  Same as the development branch workflow
• Outputs:
  • build.wheel - from publish job
  • build.tar - from publish job
  • pypi.url - from publish job
  • pypi.command - from publish job

Nightly Workflow (nightly.yml)

• Conditions:
  • Runs nightly at a defined time using a cron schedule: (cron: 0 0 * * *)
• Jobs:
  • Run unit tests with coverage on {min, max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Run integration tests on {min, max} supported Python versions on ubuntu-latest: {smoke, sanity}
  • Run e2e tests on {min, max} supported Python versions on ubuntu-latest: {smoke}
  • Check publish by ensuring all the tests listed above passed, and there have been changes to the main branch (check since the last nightly tag or no nightly tag exists)
  • If check publish, publish nightly build {wheel, tar} for {min - max} Python versions: public pypi, guidellm-nightly package name, #.#.#.{YYYYMMDD} package version
  • If publish, then tag main with nightly release version: nightly-{YYYYMMDD}
• Outputs:
  • build.wheel - from publish job
  • build.tar - from publish job
  • pypi.url - from publish job
  • pypi.command - from publish job
  • tag - from tag job
  • coverage.unit.PYTHON_VERSION.percentage - from unit tests job
  • coverage.unit.PYTHON_VERSION.report - from unit tests job

Weekly Workflow (weekly.yml)

• Conditions:
  • Runs weekly at a defined time using a cron schedule: (cron: 0 0 * * 0)
• Jobs:
  • Run unit tests with coverage on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Run integration tests with coverage on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Run e2e tests with coverage on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
• Outputs:
  • coverage.integration.PYTHON_VERSION.percentage - from integration tests job
  • coverage.integration.PYTHON_VERSION.report - from integration tests job
  • coverage.e2e.PYTHON_VERSION.percentage - from e2e tests job
  • coverage.e2e.PYTHON_VERSION.report - from e2e tests job

Release Staging/Branch Workflow (release-staging.yml)

• Conditions:
  • Runs on any commit pushed to a release branch / a release branch is created (release/#.#).
• Jobs:
  • Run unit tests on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Run integration tests on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Run e2e tests on {min - max} supported Python versions on ubuntu-latest: levels {smoke, sanity, regression}
  • Always publish a candidate release build {wheel, tar} for {min - max} Python versions: neural magic pypi, guidellm package name, #.#.# package version
• Outputs:
  • build.wheel - from publish job
  • build.tar - from publish job
  • pypi.url - from publish job
  • pypi.command - from publish job

Release Production/Tag Workflow (release-production.yml)

• Conditions:
  • Runs on any release tag creation (v#.#.#).
• Jobs:
  • Always publish a release build {wheel, tar} for {min - max} Python versions: public pypi, guidellm package name, #.#.# package version
  • Update release, if available, with the release builds
  • Run unit tests for the published build on {all} supported Python versions on ubuntu-latest: levels {smoke}
  • Run integration tests for the published build on {all} supported Python versions on ubuntu-latest: levels {smoke}
  • Run e2e tests for the published build on {all} supported Python versions on ubuntu-latest: levels {smoke}
• Outputs:
  • build.wheel - from publish job
  • build.tar - from publish job
  • pypi.url - from publish job
  • pypi.command - from publish job

Testing & Documentation

Testing Approaches:

Manual testing to ensure jobs are working as expected as opportunities to do so come up. We must wait until the next release process for some of the flows.

Documentation Requirements:

• Update the README.md with CI/CD status badges.
• Create new sections in the documentation covering the following:
  • CI/CD workflows and how they are triggered.
  • Release process and expected conventions.
  • Guidelines for contributing and running tests locally.