Explore and Create a PoC PR for migrating our CI to use Data Platform workflows

[NohaIhab]

Context

Based on the conclusions made in canonical/bundle-kubeflow#1070 and the limitations documented there, we need to further explore migrating our CI to use workflows from https://github.com/canonical/data-platform-workflows.
This is an exploration task that will later be used to write a spec on the new design of the CI.

What needs to get done

• Create a PR that uses data-platform-workflows
• Identify and explore the changes to the current CI and design decisions we need to make to integrate the workflow in the kfp-operators repo's CI

Definition of Done

a PoC PR is created and what we need to change to migrate to the new CI are documented in the issue. Note that the How we will make these changes i.e. the implementation specification will be covered in task [TODO: link task] for writing the spec.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6684.

This message was autogenerated

[DnPlas]

The steps for solving this would be to:

Individual integration tests

1. Create a common workflow for getting paths and building charms with cache (ci: create a new workflow for getting charm paths and building them with cache charmed-kubeflow-workflows#88)
2. Modify each of the integration tests in kfp-operators repository to use the .charm artefacts generated by the build_charm.yaml workflow (from 1) (ci, tests: pass charm artefacts to deploy and test charms #640)
3. Modify the kfp-operators CI slightly to download the .charm artefacts so they can be used in the integration tests.

Bundle integration tests

1. Modify each of the integration tests in kfp-operators repository to use the .charm artefacts generated by the build_charm.yaml workflow
2. Modify the kfp-operators CI slightly to download the .charm artefacts so they can be used in the integration tests.

Publish job

1. Pass the charm artefacts so they can be re-uploaded to charmhub