Referencing a non-existent RunConfiguration in a runtime parameter causes other updates to be ignored and is not reflected in the status

[aidandunlop]

Describe the bug
If a non-existent RunConfiguration is referenced using runConfigurationRef in a runtime parameter, any other changes to the RunConfiguration are ignored, but the RunConfiguration's status is still succeeded. There is no indication in the resource that the RunConfiguration referenced in the resource cannot be found, or that it failed to update other changes in the RunConfiguration.

As an example, if a RunConfiguration is updated to add a runtime parameter pointing to a non-existent RunConfiguration AND a schedule is updated or removed, the schedule will not be updated/removed in the associated Provider.

To Reproduce
Create a Pipeline resource:

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: Pipeline
metadata:
  name: quickstart
  namespace: mlops
spec:
  provider: "vai"
  image: kfp-quickstart:v1
  tfxComponents: base_pipeline.create_components

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: RunConfiguration
metadata:
    name: quickstart-rc
    namespace: mlops
spec:
    run:
      provider: vai
      pipeline: quickstart
    triggers:
      onChange:
        - pipeline
      schedules:
        - cronExpression: "0 0 * * *"

Then update the same RunConfiguration to add a non-existent artifact dependency as a runtimeParameter AND remove the schedules field:

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: RunConfiguration
metadata:
    name: quickstart-rc
    namespace: mlops
spec:
    run:
      provider: vai
      pipeline: quickstart
      runtimeParameters:
      - name: artifact_dependency
        valueFrom:
          runConfigurationRef:
            name: a-different-pipeline
            outputArtifact: my_artifact
    triggers:
      onChange:
        - pipeline

In the Schedules Tab of Vertex AI Pipelines, the schedule will still exist, despite it being removed from the RunConfiguration.

Expected behavior

• Pointing to a non-existent RunConfiguration with runConfigurationRef should either cause the RunConfiguration resource to fail or a warning message should be returned.
• Updating/removing any other part of the RunConfiguration should be updated accordingly even if the runConfigurationRef cannot be found
• Updating/removing any schedules from the RunConfiguration schedules list should remove the associated schedule in the given Provider.

Version and configuration
Using CRD versions v1alpha5 & 6 and KFP Operator version 0.6.0-38-g33a3300

Logs
This is logged when the RunConfiguration cannot be found:

{"level":"info","ts":1737026773.5395613,"logger":"controller.runconfiguration","msg":"object not found","reconciler group":"pipelines.kubeflow.org","reconciler kind":"RunConfiguration","name":"quickstart-rc","namespace":"mlops"}

Additional context
Not sure if this is just relevant for the VAI Provider or whether the KFP Provider is affected too.