feat: The Stackable scaler now waits 5s after TrinoClusters turn ready

[sbernauer]

This seems to work well in combination with stackabletech/trino-operator#715

[Techassi]

Overall lgtm, a few small comments.

[Techassi]

note: These changes seem unrelated to the changes introduced by this PR. Should these instead be introduced by a separate PR?

[sbernauer]

While playing around with restarting coordinator and workers and seeing the effect in trino-lb, I noticed that the works don't shut down gracefully: https://docs.stackable.tech/home/stable/trino/usage-guide/operations/graceful-shutdown/#_authorization_requirements
To get the graceful shutdown to work we need to add OPA authorization.

If you want to I can put it in a separate PR, shall I?

[Techassi]

note: This seems to be formatted weirdly. Can we do anything about that?

[sbernauer]

I noticed this during development as well.

This formats wrong

            let last_transition_time: Time = serde_json::from_value(last_transition_time.clone())
                .with_context(|_| ParseLastTransitionTimeSnafu {
                last_transition_time: last_transition_time.clone(),
                cluster: &cluster.name,
                namespace: &cluster.namespace,
            })?;

The following formats correctly, but does not compile (type annotations needed)

            let last_transition_time = serde_json::from_value(last_transition_time.clone())
                .with_context(|_| ParseLastTransitionTimeSnafu {
                    last_transition_time: last_transition_time.clone(),
                    cluster: &cluster.name,
                    namespace: &cluster.namespace,
                })?;

This works

            let last_transition_time = serde_json::from_value::<Time>(last_transition_time.clone())
                .with_context(|_| ParseLastTransitionTimeSnafu {
                    last_transition_time: last_transition_time.clone(),
                    cluster: &cluster.name,
                    namespace: &cluster.namespace,
                })?;

=> Looks like a bug in rustfmt to me. I let you decide, I personally am fine as-is, a future rustfmt version will probably format it correctly 😅

[Techassi]

note(non-blocking): (To self) There seem to be many hoops you have to jump through to properly use the status of a resource. Can we improve this situation with better API design?

[sbernauer]

Does

trino-lb/trino-lb/src/scaling/stackable.rs

Lines 278 to 281 in ff4f5a2

	// I would prefer switching to using https://docs.rs/k8s-openapi/latest/k8s_openapi/apimachinery/pkg/apis/meta/v1/struct.Condition.html
	// for parsing. Sadly the Stackable Condition is not compatible with the k8s-openapi Condition struct, so I
	// created https://github.com/stackabletech/issues/issues/489. The branch refactor/conditions-parsing has an
	// (non-working) example code, which can be used after the Issues is resolved

explain the situation a bit?