🤖 Fluence Reboot! Questions / Features discussed

[vsoch]

This is a small list of features / notes we discussed that we want to pick up on after the refactor is finished. Please feel free to add to this list - I didn't properly capture the discussion from yesterday.

• Question: Is there metadata coming in to fluence that we aren't using (e.g., from custom scheduler)?
• Question: What other extension interfaces / plugins are active when fluence is added with a KubeSchedulerConfiguration. See the picture at the top here: https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/#interfaces. I think others might already understand this, but I want to get a cluster running and see (however much I can) for myself what is happening (and then think about how those things work together, etc).
• Fluence might have different flavors 🍨 of JobSpec. E.g., as a user I want to be able to add some metadata (label or annotation, likely annotation because labels are more limited in verbosity) to my pod and ask for the nodes to be closer together (or otherwise have a named topology).
• When fluxion supports resource graph growth, we should support that. When we change number of pods in our jobs we want to make sure that scheduling decisions continue to be what is desired.
• Better understand (and factor into experiments) how TopologySpreadConstraint is relevant https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/

Desired:

• Separate rendered docs
• Shared image builds (between the automated build/deploy and testing pipelines for GitHub actions)
• review current cleanup (saving and restoring cache) not sure is adequate!

[vsoch]

This will need further review and discussion after #69 is merged. We likely will split these into separate issues.