Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support new topologySpread scheduling constraints #852

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

jmdeal
Copy link
Member

@jmdeal jmdeal commented Dec 8, 2023

Fixes #430

Description
This PR adds support for the following topology spread constraint fields:

  • matchLabelKeys
  • nodeAffinityPolicy
  • nodeTaintsPolicy

How was this change tested?
make test

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 8, 2023
@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Dec 8, 2023
@jmdeal jmdeal changed the title feat: support new topologySpread scheduling constraints [WIP] feat: support new topologySpread scheduling constraints Dec 8, 2023
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 8, 2023
@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from aed9f49 to 54af319 Compare December 8, 2023 19:34
@coveralls
Copy link

coveralls commented Dec 8, 2023

Pull Request Test Coverage Report for Build 13139823506

Details

  • 128 of 132 (96.97%) changed or added relevant lines in 10 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.08%) to 81.331%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pkg/controllers/provisioning/provisioner.go 13 15 86.67%
pkg/controllers/provisioning/scheduling/topology.go 28 30 93.33%
Files with Coverage Reduction New Missed Lines %
pkg/utils/termination/termination.go 2 92.31%
pkg/test/expectations/expectations.go 2 94.81%
Totals Coverage Status
Change from base Build 13127485405: 0.08%
Covered Lines: 9175
Relevant Lines: 11281

💛 - Coveralls

@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch 7 times, most recently from 4329131 to f0127f2 Compare December 9, 2023 01:46
@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch 3 times, most recently from b23a605 to 0408162 Compare December 11, 2023 21:59
@jmdeal jmdeal changed the title [WIP] feat: support new topologySpread scheduling constraints feat: support new topologySpread scheduling constraints Dec 11, 2023
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 11, 2023
@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from 0408162 to 6673b5b Compare December 11, 2023 23:14
@jmdeal jmdeal changed the title feat: support new topologySpread scheduling constraints [WIP] feat: support new topologySpread scheduling constraints Dec 13, 2023
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 13, 2023
@jmdeal
Copy link
Member Author

jmdeal commented Dec 13, 2023

Holding to include matchLabelKeys for pod affinity with k8s v1.29.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 14, 2023
@jmdeal
Copy link
Member Author

jmdeal commented Jun 13, 2024

/remove blocked

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 29, 2024
@abrazouski
Copy link

Sorry @jmdeal for bothering, but any plans for moving forward with this?
We're really rely on this feature

@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from dae9cfd to 1d46f34 Compare August 15, 2024 23:17
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 15, 2024
@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from 1d46f34 to 7b3827f Compare August 28, 2024 17:01
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 15, 2024
@hamishforbes
Copy link

Hi @jmdeal is there anything blocking this or any way I can help to move this on (testing a custom build or something)?

@engedaam
Copy link
Contributor

engedaam commented Dec 12, 2024

/assign @njtran

@k8s-ci-robot
Copy link
Contributor

@engedaam: GitHub didn't allow me to assign the following users: njtarn.

Note that only kubernetes-sigs members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign @njtarn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@engedaam
Copy link
Contributor

/assign @njtran

@jonathan-innis
Copy link
Member

/unassign njtran

@jonathan-innis
Copy link
Member

/assign jonathan-innis

@jonathan-innis
Copy link
Member

Ran this comparison a few times, this was the set of results with the largest difference. Marginally faster overall surprisingly, with a slight decrease in first round scheduled pods (though I noticed this stat varied wildly between runs).

And I think I figured out why! Turns out that we really haven't been doing our scheduling benchmarking correctly in a while! See #1930

@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from 5958693 to 01de6ac Compare January 31, 2025 23:14
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jmdeal
Once this PR has been reviewed and has the lgtm label, please ask for approval from jonathan-innis. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 31, 2025
for domain := range domains {
domainCounts[domain] = 0
}
func NewTopologyGroup(topologyType TopologyType, topologyKey string, pod *v1.Pod, namespaces sets.Set[string], labelSelector *metav1.LabelSelector, maxSkew int32, minDomains *int32, taintPolicy *v1.NodeInclusionPolicy, affinityPolicy *v1.NodeInclusionPolicy, domainGroup TopologyDomainGroup) *TopologyGroup {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func NewTopologyGroup(topologyType TopologyType, topologyKey string, pod *v1.Pod, namespaces sets.Set[string], labelSelector *metav1.LabelSelector, maxSkew int32, minDomains *int32, taintPolicy *v1.NodeInclusionPolicy, affinityPolicy *v1.NodeInclusionPolicy, domainGroup TopologyDomainGroup) *TopologyGroup {
func NewTopologyGroup(topologyType TopologyType, topologyKey string, pod *v1.Pod, namespaces sets.Set[string], labelSelector *metav1.LabelSelector, maxSkew int32, minDomains *int32, taintPolicy, affinityPolicy *v1.NodeInclusionPolicy, domainGroup TopologyDomainGroup) *TopologyGroup {

the tiniest nit known to man

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reformatted to put each argument on it's own line, since this is beginning to wrap off of my screen. Given that, I think leaving both types in is more readable, but let me know what you think. Personally, I prefer always including the types as I find it a little more readable, but I don't feel to strongly.

pkg/utils/pod/scheduling.go Outdated Show resolved Hide resolved
pkg/controllers/provisioning/scheduling/topologygroup.go Outdated Show resolved Hide resolved
@jonathan-innis
Copy link
Member

First pass -- I didn't look at everything and have some starting questions

@rschalo
Copy link
Contributor

rschalo commented Feb 3, 2025

/remove-label blocked

@k8s-ci-robot
Copy link
Contributor

@rschalo: The label(s) /remove-label blocked cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor, ci-short, ci-extended, ci-full. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/remove-label blocked

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@jmdeal jmdeal force-pushed the support-affinity-and-taint-policy branch from 09e5867 to a00643d Compare February 4, 2025 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Unable to make progress due to some dependency cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support new topologySpread scheduling constraints