diff --git a/keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md b/keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md index 89240c7c4c9..e1c7980892b 100644 --- a/keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md +++ b/keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md @@ -90,7 +90,7 @@ tags, and then generate with `hack/update-toc.sh`. - [Possible misuse](#possible-misuse) - [The update to labels specified at matchLabelKeys isn't supported](#the-update-to-labels-specified-at-matchlabelkeys-isnt-supported) - [Design Details](#design-details) - - [[v1.33] design change and a safe upgrade path](#v133-design-change-and-a-safe-upgrade-path) + - [[v1.34] design change and a safe upgrade path](#v134-design-change-and-a-safe-upgrade-path) - [Test Plan](#test-plan) - [Prerequisite testing updates](#prerequisite-testing-updates) - [Unit tests](#unit-tests) @@ -401,13 +401,13 @@ kube-apiserver modifies the `labelSelector` like the following: In addition, kube-scheduler will handle `matchLabelKeys` within the cluster-level default constraints in the scheduler configuration in the future (see https://github.com/kubernetes/kubernetes/issues/129198). -Finally, the feature will be guarded by a new feature flag. If the feature is +Finally, the feature will be guarded by a new feature flag `MatchLabelKeysInPodTopologySpread`. If the feature is disabled, the field `matchLabelKeys` and corresponding `labelSelector` are preserved if it was already set in the persisted Pod object, otherwise new Pod with the field creation will be rejected by kube-apiserver. Also kube-scheduler will ignore `matchLabelKeys` in the cluster-level default constraints configuration. -### [v1.33] design change and a safe upgrade path +### [v1.34] design change and a safe upgrade path Previously, kube-scheduler just internally handled `matchLabelKeys` before the calculation of scheduling results. But, we changed the implementation design to the current form to make the design align with PodAffinity's `matchLabelKeys`. (See the detailed discussion in [the alternative section](#implement-matchlabelkeys-in-only-either-the-scheduler-plugin-or-kube-apiserver)) @@ -415,11 +415,15 @@ But, we changed the implementation design to the current form to make the design However, this implementation change could break `matchLabelKeys` of unscheduled pods created before the upgrade because kube-apiserver only handles `matchLabelKeys` at pods creation, that is, it doesn't handle `matchLabelKeys` at existing unscheduled pods. -So, for a safe upgrade path from v1.32 to v1.33, kube-scheduler would handle not only `matchLabelKeys` -from the default constraints, but also all incoming pods during v1.33. -We're going to change kube-scheduler to only concern `matchLabelKeys` from the default constraints at v1.34 for efficiency, +So, for a safe upgrade path from v1.33 to v1.34, kube-scheduler would handle not only `matchLabelKeys` +from the default constraints, but also all incoming pods during v1.34. +We're going to change kube-scheduler to only concern `matchLabelKeys` from the default constraints at v1.35 for efficiency, assuming kube-apiserver handles `matchLabelKeys` of all incoming pods. +Also, in case of bugs in this new design, users can disable this feature through a new feature flag, +`MatchLabelKeysInPodTopologySpreadSelectorMerge` (enabled by default). +(See more details in [Feature Enablement and Rollback](#feature-enablement-and-rollback)) + ### Test Plan +- `MatchLabelKeysInPodTopologySpread` feature flag enables the `MatchLabelKeys` feature in `TopologySpreadConstraint`. +- `MatchLabelKeysInPodTopologySpreadSelectorMerge` feature flag enables the new design described at + [[v1.34] design change and a safe upgrade path](#v134-design-change-and-a-safe-upgrade-path). + - If `MatchLabelKeysInPodTopologySpreadSelectorMerge` is disabled while `MatchLabelKeysInPodTopologySpread` is enabled, + Kubernetes handles `MatchLabelKeys` with the classic design, kube-scheduler handles it. + However, that's basically not recommended unless you encounter a bug in a new design behavior. + - This flag cannot be enabled on its own, and has to be enabled together with `MatchLabelKeysInPodTopologySpread`. + Enabling `MatchLabelKeysInPodTopologySpreadSelectorMerge` alone has no effect, and `matchLabelKeys` will be ignored. + +The `MatchLabelKeysInPodTopologySpreadSelectorMerge` feature flag has been added in v1.34 and enabled by default. +This flag can be disabled to revert [the implementation design change in v1.34](#v134-design-change-and-a-safe-upgrade-path) +and go back to the previous behavior in case of bug. + ###### How can this feature be enabled / disabled in a live cluster?