Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GEP-1713: Standard Mechanism to Gateway Merge Listeners (rev 2) #3213

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

dprotaso
Copy link
Contributor

@dprotaso dprotaso commented Jul 23, 2024

This replaces #1863
What type of PR is this?

/kind gep

What this PR does / why we need it:

Outlines a mechanism to merge Gateway Listeners

Which issue(s) this PR fixes:

Fixes #1713

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added kind/gep PRs related to Gateway Enhancement Proposal(GEP) do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jul 23, 2024
Copy link
Contributor

@youngnick youngnick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a few comments here, thanks for keeping working on pushing this forward.

geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
Copy link
Member

@robscott robscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the work on this @dprotaso!

geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
Copy link
Contributor

@youngnick youngnick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM for Provisional (or maybe even Implementable, since we'll need to implement it and try it out) now.

@mlavacca
Copy link
Member

/cc @mlavacca

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 31, 2024
Copy link
Member

@robscott robscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all the work on this @dprotaso! I think this is heading in a good direction, but just lots of details + edge cases to work out still.

geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved

### Gateway Changes

When an implementation supports `ListenerSets` `Gateways` MUST allow the list of listeners to be empty. Thus the present `minItems=1` constraint on the listener list will be removed. This allows implementations to avoid security, cost etc. concerns with having dummy listeners.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a rather scary change to me. Do we need this to start? Ultimately relaxing validation like this means that every Gateway API implementation needs to deal with empty Gateways, even if they're only theoretically useful when combined with ListenerSets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other folks feel pretty strongly about this

see: #3213 (comment)

geps/gep-1713/index.md Outdated Show resolved Hide resolved
geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Show resolved Hide resolved
geps/gep-1713/index.md Outdated Show resolved Hide resolved
The `Gateway` Resource is a point of contention since it is the only place to attach listeners with certificates. We propose a new resource called `ListenerSet` to allow a shared list of listeners to be attached to a single `Gateway`.

## Goals
- Define a mechanism to merge listeners into a single `Gateway`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This GEP is missing guidelines for when Listeners are "compatible" or can safely be merged together, is that meant to be part of this GEP or is it a non-goal?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbh I think that one of the best things about this proposal is that we can let go of the compatible language, since we're being explicit on the fact that Listeners must be distinct across ListenerSets - meaning that the combination of the primary key fields (port, protocol, tls details, and, dependent on protocol, hostname) must be unique.

Practically, to be compatible, Listeners used to have to be distinct anyway, so I think that having the additional complexity of allowing merging has not bought us anything except confusion - quite a few people have asked me about what the difference between "distinct" and "compatible" is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard for me to know what action to take here - it seems like this discussion is beyond this GEP and applies to regular Gateway listeners.

geps/gep-1713/index.md Outdated Show resolved Hide resolved
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dprotaso
Once this PR has been reviewed and has the lgtm label, please assign mlavacca for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 1, 2024
@robscott
Copy link
Member

robscott commented Aug 2, 2024

Status update: We're at the end of the 2-day extension for this GEP to make it into this release, so unfortunately I think this is going to have to switch to targeting v1.3. Fortunately, this new shorter release cycle means that's actually not that far away.

Ultimately, I think you're right. For this to make sense, we'd need some structural changes to Gateway, particularly making Listeners optional. That's a huge change to a GA API (even if it would initially be limited to experimental channel, breaking changes to something labeled as "v1"can be confusing). Combined with the huge scope of the PR, and most of the feedback just coming from Nick and I, I think this particular GEP would benefit from some more soak time for additional reviewers + to smooth out any areas we're uncertain. Unfortunately, I think it's just too big to try to squeeze in today.

With that said, I would like to see this GEP make it in early in the next release cycle (similar to how named Route rules just barely missed the previous cycle but was merged at the beginning of this one).

I think it would be useful to do the following between now and then:

  1. Continue to iterate on this GEP, discussing in meetings as necessary (this can include merging this as provisional with some unresolved sections and iterating in smaller PRs)
  2. Build out a clearer vision for how this can be expanded to cover more use cases in the future (particularly the ones we discussed in the hierarchy doc) - that will help make a stronger case for why this is important
  3. Present these ideas at the SIG-Network review session for v1.2 so we can get early feedback on this overall idea and adjust the proposal accordingly
  4. Warn all controller authors that it's possible that Listeners may become optional in a future release of Gateway API and they should make sure they can handle that case

Thanks again for all the work you've been putting in to push this forward @dprotaso! I really do appreciate it.

type ListenerSetSpec struct {
// ParentRefs references the Gateway that the listeners are attached to.
//
// +kubebuilder:validation:MaxItems=2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why limit ParentRefs to 2?

Copy link
Contributor

@mikemorris mikemorris Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is to allow "moving" a set of listeners from one Gateway to another without downtime, but limiting the ability to attach the same set of listeners to many different Gateways - if we find sufficient demand for that pattern loosening this constraint in the future could be possible, but it feels like a pattern that likely should be avoided.

@k8s-ci-robot
Copy link
Contributor

@dprotaso: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-gateway-api-crds-validation-4 8b338bc link true /test pull-gateway-api-crds-validation-4

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@shaneutt shaneutt added the priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. label Sep 18, 2024
@shaneutt shaneutt added this to the v1.3.0 milestone Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. kind/gep PRs related to Gateway Enhancement Proposal(GEP) priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
Status: Review
Development

Successfully merging this pull request may close these issues.

GEP: Standard Mechanism to Merge Multiple Gateways
10 participants