Conformance and Support matrix enhancements

[youngnick]

This may end up being a GEP, but this issue is to discuss enhancements to the conformance process, with a couple of end goals in mind:

• As a user of Gateway API, I should be able to look at the status of GatewayClass and see information about what features the reconciling implementation supports.
• As a user of Gateway API, I should be able to consult a feature matrix to determine which implementations support which features. These features should be supported by and/or derived from conformance information. (Thanks @candita for the suggestion!)

A related goal is:

• as a Gateway API implementer, the conformance tests I run should be able to autodetect what features are supported and conformance test them automatically.

I've linked these together in this issue because the schema we'll need for the first will end up looking a lot like the column for the second, and that schema will probably be closely related if not identical to what we'll need from the conformance tests too.

[youngnick]

I think it's relevant to call out a few things that the schema will need to track;

• Supported Route types
• Extended features available in various objects. It may be better not to namespace the extended features by object though.

I think we'll also need to get more specific about which objects must always be watched, and which ones can be watched sometimes. I'll make another issue about that now.

[candita]

I would add:

• Supported versions (alpha, beta, experimental, standard, semantic versioning, or other)

@youngnick would you mind moving the "related goal" conformance test linking to a different issue? It could really complicate a fairly simple support matrix to add conformance test linking as a type of acceptance criteria.

[youngnick]

I agree that supported versions should also be included, agreed.

I'd like to leave that note in here, but should add that it's not necessary to complete this effort.

[shaneutt]

So technically #1709 is starting to at least talk about these ideas, but with the GA timeline it probably wont complete them. It may be helpful for us to consider this a follow up issue to GA to continue the work started in conformance profiles.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[youngnick]

Hmm I think that there are three asks here:

• See what features are supported in GatewayClass status, This is GEP-2162 (GEP: GatewayClass status supported features #2162).
• Display of conformance profiles, this is Display for conformance profiles results #2550
• Conformance tests should look at supported features to choose which tests to run. I think this is being covered in GEP: Conformance Profiles #1709 or related work. @shaneutt @mlavacca?

If that last one is the case, I think that we can close this out.

[shaneutt]

@LiorLieberman and I discussed the SupportedFeatures based configuration of conformance profiles test runs, I'll need to catch up on whether we codified that anywhere or if it was just a discussion.

[LiorLieberman]

Thats the plan, I planned to graduate SupportedFeatures from experimental flipping the bit.
We have istio already support it and a PR inflight for envoy (waiting GW upstream release I think.

We could start with adding a flag or a framework option to opt-in to read supportedFeatures from the GWC and deprecate it later when we want everyone to move to this.
It needs more elaboration though.

[youngnick]

Yes, I think that adding a flag to allow implementations to opt in to the new behavior is a must, and lets us do the standard three-phase rollout:

• new flag, defaults false
• Once we have enough implementations, default to true
• once everyone is using flag true, remove flag

The best part about having the flag is that it lets us make the tooling for testing very standard, much easier for folks to replicate the report results if necessary.

[youngnick]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[youngnick]

@LiorLieberman, are we still interested in doing this.

[LiorLieberman]

It is still the plan, envoy-gw and istio already support it.
I was chatting with @mikemorris to do that in their implementation. So we can get 3 implementors before.

@mikemorris mind sharing if and when you plan to implement this?

[mikemorris]

@LiorLieberman I believe we are intending to implement this for Azure Application Gateway for Containers but unsure on timeline/roadmap, @snehachhabria may be able to confirm.

[snehachhabria]

@LiorLieberman I believe we are intending to implement this for Azure Application Gateway for Containers but unsure on timeline/roadmap, @snehachhabria may be able to confirm.

The preference for Application Gateway for Containers is to have releases of features in Gateway API that are a part of the standard channel only and since the supported features is a part of the experimental channel, we will not have an official release supporting it until it becomes a part of the standard channel. If the goal is to implement and provide feedback to help move it to the standard channel, that's something we can do (without any release). @LiorLieberman and @mikemorris let me know if this would work for you'll.

[LiorLieberman]

Thanks @snehachhabria! How would we get the feedback if it would not be released ?

[snehachhabria]

Thanks @snehachhabria! How would we get the feedback if it would not be released ?

the feedback would be from what we encounter during implementation and testing internally only

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten