feat(conformance): Add EPP conformance test for Gateway routing #961

zetxqx · 2025-06-11T03:58:33Z

This pull request introduces a new conformance test, GatewayFollowingEPPRouting, to validate that a Gateway correctly implements routing logic based on an EPP.

The new test suite ensures that the Gateway honors the EPP's endpoint selection. We used the testing-epp to control the EPP behavior using request header. The two test cases are

request header:[pod1IP, pod2IP, pod3IP]: EPP returns pod1IP, pod2IP, pod3IP, gateway should select the first one pod1IP and send traffic to
request header:[pod3IP, pod2IP, pod1IP] EPP returns ppod3IP, pod2IP, pod1IP, gateway should select the first one pod3IP and send traffic to

Verification

The new conformance test was run against the gke-l7-regional-external-managed GatewayClass and passed successfully.

against gke-l7

Test command

go test -v ./conformance -args -debug \
  -gateway-class gke-l7-regional-external-managed \
  -cleanup-base-resources=false \
  -run-test GatwayFollowingEPPRouting

Test results

=== RUN   TestConformance/HTTPRouteInvalidInferencePoolRef
    conformance.go:68: Skipping HTTPRouteInvalidInferencePoolRef: test explicitly skipped
=== RUN   TestConformance/InferencePoolAccepted
    conformance.go:68: Skipping InferencePoolAccepted: test explicitly skipped
=== RUN   TestConformance/InferencePoolResolvedRefsCondition
    conformance.go:68: Skipping InferencePoolResolvedRefsCondition: test explicitly skipped
--- PASS: TestConformance (53.87s)
    --- PASS: TestConformance/GatewayFollowingEPPRouting (51.41s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_a_single_designated_pod (0.08s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_two_designated_pods (0.09s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_all_available_pods (0.10s)
    --- SKIP: TestConformance/HTTPRouteInvalidInferencePoolRef (0.00s)
    --- SKIP: TestConformance/InferencePoolAccepted (0.00s)
    --- SKIP: TestConformance/InferencePoolResolvedRefsCondition (0.00s)
PASS
ok  	sigs.k8s.io/gateway-api-inference-extension/conformance	54.054s

against istio (FAILED)

The istio implementation may still send traffic to pods not selected by EPP.

go test -v ./conformance -args -debug -gateway-class istio -cleanup-base-resources=false -run-test GatewayFollowingEPPRouting

Test results

error log: gateway_following_epp_routing.go:144: Not all the requests are sent to the expectedPods successfully, err: request was handled by an unexpected pod "infra-backend-deployment-678ff64fb-txms2", Reached pods with request counts: map[infra-backend-deployment-678ff64fb-tnlhd:30 infra-backend-deployment-678ff64fb-txms2:32 infra-backend-deployment-678ff64fb-zjwk2:38]

=== RUN   TestConformance/HTTPRouteInvalidInferencePoolRef
    conformance.go:68: Skipping HTTPRouteInvalidInferencePoolRef: test explicitly skipped
=== RUN   TestConformance/InferencePoolAccepted
    conformance.go:68: Skipping InferencePoolAccepted: test explicitly skipped
=== RUN   TestConformance/InferencePoolResolvedRefsCondition
    conformance.go:68: Skipping InferencePoolResolvedRefsCondition: test explicitly skipped
--- FAIL: TestConformance (139.98s)
    --- FAIL: TestConformance/GatewayFollowingEPPRouting (137.31s)
        --- FAIL: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_a_single_designated_pod (0.05s)
        --- FAIL: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_two_designated_pods (0.05s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_all_available_pods (0.05s)
    --- SKIP: TestConformance/HTTPRouteInvalidInferencePoolRef (0.00s)
    --- SKIP: TestConformance/InferencePoolAccepted (0.00s)
    --- SKIP: TestConformance/InferencePoolResolvedRefsCondition (0.00s)
FAIL
FAIL	sigs.k8s.io/gateway-api-inference-extension/conformance	140.176s
FAIL

TODO(all done now)

(Done) Currently the Mock EPP can only accept one endpoint from the request header, I need to add multi-endpoints support to test the case when EPP return multiple endpoints. Extending feat(Conformance): Add a header based filter to make a controllable epp behavior determined by request header. #922
(Done) Need to refactor the test utils. The library our conformance test is depending on need some modification. Currently the library can only make request w/o body. I try to make some minimum change to make this PR.
(Done) Need to figure out a poll and check way to make sure the whole stack is ready to server request.

netlify · 2025-06-11T03:58:38Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`1a3daae`
🔍 Latest deploy log	https://app.netlify.com/projects/gateway-api-inference-extension/deploys/6855d5437ce45f0008444470
😎 Deploy Preview	https://deploy-preview-961--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

k8s-ci-robot · 2025-06-11T03:58:42Z

Hi @zetxqx. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

conformance/utils/traffic/traffic.go

conformance/tests/basic/gateway_following_epp_routing.go

robscott

Thanks @zetxqx!

conformance/tests/basic/gateway_following_epp_routing.go

conformance/tests/basic/gateway_following_epp_routing.yaml

conformance/utils/kubernetes/helpers.go

conformance/utils/traffic/traffic.go

conformance/tests/basic/gateway_following_epp_routing.yaml

… and small refactor.

conformance/utils/traffic/traffic.go

robscott

Thanks for all the work on this @zetxqx!

conformance/conformance.go

conformance/tests/basic/gateway_following_epp_routing.go

conformance/utils/traffic/traffic.go

robscott · 2025-06-13T20:05:21Z

/ok-to-test

LiorLieberman · 2025-06-19T00:14:22Z

cc: @aslakknutsen

conformance/resources/manifests/manifests.yaml

danehans · 2025-06-20T00:00:38Z

#961 (comment) needs an answer.

/hold

conformance/utils/traffic/traffic.go

zetxqx · 2025-06-20T20:06:03Z

I reverted the dependency on gateway-1.3.1 commit sha and duplicate some code from gateway-api. I also created a github issue #1031 for tracking to remove the duplicate when the gateway-api is released. PTAL @robscott @danehans

robscott

Thanks @zetxqx!

conformance/tests/basic/gateway_following_epp_routing.yaml

conformance/utils/traffic/traffic.go

conformance/tests/basic/gateway_following_epp_routing.go

zetxqx · 2025-06-20T21:38:13Z

Run the test again

=== NAME  TestConformance/GatewayFollowingEPPRouting
    apply.go:283: 2025-06-20T21:33:39.368887167Z: Deleting httproute-for-primary-gw HTTPRoute
    apply.go:283: 2025-06-20T21:33:39.440315387Z: Deleting conformance-fake-model-server InferenceModel
=== RUN   TestConformance/HTTPRouteInvalidInferencePoolRef
    conformance.go:68: Skipping HTTPRouteInvalidInferencePoolRef: test explicitly skipped
=== RUN   TestConformance/InferencePoolAccepted
    conformance.go:68: Skipping InferencePoolAccepted: test explicitly skipped
=== RUN   TestConformance/InferencePoolHTTPRoutePortValidation
    conformance.go:68: Skipping InferencePoolHTTPRoutePortValidation: test explicitly skipped
=== RUN   TestConformance/InferencePoolInvalidEPPService
    conformance.go:68: Skipping InferencePoolInvalidEPPService: test explicitly skipped
=== RUN   TestConformance/HTTPRouteMultipleRulesDifferentPools
    conformance.go:68: Skipping HTTPRouteMultipleRulesDifferentPools: test explicitly skipped
=== RUN   TestConformance/InferencePoolResolvedRefsCondition
    conformance.go:68: Skipping InferencePoolResolvedRefsCondition: test explicitly skipped
--- PASS: TestConformance (148.79s)
    --- PASS: TestConformance/GatewayFollowingEPPRouting (144.34s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_a_single_designated_pod (0.09s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_two_designated_pods (0.10s)
        --- PASS: TestConformance/GatewayFollowingEPPRouting/should_route_traffic_to_all_available_pods (0.09s)
    --- SKIP: TestConformance/HTTPRouteInvalidInferencePoolRef (0.00s)
    --- SKIP: TestConformance/InferencePoolAccepted (0.00s)
    --- SKIP: TestConformance/InferencePoolHTTPRoutePortValidation (0.00s)
    --- SKIP: TestConformance/InferencePoolInvalidEPPService (0.00s)
    --- SKIP: TestConformance/HTTPRouteMultipleRulesDifferentPools (0.00s)
    --- SKIP: TestConformance/InferencePoolResolvedRefsCondition (0.00s)
PASS
ok  	sigs.k8s.io/gateway-api-inference-extension/conformance	149.046s

robscott

Thanks @zetxqx!

/lgtm

robscott · 2025-06-20T21:42:33Z

#961 (comment) needs an answer.

/hold

Good catch - I do think it's unlikely that we'll be able to consistently line up releases across repos, so some temporary duplication is probably better. I think we can resolve this now and remove the hold.

danehans · 2025-06-21T01:47:04Z

/unhold

danehans · 2025-06-21T01:48:25Z

/approve

zetxqx · 2025-06-21T02:24:49Z

/retest

k8s-ci-robot · 2025-06-21T02:30:12Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danehans, robscott, zetxqx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [danehans]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aslakknutsen · 2025-06-24T12:32:31Z

@zetxqx Replicas 3 here seems to break HTTPRouteMultipleRulesDifferentPools which is expecting only 1 pod pr pool.

zetxqx added 3 commits June 9, 2025 23:32

Add gateway_following_epp_routing test.

8d6fb58

One working version.

f879558

Okay version of GatwayFollowingEPPRouting conformance test.

9f3e382

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 11, 2025

k8s-ci-robot requested review from ahg-g and danehans June 11, 2025 03:58

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 11, 2025

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 11, 2025

zetxqx commented Jun 11, 2025

View reviewed changes

conformance/utils/traffic/traffic.go Outdated Show resolved Hide resolved

zetxqx commented Jun 11, 2025

View reviewed changes

conformance/tests/basic/gateway_following_epp_routing.go Outdated Show resolved Hide resolved

robscott reviewed Jun 11, 2025

View reviewed changes

zetxqx added 3 commits June 11, 2025 18:28

fix typos and formats.

b83290c

Merge branch 'main' into epptest

a680c76

upgrader gateway-api versino to use the updated conformance testutils…

ff3086a

… and small refactor.

zetxqx mentioned this pull request Jun 11, 2025

Update Gateway API dependency to a stable release tag once available #969

Open

zetxqx commented Jun 11, 2025

View reviewed changes

conformance/utils/traffic/traffic.go Outdated Show resolved Hide resolved

zetxqx requested a review from robscott June 11, 2025 22:54

use AllowCRDsMismatch to bypass.

5b725d0

zetxqx mentioned this pull request Jun 12, 2025

Conformance Tests for v0.5 Release #977

Open

12 tasks

format.

838b535

robscott reviewed Jun 13, 2025

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 13, 2025

refactor more.

f313e58

k8s-ci-robot removed the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 13, 2025

danehans reviewed Jun 19, 2025

View reviewed changes

conformance/resources/manifests/manifests.yaml Show resolved Hide resolved

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 20, 2025

zetxqx commented Jun 20, 2025

View reviewed changes

conformance/utils/traffic/traffic.go Outdated Show resolved Hide resolved

zetxqx mentioned this pull request Jun 20, 2025

Conformance refactor: remove duplicate waitForConvergeToExpected logic and use the new released gateway-api library #1031

Open

back to depend on gateway-api 1.30.

5b4a86c

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 20, 2025

update go.sum.

9c7788f

format.

2627f00

robscott reviewed Jun 20, 2025

View reviewed changes

zetxqx added 2 commits June 20, 2025 21:23

resolve minor comments.

72c4ec1

remove seen logic.

fa47b15

trailing new line.

1a3daae

robscott approved these changes Jun 20, 2025

View reviewed changes

k8s-ci-robot assigned robscott Jun 20, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2025

zetxqx requested a review from danehans June 20, 2025 21:45

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 21, 2025

danehans added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 21, 2025

k8s-ci-robot merged commit a0cfd24 into kubernetes-sigs:main Jun 21, 2025
9 checks passed

feat(conformance): Add EPP conformance test for Gateway routing #961

feat(conformance): Add EPP conformance test for Gateway routing #961

Conversation

zetxqx commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Verification

against gke-l7

against istio (FAILED)

TODO(all done now)

Uh oh!

netlify bot commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for gateway-api-inference-extension ready!

Uh oh!

k8s-ci-robot commented Jun 11, 2025

Uh oh!

Uh oh!

Uh oh!

robscott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

robscott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

robscott commented Jun 13, 2025

Uh oh!

LiorLieberman commented Jun 19, 2025

Uh oh!

Uh oh!

danehans commented Jun 20, 2025

Uh oh!

Uh oh!

zetxqx commented Jun 20, 2025

Uh oh!

robscott left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zetxqx commented Jun 20, 2025

Uh oh!

robscott left a comment

Choose a reason for hiding this comment

Uh oh!

robscott commented Jun 20, 2025

Uh oh!

danehans commented Jun 21, 2025

Uh oh!

danehans commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zetxqx commented Jun 21, 2025

Uh oh!

k8s-ci-robot commented Jun 21, 2025

Uh oh!

Uh oh!

aslakknutsen commented Jun 24, 2025

zetxqx commented Jun 11, 2025 •

edited

Loading

netlify bot commented Jun 11, 2025 •

edited

Loading

danehans commented Jun 21, 2025 •

edited

Loading