Skip to content

[TT-18717] Remove the MCP qualification job; run MCP tests in api-tests - #582

Merged
konrad-sol merged 1 commit into
masterfrom
TT-18717-remove-mcp-qualification
Nov 13, 2026
Merged

konrad-sol merged 1 commit into
masterfrom
TT-18717-remove-mcp-qualification

Conversation

@konrad-sol

@konrad-sol konrad-sol commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Removes the opt-in mcp-qualification feature. MCP API tests then run in the regular parallel api-tests job on tyk-pro instead of a separate serial job with four legs (msgpack/protobuf × mongo/postgres).

  • config/config.yaml: drop the feature from the master branches of tyk, tyk-analytics, tyk-pump and tyk-sink.
  • Delete mcp-qualification.gotmpl, its include in release.yml, its aggregator dependency, and the and not mcp api-tests markers in auto-test.gotmpl.
  • Delete policy/mcp_qualification_test.go and docs/mcp-qualification.md; regenerate the four golden release.yml files.

Coverage on tyk-pro after this change

  • Serializers: Mongo environments use msgpack, Postgres environments protobuf (tyk-pro#115).
  • Raw MCP analytics: raw Mongo and unsharded Postgres MCP pumps (tyk-pro#115) ; the tests read Postgres on Postgres legs (tyk-analytics#6479).
  • Not covered yet: MCP through the MDCB data plane (tyk-sink ran this job with topology: mdcb). tyk-pro has the MDCB stack, so it can be added there as a follow-up.

Why: every PR gets faster and cheaper

The separate MCP job is the slowest part of the PR pipeline. Measured on the last 10 tyk-analytics PR Release runs (warp 8x runners):

Now After this change
Time until required checks finish (Aggregated CI Status), median 36m51s ~27m08s (~8 min sooner; 4.5–14.5 min)
Slowest job MCP leg, 31–37 min (9 of 10 runs) goreleaser → api/ui tests
Runner time spent on MCP per PR ~140 min (4 legs × ~35 min) ~7 min (MCP tests add ~3.5 min to each of the 2 api-tests jobs: ~8 → ~11.5 min)

So each PR saves ~8 minutes of waiting and ~130 runner-minutes. The same applies to tyk, tyk-pump and tyk-sink, which run the same job.

Merge after (the tyk-analytics pipeline uses the latest tyk-pro release):

  1. https://github.com/TykTechnologies/tyk-analytics/pull/6479 (MCP tests pass on tyk-pro)
  2. [TT-18717] Run the MCP API tests on tyk-pro tyk-pro#115 merged and released

go test ./... passes.

🤖 Generated with Claude Code

MCP API tests now pass in parallel on tyk-pro, so the separate serial
MCP qualification job and the "and not mcp" api-tests filter go away
for tyk, tyk-analytics, tyk-pump and tyk-sink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@konrad-sol
konrad-sol merged commit 0816603 into master Nov 13, 2026
18 checks passed
@konrad-sol
konrad-sol deleted the TT-18717-remove-mcp-qualification branch November 13, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants