Skip to content

events: conformance control to change the catalog at runtime, for the seven rows nothing can reach #1465

Description

@panyam

What

A conformance control that changes the event catalog while the process runs, so the suite can observe what a server does about it:

events_conformance_catalog { "op": "add" | "remove" | "extend" | "retype", "name": "<event type>" }
  • add registers a fresh type and returns its name
  • remove drops one (RemoveSource)
  • extend adds an optional property to its inputSchema and payloadSchema, the additive case
  • retype changes an existing property's type in place, the breaking case

Registered under --conformance-events beside the nine that exist.

One tool with an op rather than four tools, because they share one argument shape and the suite always names the type — but four separate tools would match the yield_error / yield_gap / terminate precedent just as well, so take whichever reads better in conformance_events.go.

Why

Seven rows are declared and emit nothing, and all seven are about a catalog that changes underneath a client. It is the last group in the suite with no route to a verdict, and the only paired change left:

Row What the document asks
sep-9999-schema-evolution-additive schemas evolve additively for the lifetime of a name: fields SHOULD NOT be removed, renamed or retyped, enums SHOULD NOT be narrowed, inputSchema SHOULD NOT be tightened so previously accepted arguments become invalid
sep-9999-breaking-change-new-name a breaking change SHOULD be published under a new name, served alongside the old for a migration period
sep-9999-list-changed-notification any change to the set of types, or to description / delivery / inputSchema / payloadSchema, sends notifications/events/list_changed
sep-9999-removal-terminates-subscriptions removal ends live subscriptions through each mode's signal: notifications/events/terminated on push, a terminated envelope on webhook
sep-9999-removal-error-not-found after removal, -32011 NotFound with data: {"kind": "event"}
sep-9999-removal-error-schema-changed after an in-place change, -32014 Unsupported with data: {"feature": ..., "reason": "schema_changed"} — pointedly not -32012, since the principal's access is unchanged
sep-9999-removal-additive-no-terminate purely additive changes MUST NOT terminate subscriptions

The library already does all of it: Registry.AddSource, RemoveSource, terminateSubscriptions and broadcastListChanged. Nothing exposes them to a client, so a harness cannot ask, and a healthy server never does any of it during a run. The rows have reported untestable since the suite was written.

Worth noting what the last two rows are really testing, because it is the part a library gets wrong quietly: the difference between an additive change and a breaking one. Additive must keep subscriptions alive; breaking must end them, with a code that says "the shape moved" and not "you lost access". No amount of reading the source proves a server draws that line in the right place.

Hazards, from the ones already built

  • One-shot operations poison the type. remove and retype end subscriptions for the life of the process, the same trap events_conformance_terminate hit, which is why build.finished exists and carries no feeder. These ops want their own disposable types — ideally add mints a fresh one per call so a scenario can add, mutate and remove without touching anything another scenario subscribes to.
  • add needs a descriptor worth grading. A type with a typed inputSchema property and a payloadSchema, so extend has something to extend and retype something to retype.
  • Additive is only observable if the old arguments still work. After extend, a poll with the pre-change arguments must still be accepted, which is the actual content of the SHOULD.

Suite side

Not written yet, unlike #1457 where it was waiting. Once the control exists the rows land roughly as: the two schema-evolution rows and list-changed-notification in events-discovery, the removal error codes in events-poll, and the termination signals in events-push and events-webhook-delivery, which already know how to watch for terminated.

One prerequisite is ours rather than yours: grading list_changed means receiving an unsolicited server notification, and the harness has no listen channel yet — the same gap that leaves tasks-status-notifications pending. The re-list-after-mutation half needs no such thing, so that row may land in two pieces.

Conformance PR: modelcontextprotocol/conformance#504.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions