Add conformance scenario: logging/setLevel and log notifications #435

Activity

1. joy7758 commented on Aug 8, 2026

   

   joy7758

   on Aug 8, 2026

   More actions

   Hi — before implementing #435, I’d like to confirm the intended scope.

   On current main (c321dd3), I see two existing 2025-era server scenarios:

   • logging-set-level sends logging/setLevel and checks the empty result.
   • tools-call-with-logging sets debug and checks that log notifications are delivered.

   I therefore would not add another basic happy-path scenario. The remaining bounded gaps I can see are:

   1. verify that a server emitting notifications/message advertised the logging capability;
   2. exercise invalid-level handling (-32602, a SHOULD-level requirement);
   3. if maintainers want this scored, verify threshold behavior for debug versus error.

   I’d appreciate guidance on the third item: the 2025 logging page describes minimum-level filtering and illustrates it in the flow, but I did not find a direct RFC keyword for that specific behavior. I’d also use the repository’s preferred pattern for an intentionally schema-invalid level so the probe does not create a misleading wire-schema-harness-error.

   If this scope is welcome and not already owned, I would keep it 2025-only, reuse the existing runner and everything-server, avoid 2026 per-request logging, protocol changes, runner changes, and broad refactoring, and add positive plus deliberately broken negative fixtures. Before proposing a PR, I would run the repository checks/build/tests and the scenario against a pinned real SDK, then include the exact commands, source pins, checks.json and wire-schema results, and process-cleanup evidence.

   Would you prefer these as additional checks in an existing scenario or as one narrowly scoped post-release scenario?
2. elang2 commented on Aug 9, 2026

   

   elang2

   on Aug 9, 2026

   Author

   More actions

   Hey @joy7758, thanks for flagging this and for the detailed breakdown.

   So here's what happened on my end. I've been working on conformance scenarios for this repo (see my earlier PRs #436 and #438 for cancellation/progress work). I made a dumb mistake with my fork, made it private, which permanently severed the fork relationship and auto-closed those PRs without any notification. Your comment here is actually what tipped me off that something was wrong when I came back to check.

   I've fixed the fork situation and just opened #449 covering the three gaps you identified. The invalid-level and threshold-filtering scenarios are in pendingClientScenariosList to avoid wire-schema harness errors in the default suite, which I think addresses your concern about the intentionally schema-invalid probe.

   Also re-opened the cancellation/progress work as #450 (replacement for the auto-closed #436).

   I'd like to own this if that's alright. I caused myself the mess with the forking and want to see it through. But I'd really appreciate your eyes on #449, especially your point about whether threshold filtering has a direct RFC keyword. Still not sure if SKIPPED (vs FAILURE) is the right status for the no-notifications case.

   Sorry for the confusion.
3. joy7758 commented on Aug 10, 2026

   

   joy7758

   on Aug 10, 2026

   More actions

   Thanks for explaining — no problem, and yes, please go ahead and own #435. I will not open a competing implementation.

   I completed the requested read-only reviews:

   • #449 logging review: threshold-filtering RFC-keyword backing, conditional capability semantics, the zero-notification outcome, targeted wire-schema evidence, and pass/fail plus real-SDK evidence.
   • #450 cancellation/progress review: actual request-ID observability, robustness versus normative cancellation claims, strict progress increase, unsupported total warnings, and evidence-count reconciliation.

   I have left implementation choices with you and the maintainers. These comments are scoped to conformance traceability and evidence; they do not propose protocol changes.