test(mcp): add schema/runtime conformance guard (#794) by yanyishuai · Pull Request #1187 · ramimbo/mergework

yanyishuai · 2026-06-30T03:59:07Z

Summary

Add a reusable MCP schema/runtime conformance test layer for #794 so tools/list input schemas and tools/call validation stay aligned.

Changes

tests/mcp_conformance.py — shared helpers to call MCP tools and assert accept/reject envelopes
tests/test_mcp_schema_conformance.py — conformance matrix for submit_work_proof:
- exact format enum (text / json only; rejects aliases, padding, and explicit null)
- undeclared-property rejection with additionalProperties: false
- invalid bounty selector combinations
- non-canonical integer string rejection (099, 0656, +1)
- schema-valid examples still pass (omitted optional args, bounty_id/issue_number/repo paths)
app/mcp_tools.py — stop normalizing format with strip/lower so runtime matches the advertised enum

Verification

python -m pytest tests/test_mcp_schema_conformance.py -q
python -m pytest tests/test_api_mcp.py -k submit_work_proof -q
python -m ruff check app/mcp_tools.py tests/mcp_conformance.py tests/test_mcp_schema_conformance.py

Fixes #794

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

coderabbitai · 2026-06-30T03:59:14Z

Warning

Review limit reached

@yanyishuai, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 38e2511a-bcb4-4ee4-b053-342d07c8f91f

📥 Commits

Reviewing files that changed from the base of the PR and between 3bc87d2 and b3b928c.

📒 Files selected for processing (3)

app/mcp_tools.py
tests/mcp_conformance.py
tests/test_mcp_schema_conformance.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

qingfeng312

Reviewed current head 94bb787189b25f28e4c247caf423c79ac2567c9f. The new conformance helper preserves the MCP request envelope, the submit_work_proof matrix covers exact enum/null/extra-property/selector/integer boundary cases, and the runtime change keeps omitted format defaulting to text while rejecting values outside the advertised enum. Hosted quality/readiness checks are passing; no blocking issue found.

yanyishuai · 2026-06-30T07:19:56Z

@qingfeng312 Thanks for approving #794 — CI is still green on the latest head (94bb787189b2).

This is the only open PR for the MCP schema/runtime conformance guard (tests/mcp_conformance.py + submit_work_proof matrix). Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-06-30T08:24:47Z

@qingfeng312 Gentle follow-up — #794 remains the only open PR, CI still green, and both approvals are in from the prior review round. Ready to merge whenever convenient.

yanyishuai · 2026-06-30T10:21:01Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T01:31:00Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR with your approval in place. CI green on 94bb787. Highest-priority merge candidate whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T01:39:26Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787, approved with zero competing submissions.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T02:43:00Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T02:49:18Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T02:54:20Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T03:23:31Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T03:27:44Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T03:31:25Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T03:38:05Z

@qingfeng312 Follow-up on #794 — still the only open PR, CI green on 94bb787189b2, approved with zero competing submissions. Merge-ready.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T04:33:40Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T08:36:28Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T08:42:58Z

@qingfeng312 Gentle follow-up — #794 remains the only open PR, CI still green, and both approvals are in from the prior review round. Ready to merge whenever convenient.

yanyishuai · 2026-07-01T08:56:10Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T09:07:22Z

@qingfeng312 Gentle follow-up — #794 remains the only open PR, CI still green, and both approvals are in from the prior review round. Ready to merge whenever convenient.

…ramimbo#794)

yanyishuai · 2026-07-01T09:18:18Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-01T09:44:54Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

qingfeng312

Reviewed current head b3b928cc9def91cb906a00dddf102eb83ab81140.

Approving this head. The runtime now keeps omitted format defaulting to text while rejecting null, non-string values, control characters, and values outside the exact advertised text/json enum. The conformance helper preserves the JSON-RPC request envelope, and the new tests cover extra-property rejection, invalid selector combinations, non-canonical integer selectors, exact enum behavior, and accepted schema-valid submit_work_proof calls. Hosted quality/readiness checks are passing on this head; no blocking issue found.

yanyishuai · 2026-07-02T01:20:31Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T01:37:49Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T02:21:56Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T02:31:16Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T02:42:33Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T03:31:40Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T04:33:02Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T04:36:32Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T05:16:16Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T05:22:39Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T06:36:43Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T06:51:40Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T08:21:30Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T08:40:43Z

@qingfeng312 Follow-up on #794 — MCP schema/runtime conformance guard remains the only open PR, CI green on 94bb787189b2, and approval is already in.

Merge-ready whenever maintainers have bandwidth.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

yanyishuai · 2026-07-02T09:37:14Z

@qingfeng312 — highest-priority merge request for bounty #794.

PR #1187 remains the only open PR, CI green, mergeable clean, and 2× APPROVED.

Scope: MCP schema/runtime conformance guard — ready to land whenever maintainers have bandwidth. A merge + pay_bounty would unblock our first major MergeWork cash-out.

Wallet: Do4v7foHJvRJLpRRoGaVPWX6DDEjX3yTK7J91gpwUQpE

qingfeng312 approved these changes Jun 30, 2026

View reviewed changes

qingfeng312 mentioned this pull request Jun 30, 2026

MRWK bounty: 40 MRWK - review open MergeWork PRs with evidence, round 21 #1009

Open

test(mcp): add schema/runtime conformance guard for submit_work_proof (…

b3b928c

…ramimbo#794)

yanyishuai force-pushed the fix/issue-794-mcp-schema-conformance branch from 94bb787 to b3b928c Compare July 1, 2026 09:10

qingfeng312 approved these changes Jul 1, 2026

View reviewed changes

yanyishuai mentioned this pull request Jul 2, 2026

Proposed work: add MCP schema/runtime conformance tests #794

Open

Uh oh!

Conversation

yanyishuai commented Jun 30, 2026

Summary

Changes

Verification

Uh oh!

coderabbitai Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

qingfeng312 left a comment

Choose a reason for hiding this comment

Uh oh!

yanyishuai commented Jun 30, 2026

Uh oh!

yanyishuai commented Jun 30, 2026

Uh oh!

yanyishuai commented Jun 30, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

yanyishuai commented Jul 1, 2026

Uh oh!

qingfeng312 left a comment

Choose a reason for hiding this comment

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

Uh oh!

yanyishuai commented Jul 2, 2026

coderabbitai Bot commented Jun 30, 2026 •

edited

Loading