MCP Validator

A testing suite and reference implementation for the Model Context Protocol (MCP).

Summary

The MCP Validator provides a comprehensive environment for testing and validating MCP server implementations. It includes reference implementations and a testing framework to ensure compliance with the MCP specification.

Latest Protocol Support: The validator now supports MCP protocol version 2025-06-18, which includes major enhancements like structured tool output, OAuth 2.1 authentication, elicitation support, removal of JSON-RPC batching, and enhanced security features.

🔐 OAuth 2.1 Authentication Support

The validator includes comprehensive OAuth 2.1 authentication support for the 2025-06-18 protocol version, providing a complete framework for secure MCP implementations.

Authentication Features

OAuth 2.1 Compliance: Full RFC 6749, RFC 6750, and OAuth 2.1 draft compliance
Bearer Token Support: Proper Bearer token extraction and validation
WWW-Authenticate Headers: Standards-compliant 401 responses with authentication challenges
Resource Server Capabilities: Complete OAuth 2.1 resource server implementation
MCP-Protocol-Version Headers: Protocol version negotiation via HTTP headers
Security Headers: CORS, Origin validation, and DNS rebinding attack prevention

OAuth 2.1 Configuration

The reference HTTP server supports OAuth 2.1 authentication through environment variables:

# Enable OAuth 2.1 authentication
export MCP_OAUTH_ENABLED=true
export MCP_OAUTH_INTROSPECTION_URL=https://auth.example.com/oauth/introspect
export MCP_OAUTH_REQUIRED_SCOPES=mcp:read,mcp:write
export MCP_OAUTH_REALM=mcp-server

# Optional: Configure resource indicators (RFC 8707)
export MCP_OAUTH_RESOURCE_INDICATORS=https://api.example.com/mcp

Authentication Testing

Test OAuth 2.1 compliance with the built-in authentication test suite:

# Test OAuth 2.1 features (framework validation)
python mcp_testing/scripts/http_compliance_test.py --debug

# Test with authentication enabled
MCP_OAUTH_ENABLED=true python ref_http_server/reference_mcp_server.py --port 8088 &
python mcp_testing/scripts/http_compliance_test.py --server-url http://localhost:8088

# Test Bearer token handling
curl -H "Authorization: Bearer your-token-here" \
     -H "MCP-Protocol-Version: 2025-06-18" \
     -H "Content-Type: application/json" \
     -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"client_info":{"name":"Test"},"client_capabilities":{"protocol_versions":["2025-06-18"]}}}' \
     http://localhost:8088/messages

🚀 2025-06-18 Protocol Features

Core Enhancements

Structured Tool Output: Tools return structured responses with content, isError, and structuredContent fields
Enhanced Tool Schema: Tools use inputSchema (renamed from parameters), title, and outputSchema fields
JSON-RPC Batching Removal: Batch requests are properly rejected for 2025-06-18 protocol
Elicitation Support: Framework for user interaction and data collection
Protocol Version Headers: HTTP requests include MCP-Protocol-Version header

Security Improvements

OAuth 2.1 Authentication: Complete authentication framework
Enhanced Error Handling: Improved error responses with security considerations
Origin Validation: DNS rebinding attack prevention
Session Security: Secure session management with per-session protocol versions

STDIO Compliance Testing

The validator includes a comprehensive testing suite for STDIO-based MCP servers.

Running STDIO Tests

# Run compliance tests for the STDIO server (latest protocol)
python -m mcp_testing.scripts.compliance_report --server-command "python ref_stdio_server/stdio_server_2025_03_26.py" --protocol-version 2025-06-18

# Run with previous protocol versions
python -m mcp_testing.scripts.compliance_report --server-command "python ref_stdio_server/stdio_server_2025_03_26.py" --protocol-version 2025-03-26

STDIO Test Coverage

The STDIO compliance tests verify:

Protocol Initialization
Tools Functionality
- Basic tools (echo, add)
- Async tools (sleep) for 2025-03-26+ versions
- Structured tool output (2025-06-18)
Error Handling
Protocol Version Negotiation
Advanced Features (2025-06-18)
- Elicitation support
- Enhanced validation
- JSON-RPC batching restrictions

Testing Different STDIO Server Types

The validator supports testing any STDIO-based MCP server, whether it's run directly from a command or installed via pip. Here's how to test different types of servers:

Direct Command Testing

For servers that run directly from a Python file or command:

# Test a local Python file (latest protocol)
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/your/server.py" --protocol-version 2025-06-18

# Test with specific timeouts
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-06-18 --test-timeout 30 --tools-timeout 15

# Focus on tools testing with dynamic discovery
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-06-18 --test-mode tools --dynamic-only

# Test with previous protocol versions for compatibility
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-03-26

Testing Pip-Installed Servers

For servers installed via pip (like mcp-server-fetch):

# Ensure you're in the correct virtual environment
source .venv/bin/activate

# Install the server and dependencies
pip install your-mcp-server  # Replace with actual package name

# Run compliance tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m your_server_module" --protocol-version 2024-11-05

# Run tools-only tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m your_server_module" --protocol-version 2024-11-05 --test-mode tools

# example brave search server
BRAVE_API_KEY=api-key python -m mcp_testing.scripts.compliance_report --server-command "npx -y @modelcontextprotocol/server-brave-search" --protocol-version 2024-11-05

Test Configuration Options

Common options for both types:

--test-mode tools: Focus on testing tool functionality
--dynamic-only: Automatically discover and test available tools
--test-timeout 30: Set timeout for regular tests (seconds)
--tools-timeout 15: Set timeout for tool-specific tests (seconds)
--required-tools tool1,tool2: Specify required tools to test
--skip-tests test1,test2: Skip specific tests
--skip-async: Skip async tool testing

Note: Tool-related tests that timeout are treated as non-critical, allowing testing to continue.

Test Reports

Each test run generates a detailed report containing:

Server information (command, protocol version)
Test execution timestamp
Test duration
Success rate
Detailed results for each test case
Server capabilities
Session information

Running Tests

You can run different types of tests using module-style commands:

# Basic interaction test (latest protocol)
python -m mcp_testing.scripts.basic_interaction --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18

# Compliance tests with tools-only mode
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18 --test-mode tools

# Set custom timeouts for tools tests vs. other tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18 --test-timeout 30 --tools-timeout 15

# Test only specific functionality
python -m mcp_testing.scripts.compliance_report --server-command "/path/to/server" --protocol-version 2025-06-18 --test-mode tools

# Skip async tests if experiencing hanging issues
python -m mcp_testing.scripts.compliance_report --server-command "/path/to/server" --protocol-version 2025-06-18 --skip-async

# HTTP debug output
python -m mcp_testing.scripts.http_test --server-url http://localhost:8000/mcp --debug

# Test backward compatibility with older protocols
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2024-11-05

Note: Tool-related tests that timeout are treated as non-critical, allowing testing to continue.

HTTP Compliance Testing

The validator includes a comprehensive compliance testing suite for HTTP-based MCP servers with full 2025-06-18 protocol support.

Running HTTP Tests

# Start the reference HTTP server (runs on port 8088)
python ref_http_server/reference_mcp_server.py

# Run compliance tests and generate a detailed report
python -m mcp_testing.scripts.http_compliance_test --output-dir reports

# Test with OAuth 2.1 authentication enabled
MCP_OAUTH_ENABLED=true python ref_http_server/reference_mcp_server.py --port 8088 &
python -m mcp_testing.scripts.http_compliance_test --server-url http://localhost:8088 --debug

HTTP Test Coverage

The HTTP compliance test suite verifies:

Protocol Initialization
- 2025-06-18 protocol negotiation
- Session management with protocol-specific features
- MCP-Protocol-Version header handling
Tools Functionality
- Structured tool output (2025-06-18)
- Legacy tool responses (older protocols)
- Enhanced tool schema validation
Error Handling
- Enhanced error responses (2025-06-18)
- OAuth 2.1 authentication errors
- Proper HTTP status codes
Batch Request Processing
- Batch request restrictions (2025-06-18)
- Legacy batch support (older protocols)
Session Management
- Per-session protocol versions
- Session security and validation
Protocol Negotiation
- Multi-version support (2024-11-05, 2025-03-26, 2025-06-18)
- Intelligent version selection
Authentication & Security
- OAuth 2.1 Bearer token validation
- WWW-Authenticate header compliance
- CORS and origin validation
Ping Utility
- Connection testing and validation

Production Deployment

For production deployments with 2025-06-18 protocol:

# Enable HTTPS (required for OAuth 2.1 in production)
export MCP_TLS_CERT_FILE=/path/to/cert.pem
export MCP_TLS_KEY_FILE=/path/to/key.pem

# Configure OAuth 2.1 authentication
export MCP_OAUTH_ENABLED=true
export MCP_OAUTH_INTROSPECTION_URL=https://auth.example.com/oauth/introspect
export MCP_OAUTH_REQUIRED_SCOPES=mcp:read,mcp:write

# Configure CORS for web clients
export MCP_CORS_ORIGINS=https://myapp.example.com,https://anotherapp.example.com

# Enable rate limiting and security features
export MCP_RATE_LIMIT_ENABLED=true
export MCP_MAX_REQUESTS_PER_MINUTE=100

# Start the server
python ref_http_server/reference_mcp_server.py --port 443 --production

Testing Scripts Overview

The following scripts are available in mcp_testing/scripts/:

Active and Maintained

http_compliance_test.py: Primary script for HTTP server testing with 2025-06-18 support (7/7 tests passing)
compliance_report.py: Primary script for STDIO server testing with 2025-06-18 support (enhanced test coverage)

Supporting Scripts mixed working/in progress

basic_interaction.py: Simple tool for testing basic server functionality
http_test.py: Lower-level HTTP testing utilities with OAuth 2.1 support
http_compliance.py: Core HTTP compliance testing logic
http_compliance_report.py: Report generation for HTTP tests
run_stdio_tests.py: Lower-level STDIO testing utilities
session_test.py: Session management testing utilities

📋 Protocol Version Support

Protocol Version	Status	Features
2025-06-18	✅ Full Support	Structured tool output, OAuth 2.1, elicitation, no batching
2025-03-26	✅ Full Support	Async tools, enhanced capabilities, batch support
2024-11-05	✅ Full Support	Basic MCP functionality, legacy compatibility

🧪 Test Results Summary

Current test coverage for 2025-06-18 protocol:

✅ HTTP Compliance Test: 7/7 tests passed
✅ OAuth 2.1 Framework: 6/6 tests passed  
✅ Protocol Features: 7/7 tests passed
✅ Multi-Protocol Support: 3/3 versions supported
✅ Backward Compatibility: 100% maintained
✅ Security Features: Authentication, CORS, Origin validation

📚 Additional Resources

JSON Schema: Complete schemas available in schema/ directory
Specification Documents: Full specification files in specification/ directory
Reference Implementations: HTTP and STDIO servers in ref_http_server/ and ref_stdio_server/
Test Reports: Generated reports available in reports/ directory

Name		Name	Last commit message	Last commit date
Latest commit History 157 Commits
.github		.github
docs		docs
mcp_testing		mcp_testing
ref_gh_actions		ref_gh_actions
ref_http_server		ref_http_server
ref_stdio_server		ref_stdio_server
schema		schema
specification		specification
tests		tests
.flake8		.flake8
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE.txt		LICENSE.txt
MANIFEST.in		MANIFEST.in
README.md		README.md
RELEASE_NOTES.md		RELEASE_NOTES.md
SECURITY.md		SECURITY.md
cursor.json		cursor.json
pytest.ini		pytest.ini
requirements.txt		requirements.txt

License

Janix-ai/mcp-validator

Folders and files

Latest commit

History

Repository files navigation

MCP Validator

Summary

🔐 OAuth 2.1 Authentication Support

Authentication Features

OAuth 2.1 Configuration

Authentication Testing

🚀 2025-06-18 Protocol Features

Core Enhancements

Security Improvements

STDIO Compliance Testing

Running STDIO Tests

STDIO Test Coverage

Testing Different STDIO Server Types

Direct Command Testing

Testing Pip-Installed Servers

Test Configuration Options

Test Reports

Running Tests

HTTP Compliance Testing

Running HTTP Tests

HTTP Test Coverage

Production Deployment

Testing Scripts Overview

Active and Maintained

Supporting Scripts mixed working/in progress

📋 Protocol Version Support

🧪 Test Results Summary

📚 Additional Resources

License

About

Resources

License

Security policy

Uh oh!

Stars

Watchers

Forks

Releases 2

Packages 0

Uh oh!

Contributors 3

Uh oh!

Languages

Packages