Skip to content

Commit 4df16e8

Browse files
scottHATscottHAT
committed
added files to prep for release
1 parent 0bd91bf commit 4df16e8

12 files changed

+429
-1
lines changed

‎.flake8

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[flake8]
2+
# E501: Line too long
3+
# F824: Unused 'global xxx' - there's a specific case in the HTTP server that is a false positive
4+
ignore = E501, F824
5+
max-line-length = 100
6+
exclude = .git,__pycache__,docs/,build/,dist/,.venv/

‎.github/CODEOWNERS

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Default owners for everything in the repo
2+
* @scottwilcox
3+
4+
# Testing framework
5+
/mcp_testing/ @scottwilcox
6+
7+
# Minimal server implementations
8+
/minimal_mcp_server/ @scottwilcox
9+
/minimal_http_server/ @scottwilcox
10+
11+
# Protocol specifications
12+
/specification/ @scottwilcox
13+
14+
# JSON Schema
15+
/schema/ @scottwilcox

‎.github/ISSUE_TEMPLATE/bug_report.md

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: '[BUG] '
5+
labels: bug
6+
assignees: ''
7+
8+
---
9+
10+
**Describe the bug**
11+
A clear and concise description of what the bug is.
12+
13+
**To Reproduce**
14+
Steps to reproduce the behavior:
15+
1. Run command '...'
16+
2. See error
17+
18+
**Expected behavior**
19+
A clear and concise description of what you expected to happen.
20+
21+
**Logs and Error Messages**
22+
```
23+
Paste any relevant logs or error messages here
24+
```
25+
26+
**Environment (please complete the following information):**
27+
- OS: [e.g. macOS, Ubuntu, Windows]
28+
- Python version: [e.g. 3.10.4]
29+
- MCP Protocol Validator version: [e.g. 1.0.0]
30+
- Server being tested: [e.g. minimal_mcp_server, fetch server]
31+
- Protocol version: [e.g. 2024-11-05, 2025-03-26]
32+
33+
**Additional context**
34+
Add any other context about the problem here.

‎.github/ISSUE_TEMPLATE/feature_request.md

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: '[FEATURE] '
5+
labels: enhancement
6+
assignees: ''
7+
8+
---
9+
10+
**Is your feature request related to a problem? Please describe.**
11+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12+
13+
**Describe the solution you'd like**
14+
A clear and concise description of what you want to happen.
15+
16+
**Describe alternatives you've considered**
17+
A clear and concise description of any alternative solutions or features you've considered.
18+
19+
**Additional context**
20+
Add any other context or screenshots about the feature request here.
21+
22+
**Relevant MCP Protocol Specifications**
23+
If this feature is related to a specific part of the MCP protocol specification, please provide references or links.

‎.github/workflows/ci.yml

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: MCP Protocol Validator CI
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
pull_request:
7+
branches: [ main ]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
python-version: ['3.9', '3.10', '3.11']
15+
16+
steps:
17+
- uses: actions/checkout@v3
18+
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v4
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
24+
- name: Install dependencies
25+
run: |
26+
python -m pip install --upgrade pip
27+
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
28+
pip install pytest
29+
30+
- name: Run tests
31+
run: |
32+
pytest
33+
34+
- name: Run minimal server compliance tests
35+
run: |
36+
python -m mcp_testing.scripts.compliance_report \
37+
--server-command "./minimal_mcp_server/minimal_mcp_server.py" \
38+
--protocol-version 2025-03-26 \
39+
--output-dir "./reports" \
40+
--report-prefix "ci_minimal_"
41+
42+
- name: Upload test reports
43+
uses: actions/upload-artifact@v3
44+
if: always()
45+
with:
46+
name: test-reports-${{ matrix.python-version }}
47+
path: ./reports/

‎.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ __pycache__/
44
*.pyo
55
reports/*
66
plan.md
7+
plan_pip.md
8+
SERVER_COMPATIBILITY.md
79
mcp_testing/server_configs/*
810

911
# Virtual environment

‎CHANGELOG.md

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Changelog
2+
3+
All notable changes to the MCP Protocol Validator will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [Unreleased]
9+
10+
### Added
11+
- Initial public release
12+
- Support for testing MCP protocol versions 2024-11-05 and 2025-03-26
13+
- Minimal reference implementation for STDIO transport
14+
- Minimal reference implementation for HTTP transport
15+
- Comprehensive test suite for protocol compliance
16+
- Dynamic tool testing
17+
- Timeout handling for problematic servers
18+
- Detailed compliance reporting
19+
- Support for pip-installed MCP servers
20+
21+
### Changed
22+
- Improved timeout handling for problematic servers
23+
- Enhanced reporting format with more detailed diagnostics
24+
- Streamlined test execution with better progress indicators
25+
26+
### Fixed
27+
- Properly handle servers that don't implement shutdown method
28+
- Timeout gracefully for servers with tool list issues
29+
- Correctly display status for skipped tests in reports

‎CONTRIBUTING.md

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Contributing to MCP Protocol Validator
2+
3+
Thank you for considering contributing to the MCP Protocol Validator! This document provides guidelines and instructions for contributing to this project.
4+
5+
## Code of Conduct
6+
7+
Please be respectful and considerate of others when contributing to this project. Any form of harassment or disrespectful behavior will not be tolerated.
8+
9+
## How to Contribute
10+
11+
### Reporting Issues
12+
13+
If you find a bug or have a suggestion for improvement:
14+
15+
1. Check if the issue already exists in the [GitHub Issues](https://github.com/your-username/mcp-protocol-validator/issues)
16+
2. If not, create a new issue using the appropriate template
17+
3. Provide as much detail as possible, including steps to reproduce, expected behavior, and your environment
18+
19+
### Submitting Changes
20+
21+
1. Fork the repository
22+
2. Create a new branch for your changes (`git checkout -b feature/your-feature-name`)
23+
3. Make your changes
24+
4. Run tests to ensure your changes don't break existing functionality
25+
5. Commit your changes with a descriptive commit message
26+
6. Push your branch to your fork
27+
7. Submit a pull request to the main repository
28+
29+
### Pull Request Process
30+
31+
1. Ensure your code follows the project's coding style
32+
2. Update documentation as necessary
33+
3. Include tests for new functionality
34+
4. Link any relevant issues in your pull request description
35+
5. Your pull request will be reviewed by maintainers who may request changes
36+
37+
## Development Setup
38+
39+
1. Clone the repository
40+
2. Create a virtual environment: `python -m venv .venv`
41+
3. Activate the virtual environment:
42+
- Windows: `.venv\Scripts\activate`
43+
- Unix/MacOS: `source .venv/bin/activate`
44+
4. Install dependencies: `pip install -r requirements.txt`
45+
5. Run tests: `pytest`
46+
47+
## Testing
48+
49+
All changes should include appropriate tests:
50+
51+
- Unit tests for utility functions
52+
- Integration tests for protocol handling
53+
- End-to-end tests for server interaction
54+
55+
Run the test suite with `pytest` before submitting changes.
56+
57+
## Adding New Features
58+
59+
### Supporting New Protocol Versions
60+
61+
To add support for a new MCP protocol version:
62+
63+
1. Create a new protocol adapter in `mcp_testing/protocols/`
64+
2. Update the test cases to include tests for the new protocol version
65+
3. Update the server implementations to support the new protocol
66+
67+
### Adding New Transport Mechanisms
68+
69+
To add support for a new transport mechanism:
70+
71+
1. Create a new transport adapter in `mcp_testing/transports/`
72+
2. Implement the required interface methods
73+
3. Add tests for the new transport mechanism
74+
75+
## Style Guide
76+
77+
- Follow PEP 8 for Python code
78+
- Use descriptive variable names
79+
- Include docstrings for all modules, classes, and functions
80+
- Keep functions small and focused on a single responsibility
81+
82+
## License
83+
84+
By contributing to this project, you agree that your contributions will be licensed under the project's AGPL-3.0 license.

‎MANIFEST.in

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
include LICENSE.txt
2+
include README.md
3+
include CHANGELOG.md
4+
include CONTRIBUTING.md
5+
include SERVER_COMPATIBILITY.md
6+
include requirements.txt
7+
include .flake8
8+
9+
recursive-include schema *.json
10+
recursive-include specification *.md
11+
recursive-include mcp_testing/server_configs *.json
12+
recursive-include mcp_testing/bin *
13+
14+
recursive-exclude reports *
15+
recursive-exclude .github *
16+
recursive-exclude .venv *
17+
recursive-exclude __pycache__ *
18+
recursive-exclude *.py[cod]
19+
recursive-exclude *$py.class

‎RELEASE_NOTES.md

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# MCP Protocol Validator v1.0.0
2+
3+
## Initial Release
4+
5+
This is the initial public release of the MCP Protocol Validator, a testing framework and reference implementation for the [Model Conversation Protocol (MCP)](https://github.com/microsoft/aimcp).
6+
7+
### Features
8+
9+
- **Protocol Support:** Test MCP servers implementing the 2024-11-05 and 2025-03-26 protocol versions
10+
- **Transport Support:** Test servers using both STDIO and HTTP transports
11+
- **Comprehensive Testing:** Validate server implementations against the protocol specification
12+
- **Reference Implementations:** Includes minimal reference servers for both STDIO and HTTP
13+
- **Detailed Reporting:** Generate comprehensive compliance reports
14+
- **Server Adaptability:** Dynamically adapts to server capabilities
15+
- **Timeout Handling:** Gracefully handles servers that may timeout on certain operations
16+
- **PyPI Package:** Installable via pip for easy integration
17+
18+
### Minimal Reference Servers
19+
20+
The package includes two reference implementations:
21+
22+
1. **STDIO Server:** A minimal MCP-compliant server using standard input/output for transport
23+
2. **HTTP Server:** A fully-featured MCP server that supports both protocol versions over HTTP
24+
25+
### Component Organization
26+
27+
- `mcp_testing/`: Testing framework for validating protocol compliance
28+
- `minimal_mcp_server/`: Reference implementation using STDIO transport
29+
- `minimal_http_server/`: Reference implementation using HTTP transport
30+
- `schema/`: JSON Schema definitions for the MCP protocol
31+
- `specification/`: Protocol specifications and documentation
32+
33+
### Getting Started
34+
35+
Install from source:
36+
37+
```bash
38+
git clone https://github.com/your-username/mcp-protocol-validator.git
39+
cd mcp-protocol-validator
40+
pip install -e .
41+
```
42+
43+
Run basic tests:
44+
45+
```bash
46+
python -m mcp_testing.scripts.basic_interaction --server-command "./minimal_mcp_server/minimal_mcp_server.py"
47+
```
48+
49+
Generate a compliance report:
50+
51+
```bash
52+
python -m mcp_testing.scripts.compliance_report --server-command "./minimal_mcp_server/minimal_mcp_server.py" --protocol-version 2025-03-26 --output-dir "./reports"
53+
```
54+
55+
### Known Issues
56+
57+
- Some servers may timeout when requesting tool lists. Use the `--tools-timeout` parameter to handle these cases.
58+
- See SERVER_COMPATIBILITY.md for details on specific server compatibility.

0 commit comments

Comments
 (0)