Skip to content

test(tools): comprehensive test suite for submit_cves, tool_output_logger, and verify_exploit helpers - #93

Merged
manus-use merged 1 commit into
manus-use:mainfrom
manus-pi:feat/submit-cves-test-suite
Jul 6, 2026
Merged

test(tools): comprehensive test suite for submit_cves, tool_output_logger, and verify_exploit helpers#93
manus-use merged 1 commit into
manus-use:mainfrom
manus-pi:feat/submit-cves-test-suite

Conversation

@manus-pi

@manus-pi manus-pi commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds tests/test_submit_cves_and_logger.py — 63 fully-mocked unit tests covering three modules that had virtually zero functional test coverage:

Module Before After
tool_output_logger 0 tests 13 tests
submit_cves 1 importability test 30 tests
verify_exploit (helpers) 3 infra-error tests 20 tests

Suite delta: 902 → 965 passing (+63), 0 failures.


What's tested

tool_output_logger.log_tool_output_size (13 tests)

  • Text content logging (size in chars)
  • JSON content logging (str() sizing)
  • Mixed text + JSON content (sizes summed)
  • Empty content list → no output
  • Missing content key → no exception
  • Non-dict result → no exception
  • None result → no exception
  • Multiple text items → sizes correctly summed
  • Content item with unknown key → no exception
  • Non-string text value → no exception
  • json field with unserializable value → no exception
  • content is not a list → no exception
  • Tool name appears in output

submit_cves (30 tests)

Input validation:

  • Empty cve_list → error status + message
  • Missing cve_list key → error status
  • toolUseId preserved in error response

Missing URL (documents actual behavior):

  • No CVE_SUBMIT_URL env + no config webhook URL → ValueError("webhook URL …") propagates
  • ValueError message mentions config.toml

Successful submission:

  • Single CVE → success status
  • Success message contains CVE ID
  • Multiple CVEs → all POSTed, count in message
  • requests.post called with correct URL
  • toolUseId preserved on success

HTTP errors:

  • HTTPError → error status
  • Error message contains "error"
  • ConnectionError → error status
  • Unexpected exception (RuntimeError) → error status
  • toolUseId preserved on error

Critical CVE filtering:

  • CRITICAL + HIGH CVE IDs passed to analyze_affected_assets; MEDIUM + LOW excluded
  • No CRITICAL/HIGH → analyze_affected_assets called with empty list
  • CVE_SUBMIT_URL env used when config has no webhook attribute

TOOL_SPEC contract:

  • name == "submit_cves"
  • inputSchema.json present
  • callable
  • required includes cve_list

verify_exploit helpers (20 tests)

_truncate_text:

  • Short text returned unchanged
  • Empty string returned unchanged
  • Line truncation at MAX_LOG_LINES (keeps last N lines)
  • Char truncation at MAX_LOG_CHARS (keeps last N chars)
  • Within-limits text has no [truncated …] prefix
  • Exactly at limit → not truncated
  • One over limit → truncated

_result helper:

  • Correct structure (outer status == "success", inner verification_status set)
  • exploit_output.exit_code == -1
  • target_logs defaults to ""
  • target_logs passed through when provided
  • error field defaults to None
  • error field populated when provided

_error_obj helper:

  • All fields present (category, stage, retryable, exception_type, message)
  • retryable=True round-trips
  • exception_type includes module path separator

Module contracts:

  • TOOL_SPEC.name == "verify_exploit", callable
  • Required fields include dockerfile_content, exploit_code, cve_id
  • MAX_LOG_LINES and MAX_LOG_CHARS are positive

Checked against existing open PRs — no overlap

Confirmed the following open PRs cover different tools; none touch submit_cves, tool_output_logger, or verify_exploit helpers:

#51 (silent-patches), #53 (cve-timeline), #54 (version-range), #58 (vendor-response), #60 (poc-freshness), #64 (sbom-scan), #65 (temporal-priority), #67 (cluster-variants), #74 (epss-decay), #75 (exploit-maturity), #76 (vulnerability-triage), #77 (cve-report), #78 (diff-report), #79 (reachability), #80 (epss-watchlist), #82 (attack-surface), #83 (watch-alert), #85 (CLI integration tests), #86 (patch-lag), #87 (kev-context), #88 (readme-scoring), #89 (core-tools: check_cisa_kev/get_nvd_data/get_cwe_details/get_otx_cve_details), #90 (exploit-search-tools: search_exploit_db/search_packetstorm/query_threat_intel/obtain_cves), #91 (nvd-retry-backoff), #92 (github-tools: get_github_advisory/search_for_exploits)


Test design notes

  • 100% mocked — no real HTTP calls, no Docker daemon required
  • submit_cves URL-missing tests document the actual behavior (uncaught ValueError) rather than an idealized one, serving as a living regression guard if the behavior changes
  • _truncate_text tests use monkeypatch on module-level constants so they stay fast and deterministic regardless of MAX_LOG_LINES/MAX_LOG_CHARS values

@manus-use
manus-use merged commit 4a09e33 into manus-use:main Jul 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants