Skip to content

test(tools): comprehensive test suite for get_github_advisory and search_for_exploits - #92

Merged
manus-use merged 1 commit into
mainfrom
feat/github-tools-test-suite
Jul 6, 2026
Merged

test(tools): comprehensive test suite for get_github_advisory and search_for_exploits#92
manus-use merged 1 commit into
mainfrom
feat/github-tools-test-suite

Conversation

@manus-use

Copy link
Copy Markdown
Owner

Summary

Adds tests/test_github_tools.py — a 66-test suite covering get_github_advisory
and search_for_exploits, both of which power steps 4 and 7 of the manus-agent analyze
pipeline via the VI agent but had zero functional tests in the repository.


Why these two tools?

Both tools query the GitHub API and are registered directly in vi_agent.py:

  • get_github_advisory — fetches GitHub Security Advisories (GHSA) for a CVE;
    used at step 4 of the VI pipeline
  • search_for_exploits — searches GitHub for PoC exploit repositories for a CVE;
    used at step 7 alongside search_exploit_db and search_packetstorm

Before this PR:

  • get_github_advisory had 1 importability test (in test_packageability.py) and no functional coverage
  • search_for_exploits had 1 importability test (in test_packageability.py) and no functional coverage

What is tested

get_github_advisory

  • Contract: module importable, function callable, @tool decorator compatibility
  • Input validation: non-CVE strings, empty string, None, integer, GHSA-format IDs
  • Success path: first advisory returned, summary/severity/cve_id/ghsa_id fields,
    only the first entry returned when multiple advisories are present, lowercase cve- prefix
    normalisation, GitHub token injected into Authorization header when GITHUB_TOKEN is set
  • Not-found: empty list → message key; HTTP 404 → message key (not error)
  • Error paths: ConnectionError, Timeout, HTTP 500, HTTP 403, empty-dict response structure
  • VI agent wiring: vi_agent.py imports get_github_advisory; system prompt references GitHub

search_for_exploits

  • TOOL_SPEC contract: name, description, required fields, cve_id type, toolUseId echo
  • Input validation: invalid ID, empty string, None, integer, missing key → status: error
  • Success path: status: success, links present, capped at ≤5 results, required fields
    (name, url, stars, description, last_updated) in each entry, summary field with
    total_count, toolUseId echo, GitHub token injected, correct API endpoint, CVE ID in query
  • No results: status: success with links: [] and appropriate summary
  • Error paths: ConnectionError, Timeout, HTTP 403 (rate-limit), HTTP 422, JSON decode error
  • VI agent wiring: vi_agent.py imports search_for_exploits; system prompt references exploits

Test quality

  • 100% mocked — no real HTTP calls; uses @patch("...requests.get") throughout
  • All 66 tests pass in isolation and in the full suite (902 → 968 passing, 0 failures)
  • Handles env-variable isolation correctly (token tests use monkeypatch.setenv)
  • ruff check . --fix && ruff format clean

Open PRs checked (no overlap confirmed)

Checked against all open PRs before building to confirm no duplication:

#51 (silent-patches tool), #53 (cve-timeline), #54 (version-range), #58 (vendor-response),
#60 (poc-freshness), #64 (sbom-scan), #65 (temporal-priority), #67 (cluster-variants),
#74 (epss-decay-detector), #75 (exploit-maturity-classifier), #76 (vulnerability-triage-card),
#77 (cve-report-generator), #78 (diff-report), #79 (reachability-scorer),
#80 (epss-watchlist), #82 (attack-surface-scorer), #83 (watch-alert),
#85 (cli-integration-tests), #86 (patch-lag-rating), #87 (kev-context-dimension),
#88 (readme-scoring-workflow), #89 (core-tools-test-suite for check_cisa_kev/get_nvd_data/get_cwe_details/get_otx_cve_details),
#90 (exploit-search-tools-tests for search_exploit_db/search_packetstorm/query_threat_intel/obtain_cves),
#91 (nvd-retry-backoff)

PR #89 covers check_cisa_kev, get_nvd_data, get_cwe_details, get_otx_cve_details.
PR #90 covers search_exploit_db, search_packetstorm, query_threat_intelligence_feeds, obtain_cves.
This PR covers get_github_advisory and search_for_exploits — neither appears in any open or merged PR.

@manus-use manus-use left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant