test(tools): comprehensive test suite for submit_cves, tool_output_logger, and verify_exploit helpers - #93
Merged
Conversation
…gger, and verify_exploit helpers
This was referenced Jul 5, 2026
This was referenced Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
tests/test_submit_cves_and_logger.py— 63 fully-mocked unit tests covering three modules that had virtually zero functional test coverage:tool_output_loggersubmit_cvesverify_exploit(helpers)Suite delta: 902 → 965 passing (+63), 0 failures.
What's tested
tool_output_logger.log_tool_output_size(13 tests)str()sizing)contentkey → no exceptionNoneresult → no exceptiontextvalue → no exceptionjsonfield with unserializable value → no exceptioncontentis not a list → no exceptionsubmit_cves(30 tests)Input validation:
cve_list→ error status + messagecve_listkey → error statustoolUseIdpreserved in error responseMissing URL (documents actual behavior):
CVE_SUBMIT_URLenv + no config webhook URL →ValueError("webhook URL …")propagatesconfig.tomlSuccessful submission:
requests.postcalled with correct URLtoolUseIdpreserved on successHTTP errors:
HTTPError→ error statusConnectionError→ error statusRuntimeError) → error statustoolUseIdpreserved on errorCritical CVE filtering:
analyze_affected_assets; MEDIUM + LOW excludedanalyze_affected_assetscalled with empty listCVE_SUBMIT_URLenv used when config has no webhook attributeTOOL_SPEC contract:
name == "submit_cves"inputSchema.jsonpresentrequiredincludescve_listverify_exploithelpers (20 tests)_truncate_text:MAX_LOG_LINES(keeps last N lines)MAX_LOG_CHARS(keeps last N chars)[truncated …]prefix_resulthelper:status == "success", innerverification_statusset)exploit_output.exit_code == -1target_logsdefaults to""target_logspassed through when providederrorfield defaults toNoneerrorfield populated when provided_error_objhelper:category,stage,retryable,exception_type,message)retryable=Trueround-tripsexception_typeincludes module path separatorModule contracts:
TOOL_SPEC.name == "verify_exploit", callabledockerfile_content,exploit_code,cve_idMAX_LOG_LINESandMAX_LOG_CHARSare positiveChecked against existing open PRs — no overlap
Confirmed the following open PRs cover different tools; none touch
submit_cves,tool_output_logger, orverify_exploithelpers:#51 (silent-patches), #53 (cve-timeline), #54 (version-range), #58 (vendor-response), #60 (poc-freshness), #64 (sbom-scan), #65 (temporal-priority), #67 (cluster-variants), #74 (epss-decay), #75 (exploit-maturity), #76 (vulnerability-triage), #77 (cve-report), #78 (diff-report), #79 (reachability), #80 (epss-watchlist), #82 (attack-surface), #83 (watch-alert), #85 (CLI integration tests), #86 (patch-lag), #87 (kev-context), #88 (readme-scoring), #89 (core-tools: check_cisa_kev/get_nvd_data/get_cwe_details/get_otx_cve_details), #90 (exploit-search-tools: search_exploit_db/search_packetstorm/query_threat_intel/obtain_cves), #91 (nvd-retry-backoff), #92 (github-tools: get_github_advisory/search_for_exploits)
Test design notes
submit_cvesURL-missing tests document the actual behavior (uncaughtValueError) rather than an idealized one, serving as a living regression guard if the behavior changes_truncate_texttests usemonkeypatchon module-level constants so they stay fast and deterministic regardless ofMAX_LOG_LINES/MAX_LOG_CHARSvalues