Skip to content

feat(enrichers): add ip_to_security_risk — Shodan-backed weighted risk scoring for IP nodes#154

Open
adityaa206 wants to merge 3 commits into
reconurge:mainfrom
adityaa206:feat/ip-security-risk-enricher
Open

feat(enrichers): add ip_to_security_risk — Shodan-backed weighted risk scoring for IP nodes#154
adityaa206 wants to merge 3 commits into
reconurge:mainfrom
adityaa206:feat/ip-security-risk-enricher

Conversation

@adityaa206

@adityaa206 adityaa206 commented Jun 2, 2026

Copy link
Copy Markdown

Summary

  • Adds a new ip_to_security_risk enricher under flowsint-enrichers/src/flowsint_enrichers/ip/to_security_risk.py
  • Takes an Ip node → queries Shodan → emits a RiskProfile node + Port nodes, all linked in the graph
  • Adds shodan>=1.31,<2.0 to flowsint-enrichers dependencies

What problem does this solve?

Flowsint already excels at mapping entity relationships (domains, IPs, social accounts, etc.), but it has no native way to score the security danger of an IP node. An investigator looking at an IP in the graph has no immediate signal of how risky it is.

This enricher fills that gap: it enriches an IP with a 0–100 risk score, a critical / high / medium / low classification, CVE IDs, exposure-surface summary, compliance risk flags, and mitigation recommendations — all surfaced as a RiskProfile node linked directly to the IP.

Scoring model

Adapted from RedFlag, an open-source M&A cybersecurity due-diligence tool, the formula combines four weighted factors:

score = (cvss_norm × 0.35)        ← how severe are the CVEs?
      + (exposure  × 0.25)        ← is this internet-facing?
      + (sensitivity × 0.25)      ← data classification (always UNKNOWN — no asset inventory)
      + (exploit   × 0.15)        ← are CVEs weaponised?
      × evidence_multiplier        ← how confident is the data?
Factor Value Score
Exposure Internet-facing (HTTP/SSH/RDP/DB ports) 100
Exposure Unknown 50
Exploit status Public CVE linked by Shodan 65
Exploit status No CVE data 30
Evidence CORRELATED — CVEs + port confirmed ×0.95
Evidence INFERRED — host found, no CVEs ×0.85

Risk level bands: critical ≥ 75 · high ≥ 50 · medium ≥ 25 · low < 25

Graph output

(Ip) -[HAS_RISK_PROFILE]-> (RiskProfile)
(Ip) -[HAS_PORT]->         (Port)          ← one per open Shodan port

The RiskProfile node carries:

  • overall_risk_score (0–100), risk_level, assessment_date
  • vulnerabilities (CVE IDs), risk_factors, attack_vectors
  • exposure_surface (org, ASN, hostnames, open ports)
  • compliance_risks (PCI-DSS / GDPR / ISO 27001 flags)
  • mitigation_strategies, confidence, source = "Shodan"

Required vault secret

Key Where to get it
SHODAN_API_KEY https://account.shodan.io — 1 credit per api.host() call

Test plan

  • Add SHODAN_API_KEY to Flowsint vault
  • Open any Ip node in the investigation graph
  • Run enricher ip_to_security_risk on it
  • Verify RiskProfile node appears linked via HAS_RISK_PROFILE
  • Verify Port nodes appear linked via HAS_PORT
  • Confirm score and risk level are plausible for the IP under test
  • Confirm a Shodan-rate-limited or invalid key returns a graceful log warning, not a crash

Files changed

File Change
flowsint-enrichers/src/flowsint_enrichers/ip/to_security_risk.py New — full enricher implementation
flowsint-enrichers/pyproject.toml Added shodan>=1.31,<2.0 dependency

Adds a new IP enricher that queries the Shodan API and produces a
weighted risk score (0–100) for any IP node in the investigation graph.

Scoring model (adapted from RedFlag, an open-source M&A due-diligence tool):
  score = (cvss_norm × 0.35) + (exposure × 0.25) + (sensitivity × 0.25)
          + (exploit × 0.15)   × evidence_multiplier

Key behaviours:
- Exposure inferred from open-port service names (HTTP, SSH, RDP, databases, …)
- Exploit status derived from Shodan-linked CVE presence
- Evidence multiplier: CORRELATED (×0.95) when CVEs + port confirmed,
  INFERRED (×0.85) for host-only data
- CVSS floored at 6.5 when CVEs are present (prevents under-scoring)
- Risk level bands: critical ≥75, high ≥50, medium ≥25, low <25
- Compliance risk flags for PCI-DSS / GDPR / ISO 27001

Graph output per IP:
  (Ip) -[HAS_RISK_PROFILE]-> (RiskProfile)
  (Ip) -[HAS_PORT]->         (Port)           ← one node per Shodan port

Requires vault secret: SHODAN_API_KEY (1 credit per ip.host() call).
Added shodan>=1.31,<2.0 to flowsint-enrichers dependencies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@adityaa206

Copy link
Copy Markdown
Author

I am making another project kind of related to this so was interested in yours too but yea look into it if it works

@dextmorgn dextmorgn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great addition ! Have you tested it ?

adityaa206 and others added 2 commits June 2, 2026 18:37
58 tests across 6 classes covering all aspects of the enricher:

- TestRiskLevel: threshold bands (critical/high/medium/low)
- TestComputeRiskScore: formula correctness, weight isolation,
  internet-facing vs internal, evidence multiplier effects
- TestBuildRiskProfile: CVE extraction, CVSS floor at 6.5 when CVSS=0,
  exposure detection via service name and well-known port numbers,
  exploit/evidence classification, risk factors, compliance risks
  (PCI-DSS / GDPR / ISO 27001), mitigations, exposure surface metadata,
  temp _shodan_ports attribute lifecycle
- TestScan: missing API key, missing shodan package, happy path,
  multiple IPs, Shodan APIError skip, generic exception skip,
  partial failure resilience, correct api.host() call args
- TestPostprocess: node/relationship creation (HAS_RISK_PROFILE,
  HAS_PORT per port), temp attribute cleanup, None graph service
  graceful fallback, None input_data fallback
- TestEnricherMetadata: name/category/key/InputType/OutputType,
  input/output schema types, required_params, params_schema,
  vault secret type/required flags, documentation content,
  ENRICHER_REGISTRY registration
- TestVaultIntegration: API key vault resolution, missing key raises
  on async_init, user-provided vault ID takes priority

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously, any CVE where Shodan returned CVSS=0 or null was silently
scored as 6.5 — a hardcoded guess that made CVE-2014-0160 (Heartbleed,
7.5) and a trivial info-disclosure CVE score identically.

Fallback chain is now:
  1. Shodan-provided score (when > 0) — no network call needed
  2. NVD API v2 lookup (services.nvd.nist.gov/rest/json/cves/2.0)
     CVSSv3.1 > CVSSv3.0 > CVSSv2 in priority order
  3. 6.5 conservative default — only if both above fail

Changes to the enricher:
- Add _fetch_nvd_cvss(cve_id, api_key) — queries NVD, handles 404,
  retries once on 429 after a 30s sleep, returns None on all failures
- Add _resolve_cvss_scores(raw_vulns, nvd_api_key) — builds the full
  {cve_id: score} map; sleeps between NVD calls to respect rate limits
  (6.1s/req unauthenticated, 0.7s/req with key)
- scan() now calls _resolve_cvss_scores before _build_risk_profile and
  reads NVD_API_KEY from the vault (optional)
- _build_risk_profile(address, host, cvss_per_cve, max_cvss) accepts
  pre-resolved scores — no longer touches raw Shodan cvss fields
- Risk factors now show real per-CVE scores (top 3 by severity) instead
  of a single max value
- Add optional NVD_API_KEY vault param with rate-limit context in desc
- Update documentation to explain the 3-step fallback chain

Tests updated/added (74 tests total):
- TestFetchNvdCvss: CVSSv31/v30/v2 priority, 404, empty response,
  API key in header, no-key path, ConnectionError, 429 retry success,
  429 retry failure
- TestResolveCvssScores: Shodan score used, NVD called for zero/null/
  missing, fallback to 6.5, mixed sources, empty vulns, NVD key
  forwarded, delay called per NVD lookup, no delay for Shodan scores,
  max_cvss reflects highest resolved
- TestBuildRiskProfile: updated signature, NVD score used correctly,
  fallback score used, no more floor logic, real per-CVE scores shown
- TestScan: _resolve_cvss_scores patched out, NVD key forwarded, no-NVD-
  key path
- TestVaultIntegration: NVD key resolved, NVD key optional (no raise)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@adityaa206

Copy link
Copy Markdown
Author

yes I did, I also made some edits after reviewing the code again and checked for default values which I was using to test out stuff and edited it and now its fixed and yea. Please test it out yourself if you have time. Thank youuu. Let me know if anythng wrong and needs to be fixed

@adityaa206

Copy link
Copy Markdown
Author

About the API btw, its free and it doesnt use the credits for the scans so yea its free

@adityaa206 adityaa206 requested a review from dextmorgn June 3, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants