A framework for testing AI-built code. Organization-portable, plain-English, and grounded in published research. It composes with your existing controls; it does not replace them.
One-line thesis: A passing test suite does not prove code is correct — and that gap is wider when an AI wrote the code, because the AI tends to write tests that confirm what the code does rather than what it should do. This framework closes that gap with independent checks, measurable gates, and evidence — scaled to risk.
The name points at the core idea: trust comes from an independent oracle (a source of the right answer that is not the code under test), and the gate is where you prove you have one.
AI now writes a large share of new code. That shifts the hard part from writing code to trusting it. The old habit — "tests pass and coverage is high, ship it" — was always weak; with AI-written code it is actively misleading, because the same model often writes both the code and the tests that judge it. There is, as of this writing, no widely-adopted standard devoted specifically to testing AI-generated code. This framework is a synthesis of established methods into that missing layer.
It is written to be used by a lead, not just a coder: you can apply it to require the right evidence and ask the right questions, even if you do not read every line — though competent code, security, and domain review remain mandatory at the higher risk tiers.
1.1 Coverage is not fault-detection. Coverage measures whether a line ran, not whether a bug in it would be caught. A test can execute every line and assert nothing. Never accept coverage as proof of quality.
1.2 The oracle problem (circularity) — the core. A test needs an oracle: an independent source of the right answer. When the same AI writes both the code and its tests, the test tends to encode what the code does, not what it should do — baking in the very bug it should catch. The builder and the inspector share the same blind spots.
This is measured, not asserted. Tests generated after a model wrote faulty code detect 14% of faults; tests generated independently of the implementation detect 25%. On real-world code, tests written against incorrect code have a 47% worse bug-detection rate — and supplying a natural-language description of intended behavior (an independent oracle, added back) recovers +34%. Independently, LLM test-generation tools have been shown to reject bug-revealing tests because their oracles are "designed to pass." Three groups, three methods, same direction. These studies were conducted independently of this framework and two of them predate it; see EVIDENCE.md for citations and the provenance note.
1.3 AI-amplified and AI-specific failure modes. Most are not unique to AI — AI raises their frequency or makes them harder to spot:
- Insecure code, written confidently. Under specific benchmark conditions, studies have found a large share of AI-generated code (often cited around 40%) contains a security weakness — treat as directional, not a fixed rate for your code.
- Genuinely AI-specific — hallucinated dependencies (slopsquatting): importing packages that do not exist, repeating the same fake names (which attackers can register).
- Tests that encode the bug — weak/wrong assertions, tests that re-run code without checking it.
- Reproduced training-data bugs.
- Genuinely AI-specific — agents that make a test "pass" by weakening it: deleting, skipping, loosening assertions, blindly updating snapshots, or narrowing generated-input ranges.
Each layer answers a different question. Use several; judge each on fault-detection, not coverage.
| Layer | The question it answers |
|---|---|
| Static / security scanning | Well-formed and free of known-bad patterns? (cheapest) |
| Example tests (unit/integration) | Right on the cases we wrote? (coverage measures only this) |
| Property tests | Does a rule that must always hold survive many generated inputs? |
| Metamorphic tests | When there's no single right answer, do relationships between outputs hold? (a partial oracle) |
| Mutation testing | If the code were subtly broken, would our tests fail? (tests the tests) |
| Fuzzing / differential / contract | What input breaks it? Do two implementations agree? Did the interface change? |
Mutation testing deserves emphasis: a tool injects small defects ("mutants") and checks whether a test catches them; the percent caught is the mutation score. It is the strongest evidence that tests can detect faults — evidence, not proof, and it is weak for concurrent / distributed / numerical-tolerance / nondeterministic code.
Classify each deliverable by the highest applicable trigger, then apply the gates at that tier. Triggers (not vibes): sensitive data (PII/PHI/regulated); external exposure; an auth/security boundary; money above a threshold; safety impact; decisions affecting people (employment, credit, housing, education); production write access; autonomous action.
- Tier 1 — Low: internal, non-sensitive, no decision impact.
- Tier 2 — Moderate: some data sensitivity or reversible impact.
- Tier 3 — High: sensitive data, external exposure, or decisions about people/money.
- Tier 4 — Critical: safety-, security-, or rights-affecting; regulated decision logic.
So that "meets the framework" is objective, conform to a named profile (requirement words per RFC 2119: MUST / SHOULD / MAY).
This framework ships no default score thresholds, deliberately. No published threshold has an evidentiary basis — the common "aim for 80–90%" figures are practitioner heuristics, not tool guidance (see EVIDENCE.md). A threshold also invites Goodhart's law: a mutation score can be raised by writing tests, or by arguing that survivors are equivalent, and only one of those is free. The gate is surviving mutants, not a percentage. Adopters MAY declare local floors, and the reference tool enforces whatever they declare.
- Baseline (Tier 1–2): coverage reported (a disclosure, never a target and never a gate); mutation testing SHOULD run on core logic, with every undetected mutant killed or explained; an independent oracle for non-trivial logic; static analysis + secret scanning MUST pass; human review SHOULD occur.
- Regulated (Tier 3): Baseline items become MUST; mutation testing MUST run with no unexplained surviving mutants in critical code; independent oracle with traceability; full security + supply-chain gates; human code review; AI-use provenance; the evidence package; and where output affects people, fairness/adverse-impact analysis.
- Safety-or-Security-Critical (Tier 4): Regulated plus deference to the applicable domain standard (see Scope & Limitations), independent verification, and two-person review.
An explanation is not a score adjustment. It is a signed statement, by a named human, that a specific mutant cannot be killed — bound to the source file's digest, so it expires the moment that file changes. See QUICKSTART.md.
State conformance as, e.g., "meets The Oracle Gate, Regulated profile" with the evidence package as proof.
Conformance is checkable, not self-declared. The reference tool (oracle-gate conformance) validates
a filled evidence package against the profile it claims. It enforces three things, in rising order of
rigor — none of which is a claim the software is correct:
- Completeness. Every required gate has real evidence attached (a report for machine gates; an approver + artifact + date for human gates) or a valid time-boxed waiver.
- Declared value meets declared floor. Metric gates (coverage, mutation score) declare a numeric value; the checker fails a gate whose declared value is absent or below its profile floor. This closes "attachment theater" — attaching a 12%-mutation-score report no longer reads as conformant. It attests the declared number clears the bar; whether that number is true is still human review.
- Version binding. The package carries a
commit_sha, agenerated_attimestamp, and a SHA-256 for each attached artifact. The checker fails if the commit is not an ancestor ofHEAD, or if any artifact's hash doesn't match the file on disk — so a stale or borrowed evidence package can't pass forever against code it never described.
It does not, and cannot, attest that the software is correct, nor that the declared values are true — substance still requires human review. Completeness, declared floors, and provenance are machine-enforced; substance stays human. (Without this distinction, "we conform" would be the very self-declaration the framework rejects.)
Applied at the deliverable's tier/profile.
Foundation
- G0 — Approved requirements & traceability. Testing cannot repair a wrong or missing requirement (AI often faithfully implements a bad prompt). Acceptance criteria are approved; Tier 3–4 traces requirement → test/oracle → result → release.
Test-quality
- G1 — Coverage is a floor, never the target.
- G2 — Fault-detection is shown. Mutation testing (or an approved equivalent where infeasible) runs on changed and impacted code against a protected config; meets the profile minimum; survivors triaged; equivalents/exclusions need independent concurrence and are reported; raw and adjusted scores recorded.
- G3 — Independent oracle (independence established, not assumed). Expected answers come from approved requirements, an independent reference, a domain-expert-owned golden dataset, or a metamorphic relation — never from the same model's code, or the same reasoning that wrote it.
- G7 — Tests are protected. Review any deletion, skip, xfail, snapshot update, assertion loosening, fixture/mock change, generator-range narrowing, or coverage/mutation-exclusion edit.
Supply-chain & security
- G4 — Real supply-chain control. Not "the package exists": vulnerability (SCA) scan, SBOM, license scan, lockfiles/pinning, allowlist or internal mirror, dependency-confusion controls, and (Tier 3–4) provenance/signature checks.
- G4b — Security gates by tier. Static analysis + secret scanning always; add threat modeling, DAST, secure-config, and targeted review of authz / input validation / deserialization / tenant isolation / abuse cases for Tier 3–4 and exposed systems.
Provenance & accountability
- G5 — Cross-model review (advisory, constrained). A different, independent model adversarially reviews the change; findings are triaged by a human, not auto-accepted — an aid, not a substitute for review. Never send sensitive or controlled data to an unapproved model. Treat output as advisory; do not chase it to zero findings.
- G6 — Human accountability. Humans own requirements, acceptance, security posture, and the release decision; AI generates, humans approve. Human code review is required for Tier 3–4.
- G9 — AI-use provenance. Record what tools/models produced the code, tests, and specs — captured technically where feasible, not by memory alone.
AI/model components
- G8 — Trustworthy evals. A versioned eval suite, stress-tested so it can fail, reported with error bars. For people/decision-affecting models: intended/prohibited-use statements, dataset lineage and rights, predeclared metrics with sample-size/power, subgroup and adverse-impact analysis with confidence intervals, calibration, drift monitoring, and a contestability path. Judges: a single expert MAY calibrate low-risk evals; Tier 3–4 needs multiple raters/adjudication and an appropriate agreement metric (weighted kappa / Krippendorff's alpha / Gwet's AC1) plus bias checks.
Evidence & exceptions
- G10 — Evidence package. Test + mutation reports (config, survivors), oracle traceability, security/supply-chain scans + SBOM, AI-provenance log, code-review record, eval report where relevant.
- G11 — Waivers, explicitly. Anything short of profile requirements ships with a named, time-boxed waiver: residual-risk rating, owner, approver authority, compensating control, and revalidation date. Never silently.
Enforcement: enforce deterministic gates structurally (branch protection, required checks, CODEOWNERS, least-privilege CI) rather than by instruction — an instruction to an AI is not a control.
Adopt this alongside — not instead of:
- NIST SSDF (SP 800-218) — secure development (G0/G4/G4b/G6/G10).
- NIST AI RMF + Generative AI Profile — AI risk / TEVV (G8 evals, G3 oracles).
- ISO/IEC 42001 (AI management) and ISO/IEC/IEEE 29119 (the testing baseline this extends).
- Supply-chain: SLSA, SBOM (SPDX/CycloneDX), OWASP dependency guidance (G4).
Adopters instantiate — set your own thresholds and map gates to your own controls.
Being explicit here is a credibility feature, not a weakness.
- It supplements, never replaces, domain safety standards — DO-178C (avionics), IEC 62304 (medical), ISO 26262 (automotive), IEC 61508 / EN 50128. Safety-critical systems MUST follow those.
- It does not by itself satisfy regulatory or contractual regimes (e.g., privacy law, sector compliance, export control). Regulated adopters MUST map gates and evidence to their obligations.
- It does not cover AI coding-agent security (sandboxing, tool permissions, prompt-injection via repository content) beyond the G5 data-handling constraint — that is an adjacent discipline.
- It is a control pattern to be tailored locally, not a turnkey certification.
- Adopt the tiers and a classification rubric.
- Pick a conformance profile and set your thresholds.
- Wire the deterministic gates into CI structurally (not as prompts).
- Require the evidence package and provenance log on every change; keep waivers explicit and time-boxed.
- Pilot on one real project, measure (escaped defects, survivor classes, flaky-rate, waiver age, gate bypasses), then scale.
The framework was first exercised end to end on a real, public module — the ranking-metrics code in
graph-guard (evaluate / lift):
- 6 hand-written example tests; it looked well-tested.
- Mutation testing (G2) showed 9 of 112 mutants survived — nine silent ways the math could break. Coverage had hidden this completely (Principle 1.1, demonstrated).
- Property tests ("scores stay in [0,1]") + metamorphic tests ("reordering inputs must not change totals") + targeted tests killed all 7 killable survivors; 2 were reviewed and documented as equivalent mutants.
- Scores, honestly: raw 103/112 = 92.0% → 110/112 = 98.2%; adjusted (excluding the 2 reviewed equivalents) = 110/110 = 100%.
- No functional bug was found in the metric code under the tested properties and examples — the weakness was test inadequacy, not a code defect. Without Parts 1–5 you would have trusted a suite that could not catch a bug.
- Coverage — how much code ran during testing; silent on whether bugs are caught.
- Oracle — the independent source of the correct answer a test checks against.
- Mutation testing — deliberately breaking code to see if tests notice; the score grades the tests.
- Equivalent mutant — a break that changes no real behavior, so no test can catch it; expected.
- Property test — a rule that must always hold, thrown many generated inputs.
- Metamorphic test — checks a relationship between related inputs' outputs; a partial oracle.
- Eval — a repeatable scorecard for an AI/model component.
- Slopsquatting — attackers registering the fake package names AI models hallucinate.
- SBOM / SCA — a bill of materials for dependencies / scanning them for known vulnerabilities.
Claims are backed by cited, confidence-labeled sources in EVIDENCE.md.
Provenance of this framework: drafted by one AI model, adversarially reviewed across two rounds by an independent, different-lineage model, and reconciled against the cited sources by a human author who accepted some findings and deliberately scoped others out. That is the framework's own method (G5), applied to itself.
The framework text is licensed CC BY 4.0 — adopt it, adapt it, please credit it. (Toolkit code added later will carry an MIT license alongside it.) See LICENSE.
Author: Jeff Otterson · Contributions and adoption reports welcome via issues.