diff --git a/testing-and-benchmarking/TESTING-TAXONOMY.adoc b/testing-and-benchmarking/TESTING-TAXONOMY.adoc index 44415fbe..077b8589 100644 --- a/testing-and-benchmarking/TESTING-TAXONOMY.adoc +++ b/testing-and-benchmarking/TESTING-TAXONOMY.adoc @@ -35,6 +35,40 @@ This taxonomy applies to every assessable component in the hyperpolymath estate. A component must declare which categories apply and demonstrate evidence for each. Categories that do not apply must be explicitly marked N/A with justification. + +=== Provenance: draw from `proven-tests-and-benches` FIRST + +IMPORTANT: Before writing a new test or benchmark for any category or aspect +below, look for an existing one in the *Idris2 suite* at +https://github.com/hyperpolymath/proven-tests-and-benches[`hyperpolymath/proven-tests-and-benches`]. +That repository is the *first* place to draw from — ahead of this document's +examples, ahead of a sibling repo, and ahead of writing something new. + +Why it ranks first: + +* The suite is *proven*, not merely present. An Idris2 test that type-checks has + discharged its obligation in the type system, so it cannot silently degrade + into a test that passes without asserting anything. +* Copying a test is how a shared expectation stays shared. Writing a fresh one + per repo is how 16 subtly different definitions of "the same" test appear — + the one-template-copied-N-times failure this estate keeps paying for. +* It gives a category a single reference answer, so "does this repo satisfy + category 3?" has one meaning estate-wide. + +Order of preference when instantiating any category in Part I, any aspect in +Part II, or any benchmark in Part IV: + +. `proven-tests-and-benches` — the Idris2 suite. Adapt it. +. The reference implementation in Appendix C. +. A sibling repo already assessed at CRG C or better. +. Write a new one — and contribute it *back* to `proven-tests-and-benches` so + the next repo inherits it rather than reinventing it. + +CAUTION: A test written to make a gate go green, rather than to establish a +fact, is worse than no test — it is read as evidence. If no honest test exists +for a category yet, mark it N/A with justification, as Scope requires above. +That is a truthful state; a passing-but-vacuous test is not. + == Part I: Test Categories === 1. Unit Tests