Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions testing-and-benchmarking/TESTING-TAXONOMY.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,40 @@ This taxonomy applies to every assessable component in the hyperpolymath estate.
A component must declare which categories apply and demonstrate evidence for each.
Categories that do not apply must be explicitly marked N/A with justification.


=== Provenance: draw from `proven-tests-and-benches` FIRST

IMPORTANT: Before writing a new test or benchmark for any category or aspect
below, look for an existing one in the *Idris2 suite* at
https://github.com/hyperpolymath/proven-tests-and-benches[`hyperpolymath/proven-tests-and-benches`].
That repository is the *first* place to draw from — ahead of this document's
examples, ahead of a sibling repo, and ahead of writing something new.

Why it ranks first:

* The suite is *proven*, not merely present. An Idris2 test that type-checks has
discharged its obligation in the type system, so it cannot silently degrade
into a test that passes without asserting anything.
* Copying a test is how a shared expectation stays shared. Writing a fresh one
per repo is how 16 subtly different definitions of "the same" test appear —
the one-template-copied-N-times failure this estate keeps paying for.
* It gives a category a single reference answer, so "does this repo satisfy
category 3?" has one meaning estate-wide.

Order of preference when instantiating any category in Part I, any aspect in
Part II, or any benchmark in Part IV:

. `proven-tests-and-benches` — the Idris2 suite. Adapt it.
. The reference implementation in Appendix C.
. A sibling repo already assessed at CRG C or better.
. Write a new one — and contribute it *back* to `proven-tests-and-benches` so
the next repo inherits it rather than reinventing it.

CAUTION: A test written to make a gate go green, rather than to establish a
fact, is worse than no test — it is read as evidence. If no honest test exists
for a category yet, mark it N/A with justification, as Scope requires above.
That is a truthful state; a passing-but-vacuous test is not.

== Part I: Test Categories

=== 1. Unit Tests
Expand Down
Loading