Repository navigation
Disclose AI assistance and record human verification #14
Description
Model descriptions and implementations in the library may be drafted with help from AI. This issue adds two indicators to each one: how much AI was involved in producing it, and whether a person has verified it. Both describe process. Neither claims that a document is free of errors, and neither ranks documents.
AI assistance: a level from 0 to 4, shown as a four-segment bar.
| Level | Meaning |
|---|---|
| 0/4 | No AI was used in the text or code. |
| 1/4 | AI-assisted editing: people wrote every passage, and AI suggested wording, grammar or consistency fixes. |
| 2/4 | AI-assisted drafting: people wrote most of it; AI drafted some passages, which people rewrote. |
| 3/4 | AI-drafted, human-directed: AI drafted most of it to an outline and decisions from people, who edited it throughout. |
| 4/4 | AI-drafted with light human editing. |
Three rules keep the levels consistent:
- The level describes how the text and code were produced. Choices about the model, the calibration and the results always belong to the authors.
- Report the highest level that applies to any substantial part, such as one section or the code, so one AI-drafted section isn't averaged away.
- List what AI was used for, from a fixed list: drafting, editing, code, derivation checks, literature, figures.
Authors state the level in their pull request, and the reviewer confirms it as part of the review process from #6.
Human verified means a named person has read the document end to end and checked it against its sources. It is recorded with the person's name, the date and the version they verified. When a new version comes out, the indicator shows "not yet verified" until someone verifies that version. This is worked out from the recorded version, so nobody has to remember to clear it.
Display
- The AI bar uses one neutral colour and varies only how many segments are filled. A red-to-green scale would read as "more AI means less trustworthy", which is the quality judgement these indicators avoid. The bar also has to survive greyscale printing.
- Verification follows the site's existing slot style: a solid blue border for verified for this version, a dashed grey one for not yet verified. Both always carry text, so neither relies on colour alone.
- Both indicators appear in the PDF's version box from Version each model description and let implementations cite the version they follow #13, on each description's web page from Publish each model description as a web page alongside its PDF #12, and in each slot of the site's library table. Each links its level to the definitions.
Where it is recorded
| What | Where |
|---|---|
| The definitions: levels, rules and display | A section of a root CONTRIBUTING.md, next to the versioning rules from #13. Each model's submission guide links to it. |
| A model description's level and verification | The front matter of models/<model>/model-description.md: ai.level, ai.used_for, verified.by, verified.date and verified.version. |
| An implementation's level and verification | projects/<toolkit>/<model>/metadata.yml, with the same keys. |
The values live in the files they describe, so every change to them is reviewed, and each version's tag preserves that version's values.
Proposal: make a minimal metadata.yml required. #5 makes the reported numbers required but the files optional. For the site to show each implementation's version, AI level and verification automatically, those fields need a fixed place. Every implementation would include a metadata.yml with model_description_version, authors, ai and verified, and every other field would stay optional. This revises design decision 3 of #5 and the submission guide, so it needs agreement before it is built.
This is planned to land together with #12 and #13, after #5 merges.
Activity
mmcky commentedon Oct 20, 2026
Decided on 20 October 2026:
metadata.yml. The required fields aremodel_description_version,authors,aiandverified; every other field stays optional.metadata.ymlin the repository, a CI check that names any missing or invalid field, and a copyable example in the submission guide.metadata.ymlon the submitter's pull request. The site is static, so a GitHub form provides the web submission and sign-in without a server.metadata.yml. That rule will be written when the first such submission arrives.