Skip to content

development.md

Latest commit

 

History

History
1369 lines (1164 loc) · 70.3 KB

development.md

File metadata and controls

1369 lines (1164 loc) · 70.3 KB
Raw

Development Setup

This guide describes a seed development machine setup for working on Consent Scheme. The repository is still early, so the required toolchain is intentionally small and will grow as implementation tickets land.

Required Tools

  • Git
  • GitHub CLI, gh
  • Emacs, preferably a current stable release
  • GNU Make
  • ripgrep, rg, for fast repository searches

Optional but useful:

  • Chibi Scheme, chibi-scheme, for optional portable R7RS Chibi checks and the reference implementation oracle runner
  • Gauche, gosh, for additional reference implementation oracle coverage
  • Guile, guile, and Sagittarius, sagittarius, for broader optional oracle comparison coverage
  • Racket, racket, plus its r7rs package for developer oracle comparisons; install the package with raco pkg install --auto r7rs
  • CHICKEN Scheme, csi, plus its r7rs egg for developer oracle comparisons; install the egg with chicken-install r7rs
  • Gambit Scheme, gsi and gsc, for developer oracle comparisons and host-compiled portable executable checks; Homebrew packages it as gambit-scheme
  • ShellCheck or other local lint tools for future scripts

GitHub Access

Authenticate the GitHub CLI before working with issues or pull requests:

gh auth status

If authentication is missing:

gh auth login

Confirm the repository remote:

git remote -v

Clone and Branch

Clone the repository and create a topic branch for each issue:

git clone git@github.com:tahoma/consent.git
cd consent
git switch -c author-name/issue-N/short-name

Use a short contributing author name as the branch prefix no matter which tools you use.

Read First

Before implementing a ticket, read:

The GitHub roadmap issue is the source of truth for dependency ordering. Start with dependency-free or explicitly unblocked issues.

Editing Expectations

  • Keep canonical runtime concepts represented as Scheme-readable data.
  • Treat the portable R7RS implementation as a first-class peer of the Emacs Lisp bootstrap, and as the long-term path toward self-hosted or native reader, evaluator, emitter, and REPL work.
  • Treat portable R7RS Scheme as the default home for host-neutral behavior: semantic helpers, protocol datums, codecs, deterministic parsers, library surfaces, fixtures, and tests.
  • Keep the dual-core implementation surface as small as possible. Preserve architectural parity between Emacs Lisp and portable Scheme modules only for the irreducible reader, evaluator, macro, primitive, and host-effect adapter slices that cannot yet be single-sourced. If a slice lands on only one side, record the remaining parity work and why it is not yet portable Scheme before calling the issue complete.
  • Keep Emacs-specific behavior behind host adapter modules.
  • Avoid project history or personal machine details in public docs, tests, and examples.
  • Follow Scheme Style Guidelines for portable Scheme definition shape, docstrings, and rich metadata layout.
  • Follow the commit-message rules in Contributing.

Unicode data generation

The portable character implementation consumes the pinned Unicode 17.0.0 UCD inputs under vendor/unicode/17.0.0/. Regenerate the checked-in Scheme table library after changing those inputs, their hashes, or the generator:

make update-unicode-data
make check-unicode-data

The update target uses the required Emacs runtime, verifies all input SHA-256 hashes, and writes scheme/consent/unicode-data.sld. The check target performs the same transformation in memory and fails if the checked-in output differs. Build and test targets never download Unicode data; updating the pinned release is an explicit reviewed repository change. Follow Portable Character Model and Unicode Profile for the UCD source files, fallback policy, default non-Turkic casing choice, license, and upgrade procedure.

Exhaustive Unicode semantic check

Run the representation-independent Unicode oracle after changing generated tables, their lookup representation, or (scheme char) behavior:

make check-unicode-semantics
CONSENT_UNICODE_SEMANTIC_HOST=gambit make check-unicode-semantics

The portable program observes the exported (consent unicode-data) query API consumed by (scheme char) for all 1,112,064 Unicode scalar values in ascending order. It covers the four owned classification properties, decimal value, simple upper/lower/fold mappings, and full string upper/lower/fold mappings. Surrogate code points are excluded because they are not Unicode scalar values. Querying the owned data API directly prevents a test host's own (scheme char) implementation and Unicode release from entering the oracle. The ordinary character and conformance suites separately cover the public wrapper's ASCII fast paths, character/string conversions, and full-map glue. The data queries include the corresponding ASCII results, so the digest also fixes the semantic values those fast paths must preserve.

The program writes a canonical binary stream, and the wrapper compares its byte count and SHA-256 digest with the checked Unicode 17.0.0 expectation. The stream contains no generated-table representation details, so representation changes pass when exported semantics remain identical. The target has no wall-clock assertion. It is deliberately outside the default make test loop and joins the opt-in make test-full and exhaustive CI lanes instead.

Schema 1 begins with the ASCII bytes for Consent Unicode semantics, followed by NUL, byte 1, and NUL. Each ascending scalar record contains:

  • the scalar as three big-endian bytes;
  • one flag byte whose low four bits are Alphabetic, Uppercase, Lowercase, and White_Space in that order;
  • one decimal byte, using 255 when there is no decimal value;
  • the simple uppercase, lowercase, and foldcase scalars as three big-endian bytes each; and
  • each full mapping as a one-byte scalar count followed by three big-endian bytes per mapped scalar.

The scalar itself is encoded in every record, so omission, duplication, or ordering drift changes the digest as well as a property or mapping change. For the same Unicode release, treat a digest change as a semantic change: validate it against an independent representation or UCD-derived oracle before updating the checked expectation. A Unicode upgrade updates the pinned inputs, version metadata, semantic expectation, and review evidence together.

Unicode performance benchmark

Use the opt-in same-process benchmark to compare Unicode import and lookup costs across revisions on the same machine under equivalent checkout conditions:

make benchmark-unicode
CONSENT_UNICODE_BENCHMARK_ITERATIONS=500 \
  CONSENT_UNICODE_BENCHMARK_IMPORT_ITERATIONS=5 make benchmark-unicode

The command emits one Scheme-readable consent-benchmark record per metric. It measures the first (scheme char) import, warm imports into fresh contexts, and persistent ASCII and BMP classification. Simple case-mapping measurements cover a BMP hit, an occupied-region identity miss, an empty BMP-region identity miss, and a supplementary-plane hit; the final metric exercises full string upcasing. Each record includes stable metric and schema names, elapsed and per-iteration seconds, and garbage-collection counts. The benchmark has no pass/fail wall-time threshold and is not part of make test or CI; compare equivalent runs instead of treating timings from unlike machines as regressions.

Emacs Lisp Docstrings

Checked-in Emacs Lisp implementation docstrings under lisp/ must fit within the byte-compiler docstring width limit. make lint-elisp-docstrings enforces this source-level rule directly, and make lint-elisp runs it before the warnings-as-errors byte-compile gate. The source-level check deliberately avoids generated byte-compiler docstrings, such as cl-defstruct constructor signatures whose width can vary across Emacs releases.

When fixing a warning, put point inside the docstring and use M-q (fill-paragraph) in emacs-lisp-mode. Emacs fills docstrings with emacs-lisp-docstring-fill-column; setting it to 79 keeps source docstrings comfortably under the project limit while preserving a single string literal.

Scheme Source Comments

Portable Scheme comments carry the API and invariant documentation that future contributors need while editing .sld and .scm files. Runtime-visible documentation belongs to the body literal convention in Docstring Metadata Convention when a binding needs metadata that standard readers, reflection, reference tools, or compiled runtimes can preserve. Comments remain source-only and are not visible through ordinary R7RS reading. Broader Scheme formatting and definition-shape rules live in Scheme Style Guidelines.

  • Start each portable Scheme file with a ;;; header that names the library or source file responsibility and the host/core boundary it belongs to.
  • Put a leading ;; comment before top-level Scheme define-record-type, define-syntax, and plain data define forms. For procedure definitions, a simple string docstring supersedes a leading summary comment. Add a separate source comment only when it describes an invariant, policy, pass boundary, or portability concern that does not belong in the runtime docstring. Section comments may supplement per-binding comments, but they do not replace needed per-binding documentation.
  • For record types, document ownership of the record shape, any mutable fields, and whether the record is part of the public Consent Scheme datum surface or an internal implementation record.
  • For macros, document hygiene assumptions, literal identifiers, private marker syntax, and the target form or pass that receives the expansion.
  • Comment primitive/kernel boundaries, policy or capability assumptions, compiler/backend assumptions, include/load paths, and other places where a small local change would affect runtime authority or portability.
  • Keep comments concise for tiny R7RS helpers whose names and surrounding source fully state most of the contract. Do not add line-by-line narration for simple selectors, wrappers, or local loops.
  • Keep Scheme comments public-repo safe: avoid project history, personal machine paths, secrets, transcripts, and non-project branding.

These comment rules are enforced by consent-scheme-documentation-test-source-comments (in the test-emacs-tools shard), which fails any top-level (define ...) in scheme/, tests/scheme/, or fixtures/r7rs/ .scm/.sld files that lacks a leading ;; comment or a procedure docstring -- including plain value bindings such as (define program-input-stream-pulls 0), not only procedures. Only the Emacs ERT doc-lint enforces this; running the file directly under a host Scheme (Chibi, Guile) passes it, so a missing comment surfaces only in the Emacs-tools shard.

Runtime-visible documentation for public procedures belongs in a simple string docstring in the procedure body, using the convention in Docstring Metadata Convention. Add docstrings to new exported public procedures in checked-in Scheme libraries when the procedure body form supports them. Do not keep a leading ;; comment that only restates the docstring. Simple string docstrings do not document macros, record fields, library forms, renamed exports, or plain data bindings; those surfaces need future metadata records rather than a placeholder procedure docstring. Primitive bindings do not have reader-visible bodies, so public primitive documentation belongs in their manifest metadata. New public primitive manifest entries should include concise user-facing documentation and rely on implementation procedure docstrings only as fallback for internal or generated hooks; fallback reflection marks the origin as (implementation-procedure string).

Every exported procedure in the runtime scheme/ tree must carry the rich property record described in Docstring Metadata Convention: a #((parameters ...) (returns ...) (effects ...)) vector following the simple string docstring. Public parameter and return descriptors must either use the dotted string/string-list shorthand for intentional any or include both explicit (type ...) metadata and a non-empty (description ...) in expanded descriptors. Missing type metadata still normalizes to any at read time, but the project lint treats expanded public API descriptors without a type or description as incomplete documentation. The lint also rejects shorthand or expanded (type any) when descriptor prose names an obvious primitive type; use explicit types in those cases. Custom type names should normally follow local predicates by dropping the trailing ?, while tagged datums without predicates should use structural types such as list. The effects field is also required and must contain at least one symbol. Use pure when the procedure has no observable effect; otherwise name allocation, state access, callback invocation, errors, or host effects explicitly. Expanded descriptors should keep (type ...) on the same line as the parameter or returns head when the line fits within the soft line limit, with the description on the following line. Keep longer type forms on their own line. The consent-scheme-documentation-test-public-rich-docstrings gate enforces this fail-closed -- it runs over every runtime scheme/ source file by default rather than an opt-in allowlist, so a newly added file is covered automatically and any exported procedure missing the record fails the test. There is no rich-docstring exemption path; a file with no exported procedures already passes, and exported runtime procedures are part of the introspectable public surface.

Test Layout

Project tests live under tests/ and run through the repository Makefile. Emacs Lisp bootstrap tests use ERT and follow these conventions:

  • test files are named tests/consent-*-test.el
  • module tests mirror implementation modules, such as tests/consent-reader-test.el for lisp/consent-reader.el
  • shared ERT helpers should live in tests/consent-test-helper.el and provide consent-test-helper
  • the batch runner is tests/consent-test-runner.el

The runner starts Emacs with -Q --batch, adds project-local lisp/ and tests/ directories to load-path, loads test files in deterministic order, and does not load user Emacs configuration.

Future R7RS conformance fixtures should plug into make test through the same test command instead of adding a second top-level verification path.

Portable R7RS tests live under tests/scheme/ and are launched directly by tools/run-portable-tests.sh; ERT is not part of their execution path. The aggregate local targets run the full suite under Gambit, Racket with its r7rs package, Guile, Gauche, and Chibi. CI schedules Chibi as a required aggregate host and the longest direct and compiled hosts as first-class semantic plan shards (runtime, evaluator, integration, agent, library, random, and property, or their compiled counterparts), so failures and timings compare the same behavior surfaces instead of one host-sized total. The local make test-portable-chibi target uses chibi-scheme on PATH, or the command named by CONSENT_CHIBI, and skips when Chibi is unavailable.

Portable test bodies use SRFI 64 through (stdlib testing) as their result engine, with SRFI 252 property tests and SRFI 78/SRFI 42 table checks where they improve coverage. (testing harness) supplies only the suite lifecycle, batch failure, Scheme-readable summary, and SRFI 78 adapter. The scheme/testing/ namespace contains reusable portable testing libraries and is part of the runtime manifest index, so downstream users can import the harness. Executable Consent test suites and cases remain ordinary programs under tests/scheme/, outside the manifest. (stdlib ...) remains reserved for standards-derived libraries. Additional test libraries should be named for the missing facility they provide under (testing ...).

(testing plan) owns validated, Scheme-readable multi-program test plans. The project plan in tests/scheme/test-plan.scm records program paths, semantic and scheduling tags, and named shards as composable selectors. The plan remains project test data outside the runtime manifest; only the reusable plan facility is manifested. tools/run-portable-tests.sh asks (testing runner) to resolve that plan and then supplies the irreducible host-specific process invocation. R7RS provides (scheme load), but separate processes remain a deliberate test isolation policy rather than a language limitation.

tools/run-portable-test-set.sh is the local aggregate adapter: it launches the same named plan shards concurrently and retains one log per semantic group. CI invokes make test-portable-shard once per group for the historically long Guile, Gauche, Gambit-compiled, and Racket-compiled lanes. Compiled build jobs publish a freshly linked bin/consent artifact first; the downstream group matrix downloads that exact product instead of recompiling it in every test job. Scheduled noncanonical source-metadata/docstring combinations retain the full cross through aggregate shard-set jobs.

Every ordinary full program is also classified as either compiled or self-host-gap, never both. The compiled selector is evidence-based: a program joins it only after passing under both the Gambit-compiled and Racket-compiled self-host runners. A gap is not an accepted alternate tier; it is a living runtime-conformance defect that must name an implementation issue and leave the plan when that issue ships. testing-plan-test.scm enforces the exhaustive partition and guards the compiled program count against silent shrinkage.

(testing registry) supplies ERT-style named case registration, tags, composable selectors, failed-case reruns, explicit source locations, per-case timing through an injectable clock, and Scheme-readable inspection records. Its diagnostic hook lets an Emacs, CLI, or other host attach native backtraces without putting non-R7RS stack APIs in portable test code.

(testing runner) is the developer-facing batch layer. Registered suite programs call testing-runner-main with (command-line) and consequently support --list, --select SELECTOR, --verbose, --report FILE, and --rerun-failed FILE. Selectors are Scheme data: (all), (name NAME), (tag TAG), and recursively composed (and ...), (or ...), and (not ...) forms. A normal run prints a concise summary, preserves assertion-level SRFI 64 result properties in its report, records per-case timing and diagnostics, and exits zero for success, one for test failure, or two for runner/configuration failure. For example:

guile --r7rs -L scheme tests/scheme/consent-context-test.scm \
  --select '(tag property)' --report context-test-report.scm
guile --r7rs -L scheme tests/scheme/consent-context-test.scm \
  --rerun-failed context-test-report.scm

Hosts that do not expose trailing program arguments through R7RS command-line can provide the same string list as a Scheme datum in TESTING_RUNNER_ARGUMENTS; this is the portable host-adapter fallback. testing-runner-plan-main is the corresponding multi-program entry point used by the thin launcher; it reads a (testing plan) datum and emits the selected program paths for one named shard. Host-specific execution semantics remain explicit in plan data: the live model checks use live-direct for R7RS hosts that enter the Consent evaluator and live-compiled for self-hosted binaries that already execute inside a Consent interaction context.

Host-neutral semantics must have canonical portable Scheme tests unless a documented host boundary makes that impossible. Core runtime, reader, evaluator, macro, library, and standard-library changes should therefore add or update tests/scheme/ coverage first. The Scheme runner and thin host launcher own those files; ERT continues to own Emacs adapter behavior such as buffers, windows, commands, and prompts. Every ERT file must have an ownership entry in tests/scheme/ert-portable-parity-map.scm. Mixed source-backed ERT cases must name a portable case/check marker or a concrete Emacs-only boundary. consent-test-case-parity-audit-test.el rejects unmapped files, incomplete mixed-case partitions, stale portable markers, and portable programs absent from the Scheme-native test plan.

The current placement audit and justified ERT-only categories are recorded in docs/portable-test-audit.md.

The multi-host bootstrap strategy in docs/multi-host-bootstrap.md defines what belongs in portable Scheme modules versus host adapter modules. New host-neutral runtime or library behavior should gain portable fixtures where practical before a host adapter exposes it through editor, process, filesystem, model, or persistence capabilities.

The local R7RS-small report reference lives in docs/r7rs-small-report.md. The active R7RS-small conformance matrix lives in docs/r7rs-conformance.md. The canonical shared fixture corpus lives in fixtures/r7rs/conformance-cases.scm as an consent-fixture-suite. Fixture records carry id, kind, phase, category, section, status, oracle, options, source, expect, and description fields so the Emacs Lisp harness, portable Scheme harness, and conformance runner select from the same indexed cases. Fixtures marked pending, policy-gated, or unavailable are loaded and validated by ERT without being executed. Fixtures marked implemented must run through make test.

Fixtures may also carry optional oracle-eligibility and oracle-reason fields when a reference implementation should not run the case. The current eligibility values are policy-gated and not-oracle-eligible. Reasons include host-policy, agent-specific, resource-limit, agent-result-record, implementation-dependent, and unspecified.

A second consent-fixture-suite corpus, fixtures/repl/parity-cases.scm (kind repl-parity), is the host-neutral conformance corpus for the cross-host REPL interaction contract (docs/repl-interaction-contract.md). Its cases carry session, options, input, and an expect record sequence rather than the reader/evaluator fields above. Two parallel runners drive the same cases against both REPL hosts: tests/scheme/consent-repl-parity-test.scm (portable terminal REPL, in the shared host-suite file list, so it runs on every host shard including Chibi) and tests/consent-repl-parity-test.el (Emacs incremental REPL). Because both read one corpus, a host that drifts from the contract fails its runner.

Reference Oracle

Pure shared R7RS conformance fixtures can also be compared with external Scheme implementations through the oracle runner:

make conformance-oracle

The default reference adapters are Chibi Scheme and Sagittarius. Gauche, Guile, Racket, CHICKEN, and Gambit remain opt-in comparison adapters so contributors can inspect a wider implementation matrix before changing defaults. The runner uses CONSENT_CHIBI, CONSENT_GAUCHE, CONSENT_GUILE, CONSENT_SAGITTARIUS, CONSENT_RACKET, CONSENT_CHICKEN, and CONSENT_GAMBIT when set, otherwise it searches for chibi-scheme, gosh, guile, sagittarius, racket, csi, and gsi on PATH. The Racket adapter requires Racket's separate r7rs package and wraps generated fixture programs with #lang r7rs. The CHICKEN adapter requires the r7rs egg and invokes csi with -q -R r7rs -s. The Gambit adapter invokes gsi with -:r7rs,search=$REPO/scheme, where $REPO/scheme is the repository's portable R7RS library directory. Each adapter writes eligible fixtures to a temporary R7RS program and invokes the reference implementation with that file as the command-line program argument. Missing reference implementations are reported as unsupported-reference in Scheme-readable oracle reports and do not affect the default make test command.

Adapter Role Environment override Discovered command Notes
Chibi Scheme default CONSENT_CHIBI chibi-scheme Required CI host.
Sagittarius default CONSENT_SAGITTARIUS sagittarius Runs with -r 7 for R7RS mode.
Gauche opt-in comparison CONSENT_GAUCHE gosh Useful for library and writer behavior comparisons.
Guile opt-in comparison CONSENT_GUILE guile Runs with --no-auto-compile --r7rs.
Racket developer-only comparison CONSENT_RACKET racket Requires the Racket r7rs package; generated programs are wrapped with #lang r7rs.
CHICKEN Scheme developer-only comparison CONSENT_CHICKEN csi Requires the r7rs egg; runs with -q -R r7rs -s.
Gambit Scheme opt-in comparison and compile host CONSENT_GAMBIT gsi Homebrew formula gambit-scheme; gsc builds the standalone binary.

The default host-compiled portable runtime uses the same R7RS mode and library search stance as the Gambit interpreter shard. Set CONSENT_GAMBIT_COMPILER to choose a specific gsc executable; otherwise compile checks discover gsc on PATH. The oracle runner does not invoke gsc, but documenting both tools keeps interpreter and compiler setup aligned.

Oracle reports identify each fixture by case id and classify the comparison as portable-agree, implementation-variant, agent-mismatch, unsupported-reference, policy-gated, or not-oracle-eligible. The runner intentionally skips Consent Scheme-specific result fixtures, resource-limit fixtures, and host-effecting R7RS libraries such as (scheme file), (scheme load), (scheme process-context), (scheme repl), and (scheme time). It also skips fixtures whose result depends on whether a reference command reads a file as a strict R7RS program or as REPL input from a file, since R7RS permits the latter mode to accept import declarations outside the program prefix. The target is report-oriented; inspect agent-mismatch reports as conformance investigation signals.

The oracle normalizes narrow reference writer spelling variation when the same R7RS value is otherwise clear, such as Chibi's doubled plus sign in complex NaN outputs. It does not collapse semantic distinctions such as exact versus inexact numbers.

implementation-variant reports are intentionally visible. Treat them as portability notes rather than failures when Consent Scheme agrees with at least one supported reference and the remaining references differ among themselves. Current expected sources include exact versus inexact numeric results, special NaN and infinity spellings, optional reader support for datum labels in program source, bytevector port optional-argument behavior, reference-specific library loading behavior, and case-folding quirks in developer-only references. Add output normalization only for narrow writer aliases that preserve the same R7RS datum. Add oracle-eligibility metadata only when the reference command cannot exercise the same language mode as the fixture, not merely because implementations disagree.

To focus the report stream, pass a comma-separated status filter:

CONSENT_ORACLE_STATUSES='agent-mismatch,implementation-variant' make conformance-oracle

To compare a chosen reference implementation set, pass a comma-separated reference filter:

CONSENT_ORACLE_REFERENCES='chibi,gauche,guile,sagittarius,racket,chicken,gambit' make conformance-oracle

To print a compact status count before the report stream:

CONSENT_ORACLE_SUMMARY=1 make conformance-oracle

Host-Compiled Portable Executables

make compile builds executable artifacts from the portable R7RS runtime by using external Scheme host compiler toolchains. This path packages the current portable implementation through mature host compilers; it is not the future Consent Scheme LLIR/native compiler backend tracked by #115 through #121.

The default compile host is Gambit, whose gsc -exe -nopreload produces a standalone native executable with no runtime dependency — the artifact suitable for make install:

make compile

Select a host explicitly with CONSENT_COMPILE_HOST:

CONSENT_COMPILE_HOST=gambit make compile
CONSENT_COMPILE_HOST=racket make compile

The Racket binary (raco exe --cs) is relocatable as a file but loads boot files from the installed Racket, so it runs only where Racket is present (see the Installing section's caveat).

The Racket path requires both racket and raco; override discovery with:

CONSENT_RACKET=/path/to/racket CONSENT_RACO=/path/to/raco make compile

The Gambit path requires both gsi and gsc; override discovery with:

CONSENT_GAMBIT=gsi CONSENT_GAMBIT_COMPILER=gsc CONSENT_COMPILE_HOST=gambit make compile

Generated outputs stay under build/compile/<host>/ by default:

  • bin/consent: the host-compiled executable artifact
  • src/: generated host wrapper sources and, for Gambit, the mirrored portable .sld sources plus generated C files used for linking
  • collections/: generated host dependency wrappers when a host needs them, currently the Racket path
  • manifest.scm: Scheme-readable artifact manifest
  • logs/: compiler, compile-timing, and smoke-test logs

CI caches only the non-runnable intermediate subdirectories that speed up a rebuild (src/, incremental/, and Racket collections/ as applicable). It does not cache bin/consent: a dedicated build job relinks and smoke-tests the product binary from the current checkout, then publishes that run-scoped binary as an artifact consumed by the compiled Scheme-native shard matrix.

Use CONSENT_COMPILE_BUILD_DIR to place those generated files elsewhere:

CONSENT_COMPILE_BUILD_DIR=/tmp/consent-compile make compile

The build runs version and scalar-evaluation smoke commands against the executable before reporting success:

build/compile/<host>/bin/consent --version
build/compile/<host>/bin/consent --eval '(+ 1 2)'

The build then writes a temporary pure script containing a function definition and assertion and runs it with --script. Separate temporary probes verify that --eval, --script, and bare-path execution deny an ungranted file write. The smoke also runs a script by bare path (consent FILE, equivalent to consent --script FILE) and as an executable /bin/sh polyglot, exercising the shebang-handling boundary end to end. See executable-scripts.md for how to write and run an executable Consent Scheme script.

Remove generated compile artifacts with:

make clean-compile

Architecturally, this pipeline is a compiler front-end with a borrowed code-generation backend, and the embedded source store is a capability-addressable virtual filesystem underlay; see host-compiled-staging.md.

Installing

make install puts the host-compiled binary on PATH. It installs the binary built by make compile for the current CONSENT_COMPILE_HOST; it does not build it for you, because install is commonly run under sudo and a build step would leave root-owned artifacts in the source tree. The supported flow is:

make compile
sudo make install

install follows the GNU installation variables:

sudo make install PREFIX=/usr/local                 # default; binary -> /usr/local/bin/consent
make install DESTDIR=/tmp/stage PREFIX=/usr/local   # stage under a packaging root
make install bindir=$HOME/.local/bin                 # user-local, no sudo

DESTDIR is prepended to every install path (the packaging/staging convention); PREFIX (default /usr/local), bindir (default $(PREFIX)/bin), mandir (default $(PREFIX)/share/man), and datadir (default $(PREFIX)/share) select the final locations. After staging the binary, install runs the staged consent --version and fails unless it matches version.sld, catching a non-executable staging path or a Racket binary installed where Racket is absent.

install also lays the runtime-provided Consent Scheme library tree (the base prelude, syntax prelude, and source-libraries) under the versioned $(datadir)/consent/$(version). The binary carries an embedded copy of these as a zero-dependency floor, so it runs standalone even when relocated; the installed tree lets you inspect or override them. At startup the binary resolves runtime libraries in order: CONSENT_LIBRARY_PATH (colon-separated, an explicit override), then the install datadir baked in at compile time, then the source tree (in a checkout), then the embedded floor. So an installed or overridden tree wins over the embedded copy; a bare copied binary still works via the floor. (Keep PREFIX consistent between make compile and make install so the baked datadir matches the install location, or point CONSENT_LIBRARY_PATH at the installed tree.)

consent --script FILE (and a #!/usr/bin/env consent shebang) runs the file through the Consent interpreter under the non-interactive fail-closed posture — not the host Scheme — at parity with the Emacs consent-script-run-file; see executable-scripts.md.

If no binary has been built, install exits with guidance instead of silently doing nothing:

make clean-compile && make install   # exits 2: "run `make compile` first"

The host is chosen with CONSENT_COMPILE_HOST, which defaults to Gambit. The Gambit binary (gsc -exe -nopreload) is a standalone native executable: relocatable, with no runtime dependency, so the default install flow produces an artifact that works on a machine without a Scheme toolchain. The build uses Gambit's built-in eq?-hash behind Consent's identity adapter, while table storage and policy remain portable Scheme. The hot graph registry therefore has no external module dependency. Its startup smoke hides the build host's Gambit module tree, so any accidentally unlinked dependency fails before the standalone artifact is published:

make compile        # CONSENT_COMPILE_HOST defaults to gambit
sudo make install

The Racket binary (raco exe --cs) is relocatable as a file but still loads boot files from the installed Racket: it runs only while Racket remains installed on the same machine. (Bundling a full Racket runtime tree with raco distribute is out of scope.) The post-install --version check is what turns a Racket-without-Racket install into a hard failure rather than a runtime surprise.

A consent(1) man page is installed to $(mandir)/man1/consent.1 only when one exists at docs/consent.1 (CONSENT_MANPAGE). That page is generated by the documentation pipeline (#458); until it lands, install prints a notice and skips the man page. Override the source path with CONSENT_MANPAGE=....

Remove an installed binary (and man page) with the matching variables:

sudo make uninstall                                 # removes /usr/local/bin/consent (+ man page)
make uninstall DESTDIR=/tmp/stage PREFIX=/usr/local

uninstall removes exactly the paths install writes — the binary, the man page, and the versioned $(datadir)/consent/$(version) library tree — and is idempotent; per GNU convention it does not remove shared directories such as bindir.

Distribution

make dist packages the compiled binary into a versioned tarball under $(CONSENT_DIST_DIR) (default build/compile/dist/), named from version.sld and the compile host (for example consent-<version>-gambit.tar.gz). The tarball contains the binary, its manifest.scm, README.md, LICENSE, the runtime library tree under share/consent/<version>/, and the man page when present:

CONSENT_COMPILE_HOST=gambit make compile
CONSENT_COMPILE_HOST=gambit make dist
ls build/compile/dist/

Released binaries are published by the tag-triggered release workflow: pushing a v* tag builds the standalone Gambit binary, runs the smoke and native test gates, verifies a staged install, and uploads the make dist tarball as a GitHub Release asset. Only the standalone Gambit binary is published, for the Racket-needs-Racket reason above.

Verification

The default local verification command is:

make test

make test runs a trimmed default shard set for a fast local loop: one representative portable host (test-portable-racket) plus its reflection contract and catalog stress companions (test-portable-racket-reflect and test-portable-racket-reflect-stress, CONSENT_DEFAULT_PORTABLE_TEST_SHARD_TARGETS) and the Emacs-hosted shard set in CONSENT_EMACS_TEST_SHARD_TARGETS. The portable reader/writer/docstring machinery that the source-metadata and docstring-retention modes exercise is host-independent, so one portable host is enough for the default loop.

The Emacs-hosted surface is split across multiple shards (#556) so the parallelizer can overlap them on hosts with more cores than there were once shards: test-emacs-tools keeps the tools and docs cluster (CI, compile, diagnostics, doc-pass tests, ...), while test-emacs-integration carries the heavier integration surface (REPL, VCS, native-CLI daemon) that used to dominate test-emacs-tools's wall time. Reflection runs through its own contract shard (test-emacs-reflect) plus catalog, documentation/apropos, binding crosswalk, and dynamic manifest stress shards so manifest-backed discovery does not serialize the REPL/integration lane. The aggregate test-emacs-reflect-stress target remains available for ad hoc local runs. The four full host-compile + install/dist tests in consent-compile-portable-test.el are stranded into the opt-in test-emacs-native-build shard, which make test skips and make test-full runs. The native build path is already exercised separately by test-portable-gambit-native and test-portable-compiled; the test-emacs-native-build shard additionally covers the install/dist packaging surface against a single shared host build per host (built once per Emacs process, reused across the runner-smoke and install/dist tests). CONSENT_EMACS_TEST_SHARD_TARGETS and the Emacs-hosted CI matrix list the longest expected wall-time shards first. That ordering is a scheduling hint so constrained runner pools start tail rows early; it does not change the behavior surface each shard covers.

The former library/conformance aggregate is likewise only a compatibility target for ad hoc local runs. Default and CI execution split its exact 212-test union into fixture/conformance, library runtime, standard-library behavior, random/property libraries, and standard-library manifest/vendor shards. These names intentionally align with the portable plan's behavior surfaces wherever the two implementations exercise comparable semantics.

The official stdlib reference corpus has canonical portable coverage in tests/scheme/stdlib-json-reference-test.scm, including the FoundationDB JSON sample, invalid inputs and their explicit exclusions, implementation-defined fixture classifications, JSON Lines, and JSON Text Sequences. The complete corpus runs through the Gambit- and Racket-compiled self-hosted lanes as the gold-standard proof that the shipped evaluator can load and exercise its own JSON library. SRFI 180 reference semantics no longer run through ERT; the old eight-minute Emacs stress shard and its smaller companion shard were removed. Portable host failures are reported by the named test-portable-* shard and test file; Emacs adapter/bootstrap failures remain reported by the named test-emacs-* shard.

Portable reflection tests follow the same behavior-surface split. Quick manifest-input and helper contracts run in dedicated test-portable-*-reflect shards, while full catalog traversal and dynamic manifest add/remove stress coverage run in dedicated test-portable-*-reflect-stress shards. That keeps Guile and Gauche catalog-rebuild costs visible without folding them into the ordinary host timing row.

make test defaults to -j16 so up to 16 shard processes can run in parallel on a many-core host. Override with CONSENT_TEST_JOBS=N make test on narrower hardware.

The default set also runs test-parity, the cross-implementation parity gate (#374). It runs the shared fixture corpus through both in-repo cores — the Emacs-hosted implementation and the portable R7RS implementation — and fails on any result divergence, turning the docs/architecture.md "First-Class Portable Scheme" parity rule into an executable gate. Scope is the irreducible dual core (reader, evaluator, macro, runtime); cases whose source imports a library that is single-sourced from a portable .sld and loaded by both bootstraps fall out of scope automatically, because such a library cannot diverge from itself. The Emacs bridge spawns the portable emitter (tests/scheme/consent-parity-emit.scm) under the host named by CONSENT_PARITY_HOST (auto-discovered from chibi-scheme, gosh, or guile when unset) and skips when no portable host is available, so the gate is a no-op rather than a failure on a host-free machine.

The default set also runs lint-elisp, the Emacs byte-compile lint gate (#415). It byte-compiles every checked-in lisp/*.el source with byte-compile-error-on-warn enabled, so any byte-compiler warning — unbound variables, arity mismatches, unused lexicals, obsolete calls — fails the build. This is the cheapest static analysis already available for the Emacs Lisp twin and is the Emacs-side counterpart to the portable doc-lint coverage the suite carries (#407/#412). The gate redirects its bytecode into a throwaway CONSENT_LINT_BUILD_DIR (default build/lint) so it leaves no stale .elc beside the sources and never races the parallel test shards that load the .el files directly. Run it on its own with:

make lint-elisp

make compile-elisp remains available as the non-gating target that produces in-place bytecode without warnings-as-errors.

The gate disables the byte-compiler's docstring-width sub-check (byte-compile-docstring-max-column is left unbounded) so it is deterministic across Emacs versions. cl-defstruct auto-generates a constructor docstring whose (fn &key SLOT...) calling-convention line scales with the slot count; Emacs 30 excludes that machine-generated line from the width check while Emacs 29 does not, so a wide struct — notably the consent--eval-context god-object tracked by #371 — would otherwise fail the gate on the Emacs the CI runners install (29.x) but not on a newer local Emacs. Docstring style and width are owned by the separate checkdoc slice, not this gate. That batch checkdoc pass is intentionally not part of the gate yet; it surfaces a large backlog of docstring-convention findings and is tracked as its own follow-up slice.

The default set also runs lint-portable, the portable twin of lint-elisp (#421). It compiles the host-loadable portable Consent Scheme libraries under Guile with the high-signal static warning classes — unbound variable, arity mismatch, use-before-definition, unused lexical, and format/case-datum mistakes — promoted to errors, so the portable peer gets the same cheap static coverage the Emacs twin does. Run it on its own with:

make lint-portable

Guile is the gate host because, among the portable hosts already wired into CI (Gambit, Racket, Guile, Gauche), it is the only one with a usable ahead-of-time warning facility — guild compile -W ..., the same -W baseline the linting survey records for Chez/Guile. Gambit reports arity mismatches only at run time, and Racket and Gauche expose no comparable unused/unbound static warning pass, so they are intentionally out of scope for this gate (the issue bounds it to hosts already in CI). The gate drives a generated driver that imports every host-loadable library with a unique prefix; auto-compilation surfaces warnings for each transitively compiled library, and a fresh compile is forced each run so cached bytecode never masks a warning. The driver and bytecode cache live under a throwaway CONSENT_PORTABLE_LINT_BUILD_DIR (default build/lint-portable) that the gate removes when it finishes, so it leaves nothing beside the sources.

Two warning classes are deliberately not gated. unused-toplevel and unused-module fire hundreds of false positives because the project registers internal helpers and primitives through runtime dispatch tables that Guile's per-module static analysis cannot see; they are too noisy to gate cleanly. Four libraries are also out of the gate's reach because they are not host-loadable as pure R7RS — (agent diagnostics), (agent diff), and (agent test) import the runtime-virtual (agent io) module, and (consent capability) imports the host-adapter (consent capability primitive) layer; these are exercised through the runtime instead. The exclusion list in tools/lint-portable.sh is explicit so that a newly added library reaching such a module fails the gate loudly rather than being skipped silently. Like the portable host shards, the gate skips (rather than fails) when Guile is unavailable, so it is a no-op on a Guile-free machine; CI installs Guile so it always runs there.

The default set also runs lint-readability, the repository-wide narrow-width gate. It applies an 80-column soft limit and a 100-column hard limit to first-party Scheme, Emacs Lisp, tests, fixtures, shell tooling, Make recipes, and workflows. Soft-limit exceptions require a local category and rationale; hard-limit exceptions are not supported. Run it and its format-class self-test with:

make lint-readability
tools/lint-readability.sh --self-test

The compatibility target lint-line-length delegates to the same gate. See docs/readability.md for wrapping examples, exception categories, generated-source policy, provenance exclusions, and migration metrics.

The default set also runs lint-branding, the assistant/tool/vendor branding gate. It enforces the AGENTS.md rule that no assistant, tool, vendor, or workflow branding appears in branch names, pull request titles or bodies, commit messages, generated artifacts, or ordinary documentation. The dedicated project credits page is the exception for explicit tool-assisted development credit. The gate machine-checks that boundary rather than leaving it to contributor diligence, the same stance the two compiler lint gates take. Run it on its own with:

make lint-branding

The gate (tools/lint-branding.sh) uses two pattern tiers to keep false positives near zero on this project's own domain language. Self-attribution markers — a co-author trailer naming a model or tool, a "generated with/by " line, a session link, the robot emoji — are scanned everywhere: tracked file contents, commit messages, the PR title/body, and the branch name. Bare vendor or tool slugs are scanned only in those metadata contexts (commit messages, PR title/body, branch name), never in file contents, because the repository legitimately writes some of those words in prose — CLAUDE.md is a checked-in tool-config file, docs/references.md cites external papers by their publishers, Project Credits gives explicit tool-assisted development credit, and "cursor" is this project's own word for the shared stdin cursor. The slug set is therefore deliberately narrow and excludes words with a legitimate technical use here (notably "openai", as in the shipped OpenAI-compatible transport). A third tier scans each commit's author and committer identity (name <email>) — and only that — for the vendor-bot email domains a real contributor never authors from. The author email is what GitHub attributes a commit to, so a vendor "noreply" identity badges the commit as machine-authored on the PR even when the branch name, every commit message, and the PR body are clean. Locally the script scans the tree, the current branch, the origin/main..HEAD range, and that range's commit identities.

CI enforces the gate two ways — a required primary and a fallback:

  • Dedicated Branding workflow (required, primary). .github/workflows/branding.yml runs on pull-request opened, synchronize, reopened, and edited events — the edited type matters because a branding trailer is usually appended to the PR title or body after creation, and without it that change would never be re-scanned. The job checks out full history (fetch-depth: 0) and injects the PR title and body from the event payload (CONSENT_PR_TITLE / CONSENT_PR_BODY) alongside the base and head SHAs and the branch (CONSENT_BRANDING_BASE / CONSENT_BRANDING_HEAD / CONSENT_BRANDING_BRANCH), so every dimension is scanned deterministically: tracked files, branch name, commit messages, commit author/committer identity, and the PR title/body. The job's check (Assistant/tool/vendor branding gate) is a required status check in the "Protect main" ruleset, so a failure blocks merge rather than only reporting red. Two caveats it does not cover: the required check assumes the PR branch carries this workflow file (a branch cut before it landed must rebase onto main for the check to report), and direct pushes to main are not gated by it — though the ruleset makes main pull-request-only, closing that path in practice. Because it edits a file under .github/workflows/, adding or changing it needs a workflow-scoped push.
  • Piggybacked on lint-elisp (fallback, no workflow change). lint-branding is also a prerequisite of lint-elisp, so the always-run lint-elisp job runs the gate even outside a pull request — for example on a push to main. That job checks out shallow, so there it reliably scans tracked file contents (the checked-out tree) and the branch name (GITHUB_HEAD_REF); the commit-range scans (messages and identity) run only when a base ref is present, and the PR title/body are read best-effort from the public REST API unauthenticated, degrading to a loud notice rather than a failure when that read is rate-limited or blocked. This path rides on normal build code the job already executes, so it predates and backstops the dedicated workflow.

Run the exhaustive set — every portable host shard plus every Emacs shard — with the opt-in escape hatch:

make test-full

make test-full runs CONSENT_FULL_TEST_SHARD_TARGETS. You can also override the default set directly, for example to add one more host without running the whole matrix:

CONSENT_TEST_SHARD_TARGETS='test-portable-guile test-emacs-core' make test

Run make test-full (or the matching scheduled CI lane) before landing axis-sensitive changes to the reader, writer, or docstring machinery, since those are the paths the trimmed default no longer fans out across every host.

Set CONSENT_TEST_TARGET_ROOT to keep the current checkout's Makefile and portable test launcher while pointing Scheme host commands at another checkout or archive's scheme/ directory. This is useful for historical timing sweeps that replay a newer harness against an older reader/evaluator implementation:

CONSENT_TEST_TARGET_ROOT=/tmp/consent-old make test-portable-chibi

CI mirrors this trimmed default on the per-push lane (pull_request and push) and keeps the exhaustive run on a separate opt-in lane. The source_metadata × docstring_retention cross is a de-feature smoke test — stripping syntax metadata and docstrings and confirming the runtime still passes. Over the cross's first 29 PRs it never selected a different test list or caught a failure the canonical combo missed, so the per-push lane carries only a fast canary of it (#481):

  • The canonical portable host (Gambit, the test-portable-gambit job) and the canonical Emacs shard (the core language/runtime shard, the test-emacs-core job) each run the canonical on / full combo plus a single fully-stripped off / none de-feature smoke leg per push. The four intermediate combos (on/simple, on/none, off/full, off/simple) run only on the exhaustive lane.
  • The recompile-bound Gambit native shard runs on / full only per push; its de-feature cross is exhaustive-lane only, since recompiling the native executable to re-run an identical ~2 s suite is the cross's largest per-push cost.
  • Every other host and Emacs shard runs only the canonical on / full combo per push.

Every host and every Emacs shard is still represented at least once, so cross-host parity coverage is preserved; only the redundant metadata/docstring fan-out collapses. The exhaustive matrix — every host and shard across all six combos, including the Gambit-compiled cross — runs nightly on the schedule lane and on demand through workflow_dispatch. Trigger it before landing axis-sensitive changes to the reader, writer, or docstring machinery: open the Actions → Test workflow and use Run workflow (workflow_dispatch), or wait for the nightly schedule run. The trimmed jobs (test-portable-gambit, test-emacs-core, test-portable-extra-hosts, test-portable-gauche-hosts, test-emacs-hosted, and the test-parity gate) drive their source_metadata and docstring_retention matrix axes from a github.event_name expression, so those events expand them back to the full cross. The test-parity job (#374) runs the parity gate under Guile as a required check on every lane. Guile, Racket, and the Racket-compiled runner use the bare ubuntu-latest runner; Gauche is packaged only in the Ubuntu 26.04 container and runs in its own matrix job.

The lint-elisp job (#415) runs make lint-elisp on every lane as its own lightweight required check, alongside the license-reuse REUSE/SPDX job. It needs only Emacs, so it is a fast static gate that runs in parallel with the test shards rather than fanning out across the matrix axes. The lint-portable job (#421) is its portable twin: it installs Guile and runs make lint-portable on every lane as an equally lightweight required check, so the portable libraries are gated for compiler warnings on the same per-push cadence.

CI and the local aggregates share Scheme-defined behavior shards so timing and failures stay visible by architectural path. The aggregate targets remain the convenient local entry points:

CONSENT_GAMBIT=gsi make test-portable-gambit
CONSENT_GAMBIT=gsi make test-portable-gambit-reflect
CONSENT_GAMBIT=gsi make test-portable-gambit-reflect-stress
CONSENT_GAMBIT=gsi CONSENT_GAMBIT_COMPILER=gsc make test-portable-gambit-native
CONSENT_PORTABLE_HOST=racket make test-portable-owned-reader-no-host-identity
CONSENT_RACKET=racket make test-portable-racket
CONSENT_RACKET=racket make test-portable-racket-reflect
CONSENT_RACKET=racket make test-portable-racket-reflect-stress
make test-portable-compiled
CONSENT_GUILE=guile make test-portable-guile
CONSENT_GUILE=guile make test-portable-guile-reflect
CONSENT_GUILE=guile make test-portable-guile-reflect-stress
CONSENT_GAUCHE=gosh make test-portable-gauche
CONSENT_GAUCHE=gosh make test-portable-gauche-reflect
CONSENT_GAUCHE=gosh make test-portable-gauche-reflect-stress
make test-emacs-core
make test-emacs-conformance
make test-emacs-library-runtime
make test-emacs-library-stdlib-core
make test-emacs-library-stdlib-property
make test-emacs-library-stdlib-manifest
make test-emacs-agent-control
make test-emacs-agent-reliability
make test-emacs-capability-boundary
make test-emacs-agent-state
make test-emacs-tools
make test-emacs-reflect
make test-emacs-reflect-catalog-stress
make test-emacs-reflect-documentation-stress
make test-emacs-reflect-binding-crosswalk-stress
make test-emacs-reflect-dynamic-manifest-stress
make test-emacs-integration
CONSENT_PARITY_HOST=guile make test-parity

The owned-reader identity-map target runs the focused poisoned-backend regression. Set CONSENT_PORTABLE_HOST to racket, gambit, or guile; the default is Racket. The normal local aggregates include it once, and CI runs it once per available host on the canonical metadata/docstring combination.

The opt-in test-emacs-native-build shard runs the four full host-compile + install/dist tests when invoked directly (make test-emacs-native-build) or through make test-full; the trimmed make test skips it.

make test runs those shard targets in parallel by default. make test-portable remains available as the local aggregate for the default portable R7RS hosts. CI records one log and check per semantic group for the historically longest Guile, Gauche, Gambit-compiled, and Racket-compiled suites; the compiled group jobs consume product binaries built earlier in the workflow. Gambit and Racket direct-host aggregates emit the same per-group logs, allowing the timing summary to compare equivalent plan selectors even where the CI job remains host-aggregated. The required Chibi CI host uses the same aggregate target that remains available for local timing and compatibility checks:

CONSENT_CHIBI=chibi-scheme make test-portable-chibi

The full-suite host shards run the same portable Scheme test files so their timing rows compare host behavior rather than different test scopes. Each host aggregate resolves full-evaluator and full-support from the Scheme plan and runs those measured subsets concurrently; the evaluator program was the common critical path, while the aggregate log still reports one complete host suite. The Racket bridge generates temporary #lang r7rs collection wrappers for checked-in .sld libraries because Racket's R7RS package resolves imports as Racket collection modules. The compiled self-host plan is the gold-standard product corpus. Its selector contains 45 programs: 44 of the 65 ordinary full programs plus the compiled-only runtime manifest smoke program. Those 44 ordinary programs cover Scheme-visible reader and numeric behavior, registered agent semantics, testing infrastructure, models, data structures, random/property facilities, generators, and the complete SRFI 180 reference corpus. In particular, the compiled agent-reliability and native CLI daemon adapter programs read real structured fixture files through (scheme read); focused Racket-compiled runs exercise 20 and 238 assertions, respectively. The compiled-only manifest smoke also reads a labelled cycle through (scheme read), checks its self-identity, mutates it, and verifies write-shared output. Six direct private owner/adapter suites remain outside this selector: consent-reader-test.scm, consent-numeric-test.scm, consent-numeric-generated-test.scm, consent-fixture-test.scm, consent-symbol-test.scm, and consent-datum-test.scm. Their borrowed-host ABI belongs to #120. Compiled random/property suites retain public numeric coverage without crossing that private dispatcher boundary. The remaining 21 ordinary programs are explicitly tagged self-host-gap and assigned to implementation issues #120, #346 (symbol identity and macro-introduced identifiers), or #432 (compiled-rooted nested evaluation). The target is to drive that set to zero; it is not a permanent reduced suite. Local test-portable-compiled and test-portable-gambit-native targets compile before invoking the matching *-run consumer through consent --host-run. CI exposes the phases separately and prioritizes each compile step before direct host work and the compiled test consumer. Compiled-build CI caches are intentionally intermediate-only: fallback restores may warm generated sources, compiled objects, incremental hashes, or Racket bytecode, but they must not restore a runnable bin/consent for the shard to test. CI builds and caches Gambit 4.9.7 because Ubuntu 24.04's gambc package is 4.9.3 and does not accept the -:r7rs runtime option needed for the portable library search path. The Racket, Racket-compiled, and Guile extra-host shards run on the bare runner. The Gauche matrix runs inside an Ubuntu 26.04 container because Ubuntu 24.04 does not ship the Gauche package used by that shard. That container base image is pulled from the AWS ECR Public Ubuntu mirror (public.ecr.aws/ubuntu/ubuntu:26.04) instead of Docker Hub, and the Gauche matrix runs one shard at a time to avoid registry pull throttling during job provisioning. These shards contribute required host timing data. The Emacs-hosted shards split the non-portable ERT suite into core language/runtime, library/conformance, agent/capability, tools/docs, and integration groups. make test-emacs-hosted remains available as the local aggregate for all non-portable ERT tests with (not "consent-scheme-.*").

When CONSENT_TEST_SELECTOR is set, make test uses a single ERT runner with that selector instead of the local shard fan-out:

CONSENT_TEST_SELECTOR='consent-smoke-test-harness-runs' make test

Each shard uploads a test-log-* artifact and writes a job summary. On pull requests, the combined timing job also updates one PR comment with a compact shard timing table and a collapsible detail section so reviewers can see timing at a glance from the PR conversation. Portable Scheme runners may also emit fine-grained CONSENT_CI_CHECK_SECONDS diagnostics for slow checks; the combined summary keeps those details below the fold and treats shard wall time as the primary signal. The above-fold section sorts all reported shards by wall-clock time across hosts and uses actual CI shard names so full-suite, reflection contract, and reflection stress rows can be compared directly across hosts. The collapsible detail section keeps the stable diagnostic shard order.

Alongside that human-facing summary, the combined timing job emits one machine-readable, append-only record per run for longitudinal analysis. The record is JSON Lines tagged with a schema_version, uploaded as the ci-run-record artifact. See CI run record for the schema, the field set, the schema-version discipline, and the durable-sink plan.

Live local model tests require an OpenAI-compatible local model endpoint. The CI smoke target exercises ordinary completion plus forced tool-calling through the Emacs host, the portable Racket host, and the Racket-compiled Consent Scheme host. The two portable programs are selected and run directly by the Scheme test plan and portable launcher; ERT only runs the separate Emacs-host checks. Run the CI smoke selector with:

make test-live-model-ci

Run only the Scheme-native direct and compiled live lanes with:

make test-live-model-portable

Run one quick-start profile shard after pulling that profile's models:

make test-live-model-small
make test-live-model-recommended
make test-live-model-large

Run all opt-in live local model tests, including all three quick-start profile sets, with:

make test-live-model

All live targets set CONSENT_LIVE_MODEL_TEST=1. The profile and all-live targets also set CONSENT_LIVE_MODEL_MATRIX=1 and pass CONSENT_LIVE_MODEL_MATRIX_CASES as comma-separated ROLE=MODEL cases. Use CONSENT_LIVE_MODEL_ENDPOINT and CONSENT_LIVE_MODEL_ID to override the default local endpoint and smoke model id.

The normal test-emacs-tools shard also runs a deterministic quick-start contract test for docs/repl-agent-quickstart.md. That test extracts and evaluates the tutorial's known-good Scheme differentiator and checks the documented role/provider shape through a fake model transport. It deliberately does not download or call real models; use the opt-in local live targets above when you need to validate a local model set against the actual endpoint.

For documentation-only changes, also run:

git diff --check
rg -n "m[y]/consent|m[y]/mcp" README.md docs

The rg command should normally return no matches. Also search for any project-history or private-machine references relevant to the change. The pattern uses a character class so this guide does not carry the deprecated spelling as plain text. If a match is intentional, explain why in the pull request.

Test authoring and shard pitfalls

These are silent or CI-only traps when adding tests; nothing else in the build warns about them.

  • A new Emacs test file needs a matching shard selector. The ERT runner (tests/consent-test-runner.el) auto-discovers every tests/consent-*-test.el by glob, but make test does not run the whole glob -- it runs the four Emacs shard targets, each filtered by an ERT name-selector regexp in the Makefile (CONSENT_EMACS_CORE/LIBRARY/CAPABILITY/TOOLS_TEST_SELECTOR). A new file whose ert-deftest names match none of those selectors loads but never runs under make test (only under the unfiltered make test-emacs-hosted or an explicit CONSENT_TEST_SELECTOR). When adding a consent-*-test.el, add or broaden the matching shard selector. Host-spawning Scheme bridge tests (consent-scheme-* that start an external host) belong in the portable selectors; host-independent pure-elisp consent-scheme-* tests belong in an Emacs shard (for example CONSENT_EMACS_TOOLS_TEST_SELECTOR). Use a prefix clause only when the whole family is host-free; otherwise anchor exact names so host-spawning siblings stay on host shards. The selector: strings in .github/workflows/test.yml are cosmetic -- printed into the run summary only; CI selects tests by invoking make <shard-target>, which reads the Makefile selector, so the Makefile edit is what changes CI coverage. Update the YAML literal only to keep the summary honest.

  • Host-level #u8(...) literals break CI's Racket. A #u8(...) bytevector literal read at the host level in a tests/scheme/*.scm file -- a real datum the host Scheme reads, not text inside a "..." Consent source string -- fails on CI's Racket with read-syntax: bad syntax #u and fails the test-portable-racket shard. The other hosts accept it, and a newer local Racket also accepts it, so make test-portable-racket can pass locally while CI breaks; do not trust a local Racket pass for #u8 portability. Build host bytevector test data with (bytevector b ...), a plain (scheme base) call with no reader syntax that every host reader accepts. #u8(...) is only safe inside a Consent source string or an expected-value string, where the Consent reader (which supports #u8) reads it rather than the host reader.

  • Self-hosted shards yield Consent number records, not host integers. Under consent --host-run (the test-portable-compiled and test-portable-gambit-native shards) the test file is evaluated by the Consent interpreter, so a literal like '((length . 4)) yields an alist whose 4 is a Consent number record, not a host integer. A .sld procedure that compares or combines such a value with its own host-integer counters (>=, -, arithmetic, substring) works on every directly-run host but fails only on the self-hosted shards, surfacing as a generic (status error) (message "error") (host-condition error). Normalize at the boundary -- (if (consent-number? v) (consent-number-value v) v) (Emacs: consent-number-p) -- leaving host integers and #f untouched. Reproduce with a freshly built binary run directly, for example build/compile/gambit/bin/consent --host-run FILE.

  • Exercise cyclic reader data in direct and self-hosted lanes. The portable owned compound heap preserves datum-label identity for multi-element pair and vector cycles such as #0=(1 2 3 . #0#) and #0=#(1 #0#). Keep these cases in the portable reader suite, and keep representative cyclic mutation and writer cases in the shared fixture suite, so both compiled hosts and the Emacs bootstrap must preserve the same cycle behavior.

  • Compiled-host standard-source-library-*-file failures are usually a stale install. If the compiled-host shards (test-portable-gambit-native, test-portable-compiled) fail only on standard-source-library-case-lambda-file / standard-source-library-lazy-file -- expecting a relative scheme/consent/*.sld but getting an absolute /usr/local/share/consent/<version>/consent/*.sld -- it is almost always a stale local install, not a regression. The compiled binary is built with CONSENT_INSTALL_DATADIR=/usr/local/share/consent/<version> and searches that datadir before the cwd-relative source tree. Tell-tale signs: the failing version's directory exists under /usr/local/share/consent/ but main's does not, and the interpreted hosts (run with -L <source>) all pass. Clear it with sudo rm -rf /usr/local/share/consent/<that-version> (or sudo make uninstall) and re-run. Do not weaken the test -- CI has no install and always resolves source-relative.

  • Compare cross-host record streams by serializing, not equal?. To assert two Consent contract-record streams are equal in a test that runs on every host, compare their serialized forms (consent-datum->external in portable, consent-result->external in Emacs), not raw equal?. Records embed canonical-number records, and on Gauche R7RS equal? on those is identity-based, so two value-equal streams built separately are not equal? there (Chibi/Guile/Racket happen to pass). Records reloaded from a captured datum stream via the standard reader are plain Scheme data, which consent-datum->external refuses to write -- only live or replayed Consent records can be serialized.

Expected Repository Shape

The implementation module map is defined in the architecture document. Early work is expected to introduce directories such as:

lisp/
scheme/
tests/
fixtures/

Do not create broad placeholder trees without an issue that needs them. Let the first implementation tickets establish only the files they actually use.

Local State

Keep generated state, downloaded model weights, caches, transcripts, private memory, and machine-specific configuration out of git. Future tickets will define the exact ignored local-state paths.

Until those paths exist, avoid committing:

  • secrets or provider tokens
  • absolute paths from a developer machine
  • generated logs or transcripts
  • downloaded third-party assets
  • local Emacs state

Pull Request Checklist

Before opening a PR:

  • Confirm the branch only contains the intended issue work.
  • Confirm scheme/consent/version.sld matches the issue's roadmap-derived version from #53.
  • Run the available verification commands.
  • Check public docs for stale personal-config or historical-repo references.
  • Use a Conventional Commits message for each commit.
  • Reference the issue in commit footers and the PR body.