Skip to content

Latest commit

 

History

History
187 lines (152 loc) · 9.23 KB

TESTING.md

File metadata and controls

187 lines (152 loc) · 9.23 KB

Testing

Use focused checks while iterating, then hand off each affected package. Run the full gate once on the integrated tree; this routing is a starting heuristic, not automatic diff-based selection.

Focused iteration

Focused tests use the full suite's SQLX_OFFLINE=true and target/test artifacts:

just test-lib aven-core sync::
just test-target aven cli_local

test-lib takes a libtest/nextest test-name filter. Filters match substrings, so end a module filter with :: (for example, sync::) to avoid matching unrelated names containing the same text. test-target runs the named integration-test target; target names use Rust/Cargo naming, such as cli_local for tests/cli_local.rs.

Test quality

A useful test adds independent information. Ask whether a plausible wrong implementation would fail it and a correct refactor would pass. Prefer fewer, stronger tests; extend an existing test when that keeps related coverage clear. For a bug fix, verify the regression test fails without the fix when feasible.

Treat these as review signals, not deletion rules:

  • Assertions that only restate constants, defaults, or mappings; tests of pure derived or standard-library behavior; expected values built with the same helper under test.
  • Echo mocks, existence-only checks, incidental structure assertions, and coverage duplicated across layers.

Retain independently meaningful behavior, including custom codec behavior, observable API, CLI, and serialized contracts, compatibility fixtures, invariants, boundary and failure cases, and cross-module behavior.

Tests run on Rust's default 2 MiB test-thread stack. Test builds compile workspace crates at opt-level 0. At that level, an async function's poll frame reserves a separate slot for every future it awaits, so dispatchers with many arms and test bodies with many awaits use far more stack than release builds. When a test aborts with has overflowed its stack, find the large future, for example with RUSTC_BOOTSTRAP=1 cargo rustc --profile test --lib -p <package> -- -Zprint-type-sizes in a separate target directory. Box that future where it is defined, or box the call that every caller awaits, rather than raising RUST_MIN_STACK.

Ordinary database fixtures copy a blank template instead of migrating a new file per test. aven_core::test_support::open_blank_database builds one fully migrated, checkpointed, single-file database per test binary build under aven-db-templates/ next to the binary, copies it to the test's private path, and opens the copy through Database::open. The template omits meta.client_id, so every copy receives its own installation identity. Use Database::open on a new path instead when the test's subject is first-open behavior: migrations, historical schemas, WAL handling, backups, restores, or installation identity.

Container packaging

Build the checked-out source and exercise the real container on your host's architecture (linux/amd64 or linux/arm64):

docker build -t aven:local .
scripts/test-container-image aven:local linux/arm64
AVEN_IMAGE=aven:local docker compose config --quiet
shellcheck scripts/test-container-image
actionlint .github/workflows/container.yml .github/workflows/release.yml

The smoke script uses disposable containers and a named volume. It checks non-root setup, refusal of unprepared storage, the HTTP router, graceful SIGTERM, restart, and replacement-container persistence. Setup invitations are captured privately and deleted; do not upload raw setup output as test logs. Container CI builds source images on native amd64 and arm64 runners.

The release workflow checksums the existing Linux musl archives, builds the Dockerfile's release target with the release-bin named build context, and runs both architectures (arm64 under QEMU). That context contains linux-amd64/aven and linux-arm64/aven. Saved, tested images are passed to publication without rebuilding. Hyphenated tags verify images without publishing. Stable tags publish exact versions and promote latest only after both platform pulls pass and GitHub identifies the tag as its latest release.

Registry publication needs separate verification: make the GHCR package public, confirm its repository link and both platform descriptors, and pull the version anonymously on both platforms. Use a temporary empty Docker config rather than changing your normal registry credentials:

config_dir=$(mktemp -d)
docker --config "$config_dir" pull --platform linux/amd64 ghcr.io/raine/aven:VERSION
docker --config "$config_dir" pull --platform linux/arm64 ghcr.io/raine/aven:VERSION
rm -rf "$config_dir"

Replace VERSION with the published version. If Docker uses a custom context, set DOCKER_HOST to its daemon endpoint first; an empty configuration does not include saved contexts. Local builds and authenticated workflow pulls do not establish public visibility or anonymous access. Release tags and registry publishing are remote mutations; only run them when authorized.

Affected-package handoff

just test-package <package> runs that package's non-documentation test targets. just check-package <package> is the handoff check: all-target Clippy followed by those package tests.

just test-package aven-core
just check-package aven

To opt in to a report of passing tests slower than one second, use just profile-tests with the same package, target, or test-name filters as just _test, for example just profile-tests --package aven --lib peer_enrollment_http::tests::. The slow-tests nextest profile reports slow statuses without changing normal test or check-full output.

The default nextest profile uses 12 test threads. Test builds use debug level 1, which retains source-line tables for backtraces while reducing debugger detail such as local-variable information. To compare suite concurrency, warm target/test and repeat the full suite at 6, 8, and 12 threads:

env SQLX_OFFLINE=true cargo nextest run \
    --target-dir target/test --locked --workspace --all-targets \
    --no-fail-fast --status-level fail --test-threads 12

To compare test-profile builds, use separate empty target directories with cargo nextest run --no-run --locked --workspace --all-targets, setting CARGO_PROFILE_TEST_DEBUG=2 or 1 in each environment.

The test profile keeps workspace crates at opt-level 0 and builds dependencies at opt-level 2. This speeds warm full-suite runs at the cost of slower cold no-run builds. Debug level 1 retains line tables, but optimized dependency code can inline frames and omit locals, reducing stepping and backtrace detail.

To rebenchmark, compare this profile with the package override removed, using separate clean target directories for cold builds and alternating warm full suites at 12 threads. Record wall, user, and system time for both phases. Keep sccache disabled or hold its cache state equal; on macOS and Linux, RUSTC_WRAPPER=/usr/bin/env bypasses the configured wrapper.

The workspace packages are aven and aven-core; aven depends on aven-core, not the reverse. Select the package that owns the changed behavior. If a core API or contract change affects its aven consumer, check both packages. Database migration or SQLx query/metadata changes also need just sqlx-check.

The pre-commit hook runs format, dependency/static-safety, and migration-order checks; it does not infer affected packages from staged paths. For coordinated worktree changes, each agent runs just check-package for its affected package before handoff. The coordinator/integrator relies on the existing pre-push hook to run just check-full once, after integrating the changes. Do not repeat the full gate in every worktree.

Full gate on the integrated tree

just check-full preserves the repository-wide gate: just check, the full test suite, and deferred SQLx validation. All-target nextest compilation also emits the normal aven binary used by integration tests, so no separate workspace build is needed. Use the full gate once after integration. If the affected package or boundary is unclear, changes span packages, workspace configuration or shared test infrastructure changes, or focused checks leave material risk, choose the conservative full gate rather than guessing.

Changed area Initial route
Root app code outside the rows below (src/, root tests/) just check-package aven
TUI (src/tui/) just check-package aven
CLI and commands (src/cli/, src/commands/, tests/cli_*.rs) just check-package aven
Configuration (src/config/ and related CLI/TUI behavior) just check-package aven
Core (crates/aven-core/src/, crates/aven-core/tests/) just check-package aven-core
Sync (crates/aven-core/src/sync/, app/server sync paths) just check-package aven-core; include just check-package aven for affected app/server paths or consumer contracts
Database (crates/aven-core/src/db/, migrations, SQLx metadata) just check-package aven-core; also just sqlx-check for migrations or SQLx inputs
Website (docs/) cd docs && bun install --frozen-lockfile && bun run build
Cross-package, workspace, or uncertain changes Run the integrated-tree just check-full

If a change spans rows or does not fit a route, treat it as cross-package or uncertain and use the integrated full gate.