Skip to content

feat: deterministic simulation testing (DST) framework#6216

Closed
copyleftdev wants to merge 4 commits intoEffect-TS:next-minorfrom
copyleftdev:feat/dst-v4
Closed

feat: deterministic simulation testing (DST) framework#6216
copyleftdev wants to merge 4 commits intoEffect-TS:next-minorfrom
copyleftdev:feat/dst-v4

Conversation

@copyleftdev
Copy link
Copy Markdown

Summary

Adds a deterministic simulation testing framework to Effect, enabling reproducible concurrency testing with seeded scheduling.

Changes

packages/effect

  • Scheduler.ts: add stepOne() to ControlledScheduler — executes exactly one pending task, returns boolean
  • internal/fiberId.ts: replace hardcoded Date.now() in unsafeMake() with configurable clock source (setClockSource / resetClockSource)
  • DSTScheduler.ts: public API for deterministic simulation — run(), runMany(), config types
  • internal/dst/dstScheduler.ts: seeded PCG PRNG scheduler — same seed, same execution order
  • internal/dst/dstRuntime.ts: assembles deterministic runtime (scheduler + clock + random)
  • internal/dst/eventLog.ts: structured execution trace for replay
  • internal/dst/livenessChecker.ts: post-hoc starvation, deadlock, and infinite loop detection

packages/vitest

  • internal/dst.ts: it.dst() test primitive for multi-seed property-based concurrency testing

packages/dst

  • DSTRunner: multi-seed runner with optional V8 profiling and failure shrinking
  • V8Profiler: CPU profile capture with source map resolution
  • GitBlame: commit attribution for hot code paths
  • CascadeExporter: NDJSON export for external analysis tooling
  • CommitClassifier: conventional commit classification and fix-ratio scoring
  • ReportGenerator: HTML/JSON DST reports

Motivation

ControlledScheduler.step() drains all pending tasks in a single call — there is no way to execute one task and observe intermediate state. FiberId.unsafeMake() uses Date.now(), making fiber identity non-deterministic across runs.

Together, these prevent building FoundationDB/TigerBeetle-style deterministic simulation tests within Effect.

Prior work

Both issues are confirmed still present on current main after the v4 merge.

Tests

  • 9 proof-of-issue tests confirming both issues exist and the fixes resolve them
  • 82 existing tests pass (Scheduler, FiberId, Fiber, FiberRef, concurrency)
  • 7 DST-specific test suites (scheduler properties, fault injection, e2e, benchmark, vajra scoring)

Checklist

  • Tests added
  • Build passes (pnpm build on packages/effect)
  • No regressions (82/82 existing tests pass)
  • Follows Effect coding conventions

copyleftdev and others added 3 commits May 5, 2026 20:49
Ports the Deterministic Simulation Testing (DST) framework from the v3 branch:

1. fix(fiberId): replace hardcoded Date.now() with configurable clock source
   - Enables deterministic fiber identity for simulation testing
   - setClockSource() / resetClockSource() APIs
   - Original PR: Effect-TS#6167

2. feat(Scheduler): add stepOne() to ControlledScheduler
   - Execute exactly one task for fine-grained deterministic stepping
   - Prerequisite for DST scheduler
   - Original PR: Effect-TS#6168

3. feat(dst): full DST framework package
   - DSTScheduler: seeded PCG PRNG deterministic task scheduling
   - DSTRuntime: fully deterministic execution environment
   - LivenessChecker: post-hoc starvation/deadlock/infinite-loop detection
   - EventLog: structured execution trace for replay/debugging
   - V8Profiler: CPU profiling with source map resolution
   - GitBlame: commit attribution for hot code paths
   - CascadeExporter: NDJSON export for vajra analysis
   - CommitClassifier: conventional commit classification + fix-ratio scoring
   - ReportGenerator: HTML/JSON DST reports

4. feat(vitest): it.dst() test primitive
   - Multi-seed property-based concurrency testing
   - Automatic failure shrinking

Co-authored-by: Don Johnson <dj@codetestcode.io>
Proves both issues are still present in v4 and validates the fixes:

FiberId determinism:
- BEFORE: Date.now() makes fiber hashes non-deterministic across runs
- AFTER: configurable clock source → identical hashes given same seed
- MUTATION: reverting clock source re-introduces non-determinism

ControlledScheduler.stepOne():
- step() drains ALL tasks — no intermediate state observation
- stepOne() executes exactly one task — enables DST stepping
- Priority ordering verified
- Returns false when no tasks pending

Broader regression: 82 existing tests pass (Scheduler, FiberId, Fiber, FiberRef, concurrency)
- Remove instructional/tutorial-style comments
- Keep only /** @internal */ and /** @SInCE */ JSDoc tags
- Match Effect's terse documentation style
@copyleftdev copyleftdev requested a review from mikearnaldi as a code owner May 6, 2026 04:00
@github-project-automation github-project-automation Bot moved this to Discussion Ongoing in PR Backlog May 6, 2026
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 6, 2026

🦋 Changeset detected

Latest commit: bde42b9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 37 packages
Name Type
effect Minor
@effect/vitest Major
@effect/cli Major
@effect/cluster Major
@effect/dst Major
@effect/experimental Major
@effect/opentelemetry Major
@effect/platform-browser Major
@effect/platform-bun Major
@effect/platform-node-shared Major
@effect/platform-node Major
@effect/platform Major
@effect/printer-ansi Major
@effect/printer Major
@effect/rpc Major
@effect/sql-clickhouse Major
@effect/sql-d1 Major
@effect/sql-drizzle Major
@effect/sql-kysely Major
@effect/sql-libsql Major
@effect/sql-mssql Major
@effect/sql-mysql2 Major
@effect/sql-pg Major
@effect/sql-sqlite-bun Major
@effect/sql-sqlite-do Major
@effect/sql-sqlite-node Major
@effect/sql-sqlite-react-native Major
@effect/sql-sqlite-wasm Major
@effect/sql Major
@effect/typeclass Major
@effect/workflow Major
@effect/ai Major
@effect/ai-amazon-bedrock Major
@effect/ai-anthropic Major
@effect/ai-google Major
@effect/ai-openai Major
@effect/ai-openrouter Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@effect-bot effect-bot changed the base branch from main to next-minor May 6, 2026 05:15
@tim-smart
Copy link
Copy Markdown
Contributor

Effect v3 is under a feature freeze, feel free to move this contribution to the effect-smol repository.

@tim-smart tim-smart closed this May 6, 2026
@github-project-automation github-project-automation Bot moved this from Discussion Ongoing to Done in PR Backlog May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants