Skip to content

claude.md

Latest commit

 

History

History
317 lines (225 loc) · 16.9 KB

claude.md

File metadata and controls

317 lines (225 loc) · 16.9 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Overview

Verify is a snapshot testing framework for .NET that simplifies assertion of complex data models and documents. It serializes test results to .verified.* files and compares them against actual results on subsequent runs.

Solution file: src\Verify.slnx (uses modern .slnx format)

Build & Test Commands

Always pass src/Verify.slnx explicitly to dotnet build. There are multiple .slnx files in src/ and omitting the solution file causes ambiguity errors.

# Build the whole solution
dotnet build src/Verify.slnx

Test runner setup (Microsoft.Testing.Platform)

global.json sets "test": { "runner": "Microsoft.Testing.Platform" }, so dotnet test runs in MTP mode. Test projects follow these conventions:

  • No Microsoft.NET.Test.Sdk in xUnit v3, NUnit, MSTest or TUnit projects. It only exists for VSTest. The only project that still references it is Verify.Fixie.Tests, which uses VSTest. The Expecto projects don't reference it either: they are console executables run via dotnet run.
  • <OutputType>Exe</OutputType> is required in every MTP test project. Microsoft.NET.Test.Sdk used to set it implicitly; without it xunit.v3 fails the build ("xUnit.net v3 test projects must be executable"). TUnit sets it itself.
  • Runner opt-in: NUnit projects need <EnableNUnitRunner>true</EnableNUnitRunner> and MSTest projects need <EnableMSTestRunner>true</EnableMSTestRunner>. xUnit v3 and TUnit are MTP apps without any opt-in.
  • <RootNamespace>Fake</RootNamespace> in NUnit/MSTest projects stops the generated MTP entry point introducing a Verify namespace that shadows the Verify() method.

Running the full test suite

Because Verify.Fixie.Tests still uses the VSTest runner, dotnet test src/Verify.slnx does not work — it aborts with a mixed-runner error ("All projects must use that test runner") before running anything. There is no single command that runs every test project.

Instead, build once and run each test project individually. .github/workflows/build.yml is the source of truth for the complete, ordered list and how each project is invoked. The commands differ by runner:

# Build once in the target configuration, then run with --no-build --no-restore
dotnet build src/Verify.slnx -c Release

# Most projects (xUnit v3, NUnit, MSTest) run via `dotnet test`.
# Multi-targeted projects need an ABSOLUTE path: `dotnet test` re-resolves a relative
# project/directory path against the project directory, giving
# "src/X/src/X/X.csproj could not be found".
dotnet test $PWD/src/Verify.Tests -c Release --no-build --no-restore
dotnet test $PWD/src/Verify.NUnit.Tests -c Release --no-build --no-restore

# Expecto (F#) and TUnit projects are console executables — run via `dotnet run`
dotnet run --project src/Verify.Expecto.Tests -c Release --no-build --no-restore
dotnet run --project src/Verify.TUnit.Tests/Verify.TUnit.Tests.csproj -c Release --no-build --no-restore

# Fixie has no MTP runner — run via the fixie.console tool from the `src` directory
# (first: dotnet tool restore --tool-manifest src/.config/dotnet-tools.json)
cd src && dotnet fixie Verify.Fixie.Tests -c Release --no-build

src/VerifyDangling.slnx (the DanglingSnapshots*Usage projects) is built separately by CI; its tests are not run.

Running a single test

MTP does not accept the VSTest --filter "FullyQualifiedName~..." syntax. For xUnit v3 projects, either pass xUnit's MTP filters through dotnet test (use -f to pick one framework):

dotnet test $PWD/src/Verify.Tests -c Release --no-build --no-restore -f net9.0 --filter-method "*NotInlineRetiresTheCallSite*"

or build the project for a single framework and run the compiled test executable directly with the xUnit v3 -class / -method filters (leading/trailing * wildcard supported):

dotnet build src/Verify.Tests/Verify.Tests.csproj -f net11.0 -c Debug
src/Verify.Tests/bin/Debug/net11.0/Verify.Tests.exe -class "PluginConventionTests"
src/Verify.Tests/bin/Debug/net11.0/Verify.Tests.exe -method "*StaleUnreferencedPluginIsSkipped*"

Common Test Projects

  • Verify.Tests - Core verification engine tests
  • Verify.Xunit.Tests / Verify.XunitV3.Tests - xUnit adapter tests
  • Verify.NUnit.Tests - NUnit adapter tests
  • Verify.MSTest.Tests - MSTest adapter tests
  • Verify.Fixie.Tests - Fixie adapter tests
  • Verify.Expecto.Tests / Verify.Expecto.FSharpTests - Expecto (F#) adapter tests
  • Verify.TUnit.Tests - TUnit adapter tests
  • DeterministicTests - Deterministic serialization tests
  • StaticSettingsTests - Global configuration tests
  • StrictJsonTests - JSON parsing tests

Tool Installation

# Restore dotnet tools (required for Fixie tests)
dotnet tool restore --tool-manifest src/.config/dotnet-tools.json

Architecture

Three-Layer Design

  1. Test Framework Adapters (Verify.NUnit, Verify.Xunit, Verify.MSTest, etc.)

    • Extract test metadata (type name, method name, parameters) from framework context
    • Implement BuildVerifier() to create InnerVerifier instances
    • Each adapter is framework-specific but follows the same pattern

  2. Core Verification Engine (Verify project)

    • InnerVerifier - Orchestrates verification workflow (split across 12 partial files by data type)
    • VerifyEngine - Comparison logic, diff tool launching, callback execution
    • SettingsTask - Fluent builder API for configuration

  3. Serialization & Comparison

    • VerifyJsonWriter - Custom JSON writer extending Argon library
    • SerializationSettings - Registry of converters and scrubbers
    • Comparer / FileComparer / StreamComparer - Different comparison strategies

Key Design Patterns

Adapter Pattern: Each test framework has a minimal adapter that extracts framework metadata differently:

  • Xunit: Uses UseVerifyAttribute (a BeforeAfterTestAttribute) injected via MSBuild
  • NUnit: Direct access to TestContext.CurrentContext.Test
  • MSTest: Requires [UsesVerify] attribute with source generator
  • Fixie: Uses ExecutionState.Current

Partial Classes: InnerVerifier is split across 12 files:

  • InnerVerifier_Object.cs - Object serialization
  • InnerVerifier_String.cs - Text comparison
  • InnerVerifier_Stream.cs - Binary files
  • InnerVerifier_Json.cs - JSON handling
  • InnerVerifier_Xml.cs - XML serialization
  • InnerVerifier_Archive.cs - ZIP files
  • InnerVerifier_Directory.cs - Directory trees
  • InnerVerifier_File.cs - Individual files
  • InnerVerifier_Task.cs - Async unwrapping
  • InnerVerifier_Throws.cs - Exception capture
  • And others...

Builder Pattern: SettingsTask provides fluent API with ~50 configuration methods that are lazily evaluated at verification time.

Source and derived targets: A converter can say which of its targets is the document and which were computed from it (ConversionResult(info, source, derived), or the PagedConversion builder for documents with pages, both in src/Verify/Splitters/):

  • InnerVerifier.Adopt (InnerVerifier_Stream.cs) is the only place a ConversionToken is created. It stamps the targets of a conversion, names them relative to the target that was converted, and marks the source, which is never converted again.
  • VerifyEngine.HandleResults decides every file name first (Plan), compares sources ahead of the rest (CompareOrder, so a differing source makes its derived targets bypass their comparers), and hands results on in planned order, which is the order callbacks and the exception message have always used.
  • VerifyEngine.Report runs once, after the delete/new/not-equal callbacks, and is the only caller of DiffEngine for pending files: deletes that stand alone, moves that stand alone (sources among them), moves derived from a pending source (DiffRunner.LaunchDerived*), then deletes derived from one. Test seams: VerifyEngine.LaunchDiff and RaisedDeletes.AddDelete.
  • The settings a paged converter reads (PageText, PagesToInclude, ExcludeDerivedTargets) follow the ExcludeTargets pattern: a static on VerifierSettings, a Context key on VerifySettings, a SettingsTask wrapper, and a reader on the converter's context.
  • User and plugin-author docs: docs/paged-documents.md and the "Source and derived targets" section of docs/converter.md.

Counter Pattern: Deduplicates repeated values in filenames:

  • First occurrence: Date, second: Date_1, third: Date_2, etc.
  • Separate counters for DateTime, DateTimeOffset, Guid, etc.

File Naming Convention

Format: {TypeName}.{MethodName}_{Parameters}_{UniqueFor}.{Status}.{Extension}

Example: MyTests.MyTest_param1_param2.verified.txt

Key components:

  • Namer.cs - Central naming logic with "unique for" variants (runtime, framework, etc.)
  • FileNameBuilder.cs - Builds filename segments from test metadata
  • ParameterToName - Converts parameters to filename-safe strings
  • FileNameCleaner.cs - Removes invalid characters (: * ? / < > |)

Critical Code Locations

  • Verification flow: src/Verify/Verifier/VerifyEngine.cs (main orchestrator)
  • Serialization: src/Verify/Serialization/VerifyJsonWriter.cs and SerializationSettings.cs
  • Comparison logic: src/Verify/Compare/Comparer.cs
  • File naming: src/Verify/Naming/FileNameBuilder.cs
  • Test adapter examples: src/Verify.Xunit/Verifier.cs, src/Verify.NUnit/Verifier.cs

Important Conventions

Snapshot Files

  • Exclude from source control: All *.received.* files (mismatches during test runs)
  • Include in source control: All *.verified.* files (approved snapshots)
  • File characteristics: UTF-8 with BOM, LF line endings, no trailing newline

.gitattributes Requirements

*.received.*
*.verified.txt text eol=lf working-tree-encoding=UTF-8
*.verified.xml text eol=lf working-tree-encoding=UTF-8
*.verified.json text eol=lf working-tree-encoding=UTF-8
*.verified.bin binary

EditorConfig Settings

Verify files should use:

  • charset = utf-8-bom
  • end_of_line = lf
  • insert_final_newline = false

Project Structure

src/
├── Verify/                          # Core library
│   ├── Verifier/                    # Verification engine
│   ├── Naming/                      # File naming logic
│   ├── Serialization/               # JSON serialization & converters
│   ├── Compare/                     # Comparison strategies
│   ├── Recording/                   # Data recording during tests
│   ├── Combinations/                # Cartesian product testing
│   ├── DerivePaths/                 # Build metadata integration
│   └── buildTransitive/             # MSBuild props/targets
├── Verify.NUnit/                    # NUnit adapter
├── Verify.XunitV3/                  # xUnit v3 adapter
├── Verify.MSTest/                   # MSTest adapter
├── Verify.MSTest.SourceGenerator/   # Code generation for MSTest
├── Verify.Fixie/                    # Fixie adapter
├── Verify.Expecto/                  # Expecto (F#) adapter
├── Verify.TUnit/                    # TUnit adapter
├── [Various test projects]
└── .config/dotnet-tools.json        # Local tool manifest

Development Workflow

Adding a Test Framework Adapter

  1. Create new project (e.g., Verify.NewFramework)
  2. Implement BuildVerifier() to extract test context from framework
  3. Create partial Verifier classes for each verification method (Object, String, Stream, etc.)
  4. Add MSBuild props file to inject any required attributes/configuration
  5. Optionally add module initializer to register test attachment callbacks

Reference existing adapters like Verify.Xunit/Verifier.cs or Verify.NUnit/Verifier.cs.

Extending Serialization

  1. Create converter class implementing appropriate interface in src/Verify/Serialization/Converters/
  2. Register via VerifierSettings.AddSerializer<T>(converter) in module initializer
  3. Converters can access SerializationSettings for configuration

Modifying File Naming

  • Primary logic in src/Verify/Naming/FileNameBuilder.cs
  • Parameter conversion in src/Verify/Naming/ParameterToName.cs
  • Sanitization in src/Verify/FileNameCleaner.cs

Accepting New Snapshots

When tests fail, .received.* files are created showing actual output. To accept a new snapshot, copy the .received.* file to the path on the Verified: line of the exception message. Do not derive the verified name from the received name — the two are not always the same.

Multi-target naming: When a test project targets multiple frameworks (e.g., Verify.Tests), the .received. file always carries a runtime and version suffix (e.g., .DotNet11_0.received.txt). The .verified. file carries whatever UniqueFor* the test asked for, which may be nothing (.verified.txt), the runtime (.DotNet.verified.txt), or the same runtime and version (.DotNet11_0.verified.txt).

Single-target naming: When a test project targets a single framework (e.g., StaticSettingsTests), the .received. file uses the same uniqueness as its .verified. file, including any UniqueForRuntime/UniqueForRuntimeAndVersion suffix.

Parameters: Ignored parameters (IgnoreParameters, IgnoreParametersForVerified, IgnoreConstructorParameters) are kept in the .received. name and dropped from the .verified. name, regardless of how many frameworks are targeted.

Debugging Tests

On CI (GitHub Actions), failed test artifacts are uploaded for inspection.

Multi-Targeting

The solution supports:

  • .NET Framework: net462, net472, net48 (Windows only)
  • .NET: net6.0, net7.0, net8.0, net9.0, net10.0, net11.0

SDK version: 11.0 preview (see src/global.json)

Platform-specific code uses conditional compilation:

#if NET6_0_OR_GREATER
// Modern .NET code
#endif

#if NET8_0_OR_GREATER
// .NET 8+ optimizations (e.g., SearchValues<T>)
#endif

Common Gotchas

  1. Module Initializers: Global configuration via VerifierSettings must be done in a module initializer ([ModuleInitializer]) before any tests run.

  2. MSTest Requires Opt-in: MSTest tests need [UsesVerify] attribute on class/assembly, or inherit from VerifyBase.

  3. Fixie Requires Custom Convention: Fixie needs ITestProject and IExecution implementations that call VerifierSettings.AssignTargetAssembly() and ExecutionState.Set().

  4. Snapshot File Nesting: Verify includes MSBuild props that nest snapshot files under test files in IDE. Disable with <DisableVerifyFileNesting>true</DisableVerifyFileNesting>.

  5. TUnit Tests: Use dotnet run instead of dotnet test for TUnit projects.

  6. Tool Restoration: Run dotnet tool restore before running Fixie tests (requires fixie.console tool).

  7. New Test Projects: Do not add Microsoft.NET.Test.Sdk to an MTP test project, and remember <OutputType>Exe</OutputType> plus the runner opt-in (see "Test runner setup"). Without Microsoft.NET.Test.Sdk, the runner opt-in is what makes dotnet test treat an NUnit/MSTest project as a test project.

Source Generator (MSTest)

The MSTest adapter includes a source generator at src/Verify.MSTest.SourceGenerator/ that generates code for tests marked with [UsesVerify]. This handles test context plumbing automatically.

Documentation Generation

README and docs are generated from source files using MarkdownSnippets:

  • Source: readme.source.md
  • Generated: readme.md (DO NOT EDIT)
  • Doc sources: docs/mdsource/*.source.md → docs/*.md (generated)
  • Includes: docs/mdsource/*.include.md (reusable fragments)
  • Run mdsnippets to regenerate (globally installed dotnet tool)

Snippet Conventions

  • Code snippets use #region SnippetName / #endregion markers in source files
  • Referenced in docs as snippet: SnippetName
  • Verified files can be included directly: snippet: TestClass.Method.verified.txt
  • src/ModuleInitDocs/ contains purpose-built snippet classes for documentation (each file wraps a single #region)
  • src/Verify.ExceptionParsing.Tests/ExceptionMessageFormatSamples.cs contains tests whose sole purpose is producing verified snapshots for the exception format docs

Content Validation

src/mdsnippets.json has ValidateContent: true. This forbids certain words (e.g., "you", "your") in .source.md files outside of fenced code blocks. Use third-person phrasing instead.

Exception Message Format

When a Verify test fails, VerifyExceptionMessageBuilder (src/Verify/Verifier/VerifyExceptionMessageBuilder.cs) produces a structured, machine-parsable exception message with sections: Directory:, New:, NotEqual:, Delete:, Equal:, and FileContent:. See docs/exception-message-format.md for the full format.

Verify.ExceptionParsing (src/Verify.ExceptionParsing/) is a separate library (published as a NuGet package) that parses these exception messages back into structured data.