diff --git a/CLAUDE.md b/CLAUDE.md index 73650a4..330aa13 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,6 +27,28 @@ fidelity/tuning of the temporal path (accumulation, disocclusion thresholds, mot convention) is correct enough to render cleanly but **not tuned** — the "landmines still live" section remains the guide for visual regressions. +**Parity program concluded (2026-07-21):** the three source-style FSR 3.1.5 candidate +graphs were GPU-verified and A/B-benchmarked against production — **+36% / +6.5% / ++76% GPU compute with no visual win**; none adopted. Consumer-facing rationale in +`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-DECISIONS.md` / +`PARITY-CANDIDATES.md`. **Post-parity +items 1–3 landed the same day** (see `bench/docs/NEXT-STEPS.md` for evidence): +(1) RCAS now sharpens in conditioned tonemap space, inverting once — **−34% RCAS, +−5.7% total** with capture-identical output; the old form is frozen as +`RCAS_PER_TAP_SHADER` under the `rcas-fsr315-limiter` bench identity. (2) Host +pre-exposure (`preExposureTexture`) is honored end-to-end — DeltaPreExposure history +correction + host-invariant auto-exposure metering, validated on the new **Q11** +scenario, byte-identical when absent. (3) The reconstruct pass uses AMD's +viewport/depth-scaled disocclusion (per-tap confidence voting) inside the fused +single pass. **Item 4 (the multi-scale shading-change detector) landed the same session**: the +3×3-neighborhood shading heuristic is replaced by `shadingChange.ts` — a fused +multi-scale block-mean detector (0.044 ms, 5× cheaper than the candidate's +two-pass form, measurably fewer false positives under motion; five GPU tuning +iterations documented in NEXT-STEPS). Nothing from the parity program remains +open. Candidate +A/B runs: `node scripts/run-benchmark.mjs --smoke --variant --comparison ` +(see `--help`). + If you touch shaders/passes, re-verify on a real GPU. A dependency-free way to do it headlessly (no Playwright): launch Chrome with `--headless=new --enable-unsafe-webgpu --remote-debugging-port=N`, drive it over the DevTools Protocol (Node 22 has a native @@ -46,6 +68,15 @@ When something breaks after an edit, expect failures in this order of likelihood Don't trust "it builds" as "it works." Drive the real bench. +Bench caveat (measured 2026-07-21): benchmark runs launched from a scratchpad +**git worktree** read ~3× slower absolute GPU times, uniformly across all passes +(the GPU never leaves its low power state — likely cold-vite frame delivery). +A/B comparisons *within* that environment are valid; never compare worktree +absolutes against repo-run records. Also: a `node_modules` **symlink** in a +worktree isn't matched by the root `.gitignore`'s `node_modules/` pattern +(trailing slash ≠ symlink) and crashes the benchmark's working-tree digest — +`.git/info/exclude` carries a slash-less `node_modules` entry for this. + --- ## Commands @@ -57,6 +88,11 @@ npm test # vitest — jitter math, quality presets, WGSL module assemb npm run typecheck # tsc --noEmit npm run lint # eslint npm run build # library build → dist/ (vite lib + tsc declarations) + +# Still-scene convergence meter (GPU, headless Chrome + CDP): consecutive + +# same-jitter-phase frame diffs plus debug-view PNGs on a deterministic +# scenario. Q12 = cornell + point-light shadow dither (consumer report 3 repro). +node scripts/measure-convergence.mjs --scenario Q12 --ratio 2 ``` CI (`.github/workflows/ci.yml`) runs lint → typecheck → test → build on push/PR. No GPU in CI, so tests are deliberately GPU-free (pure math + shader-string structure). **Keep it that way** — don't add tests that need a device to CI; they'll hang or fail. @@ -76,7 +112,7 @@ src/ shaders/ common.ts — shared WGSL chunks: FsrConstants UBO, color/depth/tonemap helpers, FLAG_* bits wgsl.ts — assembleShader() dedup concatenator (WGSL has no #include) - blit / easu / rcas / reconstruct / accumulate / luminancePyramid / generateReactive / debug .ts — the passes + blit / easu / rcas / reconstruct / shadingChange / accumulate / luminancePyramid / generateReactive / debug .ts — the passes README.md — per-pass fidelity vs FidelityFX reference + debugging guide internal/ threeWebGPU.ts — getDevice() / getGPUTexture(): the three-internals bridge @@ -109,7 +145,7 @@ Read `Upscaler.ts` and `bench/src/BenchPipeline.ts` together first — the secon - These are **private**. `threeWebGPU.ts` throws loudly if the shape changes. **If you bump three, re-verify these two accessors first** — they're the most likely thing to break on upgrade. - **Output is a three `StorageTexture`** — the upscaler writes into it, the caller samples it like any texture. Presenting is ordinary three code (a fullscreen quad). - **One shared 256-byte constants UBO** (`ConstantsBuffer` ↔ `WGSL_CONSTANTS` in `common.ts`), bound at `@group(0) @binding(0)` in every pass, written once per frame. **The two layouts must stay byte-for-byte in sync** — f32/u32 indices in `ConstantsBuffer.ts` map to documented byte offsets in the WGSL struct. If you add/reorder a field, change **both** or everything silently corrupts. -- **Color spaces:** temporal accumulation happens in invertible-tonemap space (`c/(1+max(c))`, FSR2's firefly guard); EASU/RCAS run display-referred; every path exits through one shared ACES+sRGB `displayTransform`. That's why the present quad must NOT re-tonemap (see landmines). +- **Color spaces:** temporal accumulation happens in invertible-tonemap space (`c/(1+max(c))`, FSR2's firefly guard), then RCAS/blit reverse that conditioning and write `rgba16float` in the caller's linear/HDR domain. The upscaler never applies ACES, an output transfer function, or another presentation transform. EASU leaves its caller-provided input domain unchanged. Presentation belongs to the consuming renderer/post graph. - **Jitter/velocity:** projection is jittered via `camera.setViewOffset` (same as three's TRAA). Motion vectors must be **jitter-free**, so the scene's `velocity` node gets `upscaler.unjitteredProjectionMatrix` (a stable Matrix4 instance whose contents refresh each `beginFrame`). --- @@ -122,7 +158,7 @@ These were discovered by reading three's source; they're non-obvious and easy to 2. **`StorageTexture` defaults `generateMipmaps = true`** → three allocates a mip chain and the storage view breaks. The output texture sets `generateMipmaps = false`, and storage views are pinned to `{ baseMipLevel: 0, mipLevelCount: 1 }` (`_outputView()`). Keep both. 3. **Combined depth-stencil formats need a depth-only view** (`{ aspect: 'depth-only' }`) to bind as `texture_depth_2d`. `_encodeTemporal` branches on `format.includes('stencil')`. 4. **Reversed depth:** we read `renderer.reversedDepthBuffer` and set a flag; `linearizeDepth` in `common.ts` handles both conventions + ortho. If depth debug view is full-screen-flashing, this flag is the suspect. -5. **Present quad setup:** `NoToneMapping` + `LinearSRGBColorSpace` on the renderer, and `depthTest/depthWrite/fog = false` on the quad material. The FSR output is already display-referred sRGB; re-encoding double-applies the transform. +5. **Present quad setup:** `depthTest/depthWrite/fog = false` on the quad material. The FSR output is linear/HDR, so the renderer's normal output transform should run when presenting to screen. The examples choose `ACESFilmicToneMapping` + `SRGBColorSpace`; library users own that policy. 6. **Bench Vite needs `build.target: 'esnext'`** — `main.ts` uses top-level `await renderer.init()`. 7. **MRT output count must match the RT attachment count.** Rendering the scene into a `count: 2` `RenderTarget` while `setMRT(null)` (or an MRT without the velocity output) leaves **color attachment 0 unwritten → black output**. This is why non-temporal bench modes were black on first GPU boot. Fix: size the RT to the mode (temporal → `count: 2` + `mrt({ output, velocity })`; everything else → `count: 1` + `mrt({ output })`). See `BenchPipeline.configure`/`render`. **Any integration (incl. examples / `UpscalePresenter`) must keep these two counts in lockstep.** 8. **WGSL parses `a < b, c > d` as a template argument list.** `select(d < bestDepth, d > bestDepth, reversed)` failed to compile (`parsed as template list`). Wrap comparisons that put `<` … `>` in one expression in parentheses: `select((d < bestDepth), (d > bestDepth), reversed)`. Watch for this whenever a shader compares with both `<` and `>` nearby. @@ -133,10 +169,10 @@ These were discovered by reading three's source; they're non-obvious and easy to ## Landmines still live (unverified — prime suspects when it misbehaves) - **Motion vector sign/scale.** `motionScale = (0.5, -0.5)` converts the `velocity` node's NDC delta to a UV delta, and reprojection is `prevUV = uv - motion`. **Corroborated against three's own `TAAUNode`** (2026-07-08), which uses the identical `velocity.xy * vec2(0.5, -0.5)` scale and `historyUV = uv - offset` reprojection, plus the same `setViewOffset` jitter in render-pixel units — so this is no longer a guess. Still the first thing to re-check if history smears or tears under camera motion; **Motion Vectors debug view is your friend.** -- **Accumulation tuning.** Catmull-Rom history filter, YCoCg variance-clip gamma (`CLIP_GAMMA = 1.0`), disocclusion/clip weight falloffs, **and the luminance-lock constants (`LOCK_*`)** in `accumulate.ts` are sensible defaults, not tuned. Ghosting → tighten (lower `LOCK_CLAMP_RELAX`/`LOCK_HISTORY_BOOST`, raise the peak/contrast thresholds); instability/shimmer or thin features dimming → loosen. Use `DebugView.Locks` to see where locks form. -- **Shading-change constants** (`accumulate.ts`: `SHADING_LO/HI/AGE`) are conservative defaults. If stable, steadily-lit surfaces shimmer or fail to converge, `SHADING_LO` is too low (it's aging history that didn't change) — raise it; if a genuine lighting change ghosts its old shading, lower it. `DebugView.ShadingChange` should be black on a still scene — check it before suspecting the accumulate blend. The detector reuses the 3×3 neighborhood mean, not a coarse pyramid mip, so it can false-positive on very high-frequency content under heavy motion; that's the Phase-5 SPD-mip refinement. +- **Accumulation tuning.** Catmull-Rom history filter, YCoCg variance-clip gamma (`CLIP_GAMMA = 1.0`), disocclusion weight falloffs, **and the luminance-lock constants (`LOCK_*`)** in `accumulate.ts` are sensible defaults, not tuned. Ghosting → tighten (lower `LOCK_CLAMP_RELAX`/`LOCK_HISTORY_BOOST`, raise the peak/contrast thresholds); instability/shimmer or thin features dimming → loosen. Use `DebugView.Locks` to see where locks form. **Two convergence rules learned from consumer report 3 (2026-07-24, NEXT-STEPS §5) — do not re-break:** (1) never age `sampleCount` by clip magnitude (an earlier `clipAmount` aging made still-scene convergence unreachable — rolling age moiré); (2) the blend stores the *clipped* history, so still+converged+quiet pixels get a `STILL_CLAMP_RELAX`(×9) widened box or each jitter phase's variance box re-snaps the buffer forever (phase-locked frame diff 0.18 vs 0.02 shipped). Ghosting on *still* scenes after a missed lighting change → lower `STILL_CLAMP_RELAX` before touching anything else; motion is unaffected (relax fades out above 0.5 texel/frame). +- **Shading-change tuning** (`shadingChange.ts`: `SHADING_FLOOR_MID/COARSE/CV`; `accumulate.ts`: `SHADING_AGE`) — GPU-tuned on Q1/Q4/Q9/Q11 (2026-07-21) but still constants, not laws. Ghosting after a genuine lighting change → lower the floors; flat steadily-lit surfaces shimmering → raise them (raise `SHADING_FLOOR_CV` if the noise sits on textured regions). `DebugView.ShadingChange` should be near-black on a still scene — check it before suspecting the accumulate blend. Slow ramps deliberately don't fire (1-frame comparison; the blend tracks ramps — verified no lag on Q9). - **Auto-exposure constants** (`luminancePyramid.ts`: `EXPOSURE_KEY`, `EXPOSURE_MIN/MAX`, `ADAPT_SPEED`) are defaults. Because exposure is divided back out before display, the *visible* effect is subtle (better accumulation stability, not a brightness change). If a scene pulses in brightness, `ADAPT_SPEED` is the suspect; if the image goes flat/washed on a very bright or dark scene, check the min/max clamp. `DebugView.Exposure` should read near mid-grey — verify there before suspecting the accumulate math. -- **Depth separation threshold** in `reconstruct.ts` (`DEPTH_SEPARATION_SCALE`, `DEPTH_SIMILARITY_FLOOR`) is a guess. Too aggressive = history thrown away everywhere (no convergence); too slack = ghost trails behind moving objects. +- ~~**Depth separation threshold** in `reconstruct.ts` is a guess~~ — **resolved 2026-07-21**: replaced by AMD's viewport/depth-scaled formulation (`1.37e-5 · halfViewportWidth · maxDepth`, per-bilinear-tap confidence voting from `ffx_fsr2_depth_clip.h`), GPU-validated on Q3 (thin stable outlines, still scenes quiet, age resets confined to trails). **Amended 2026-07-22** (grazing-plane flicker found via example 12): reprojection is jitter-delta-compensated; tolerance is widened by the dilation ring's own depth relief (cross-frame gather must absorb one-texel slope mismatch that upstream's same-frame scatter sidesteps). **Amended again 2026-07-24** (still-camera silhouette flicker, consumer report 3): every valid tap votes — taps at/behind the current surface vote full confidence — and the **best tap wins** (max aggregation, not a weighted mean of positive separations only; the 07-22 "skip, never veto" semantics still let one boundary-quantization-straddling tap fully disocclude a still edge every phase). Genuine trails still read ~1 (all taps on the old occluder). Still no scene-tuned constants. Convergence regressions: measure with `scripts/measure-convergence.mjs` (Q1/Q12) before touching anything; evidence in `bench/docs/NEXT-STEPS.md` §5. - **`timestamp-query`** may be absent; `GpuTimer` no-ops gracefully, but confirm the GPU-ms readout actually appears where supported. --- @@ -150,48 +186,100 @@ These were discovered by reading three's source; they're non-obvious and easy to 3. **Accumulation age** — should saturate to white within ~1s when still, reset along disocclusion trails. Never whitening ⇒ history not persisting (ping-pong or reset logic). 4. **Locks** — lights up on thin high-contrast features (grid lines, wire/fence edges, specular silhouettes), black on flat surfaces. All-black ⇒ thresholds too high (thin features dim); lit everywhere ⇒ too low (ghosting). 5. **Exposure** — exposed scene luma should read near mid-grey everywhere. All-black/all-white ⇒ exposure pinned at its min/max clamp. -6. **Shading change** — black on a static steadily-lit scene; lights up on surfaces whose shading actually changes (moving specular, animating light). Lit everywhere while still ⇒ `SHADING_LO` too low. +6. **Shading change** — black on a static steadily-lit scene; fires as clean single-frame spikes on light steps (moving specular, animating light). Lit everywhere while still ⇒ `SHADING_FLOOR_*` too low (see `shadingChange.ts`). 7. **Reactivity** — the caller's reactive mask (white on flagged transparents/particles, black on opaque). Empty/misaligned ⇒ the mask isn't authored/passed right. Full guide in `src/shaders/README.md`. --- -## Roadmap (what "finished" looks like) - -Phases 0–2 are **written and GPU-verified** (bench renders all paths correctly; six -standalone examples ship in `examples/`). The examples double as the validation harness -for the remaining phases: prove luma-stability locks on `04-aliasing-torture`, the -reactive mask on `05-transparency` (its explicit acceptance test), the RCAS-denoise -variant on `06-screenspace-gi`, and expose new toggles in `02-fsr1-vs-fsr3`. Remaining: - -- **Phase 3 — fidelity to real FSR3:** - - ~~**Luminance-stability locks**~~ — **done & GPU-verified.** Persistent display-res lock buffer in `accumulate.ts` (r = lifetime, g = locked luma), reprojected through motion; detects thin luminance outliers, grows a lock while present, breaks on disocclusion/shading change, then widens the rectification AABB + boosts history for locked pixels. Toggle `settings.lockThinFeatures` (`FLAG_LOCKS`); inspect via `DebugView.Locks`. Tuning constants (top of `accumulate.ts`) are defaults, not final — tighten if thin features ghost, loosen if they still dim. - - ~~**Luminance pyramid + auto-exposure**~~ — **done.** `luminancePyramid.ts` reduces the scene to a single log-average luminance (one-workgroup 32×32-tap reduction, not yet an SPD mip chain) → a pre-exposure eased over time (eye-adaptation). `accumulate.ts` pre-exposes the input before the invertible tonemap; `rcas.ts`/`blit.ts` divide it back out before display — so HDR scenes of very different brightness accumulate in the same well-conditioned range **without changing final brightness**. This also fixed a latent bug: the temporal path previously baked in `settings.exposure` and never divided it out. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); inspect via `DebugView.Exposure`. Constants (top of `luminancePyramid.ts`: key/min/max/adapt-speed) are defaults. The SPD mip chain is deferred until the shading-change detector needs the intermediate mips. - - **External exposure input** — **done & GPU-verified.** An app that meters its own exposure feeds it via `dispatch({ exposureTexture })` (value in the red texel, any float format); it overrides both auto and fixed exposure and is still divided back out before display (conditions accumulation, not brightness). Wired as binding 5 of the pyramid pass behind `FLAG_EXTERNAL_EXPOSURE` (the 1024 bit) and funnelled through the same single `select`, so downstream passes are untouched and `avgLum` stays our own measurement for the shading detector. Bound to the reactive dummy as a placeholder when absent. Also on the composable node as `options.exposureTexture` (mirrors FSR3's `exposure` dispatch resource). - - ~~**Shading-change detector**~~ — **done & GPU-verified.** `accumulate.ts` compares the reprojected history luma to the current 3×3 neighborhood mean, normalized by the neighborhood's own variance; a coherent disagreement the variance can't explain reads as a genuine shading change (light turning on, animated material) and ages **non-locked** history so it re-converges. A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with the background-dominated neighborhood mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`); inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). **Simplification:** uses the neighborhood mean, not a dedicated coarse pyramid mip — a true SPD mip (steadier on high-frequency content) is deferred to Phase 5. Constants `SHADING_LO/HI/AGE` (top of `accumulate.ts`) are conservative defaults. - - ~~**Reactive-mask input**~~ — **done & GPU-verified.** Optional `dispatch({ reactive })` render-res mask (red = reactivity); flagged pixels suppress locks, keep near-zero accumulation, and snap to the current frame (`REACTIVE_STRENGTH` in `accumulate.ts`). No mask → a 1×1 zero texture is bound and `FLAG_REACTIVE` stays off (zero cost). `UpscalePresenter.setReactiveMask()` threads it through; `examples/05-transparency` authors one by rendering the transparents' coverage and is the acceptance demo. Inspect via `DebugView.Reactivity`. **Authoring helpers** (auto-generating the mask) — **done**, see the next bullet. **Composable-node parity** — **done.** `upscale()` / `UpscalerNode` now take `options.reactive` and `options.reactiveOpaqueColor` texture nodes (registered as graph deps so the opaque buffer renders in-pipeline, jittered, aligned with color), closing an asymmetry with the raw `Upscaler`/`UpscalePass`. No dedicated node demo yet — it reuses the GPU-verified reactive dispatch (example 05, imperative) + the proven graph-dep mechanism (examples 07/09), so worst case on a plumbing miss is a silent no-op, not a crash. -- **Phase 4 — API & ecosystem:** - - ~~**RCAS denoise variant**~~ — **done & GPU-verified.** `rcas.ts` gains FSR1's `FSR_RCAS_DENOISE` path (attenuate the sharpening lobe on lone luma outliers so grain from noisy inputs isn't amplified), gated by `settings.rcasDenoise` (`FLAG_RCAS_DENOISE`, off by default). Pairs with an upstream spatial denoiser; `examples/06-screenspace-gi` toggles it on for the reduced-res SSR/GI. One of Dennis's original asks. - - ~~**Reactive-mask authoring helper**~~ — **done & GPU-verified.** `dispatch({ reactiveOpaqueColor })` auto-generates the mask from the opaque-vs-final color diff (`generateReactive.ts`, FSR2's `GenerateReactiveMask`); no explicit `reactive` mask needed. `UpscalePresenter.setReactiveOpaqueColor()` threads it; `examples/05-transparency` offers manual-coverage vs auto-diff. Caveat: jitter the opaque pass like the final or high-contrast edges leave faint reactivity (sub-pixel misalignment). - - **Transparency & Composition (T&C) mask** — **deliberately deferred** (assessed 2026-07-10). FSR2/3 takes a second render-res mask alongside reactive, but it is *not* a clean parallel: in FSR2 the T&C mask has a distinct-but-overlapping effect (a softer history-distrust than reactive, it widens the rectification AABB and interacts with locks) that is genuinely tuned. Adding it means new tuning constants and touching the accumulate blend/lock path — exactly the "if it messes us up, delay" risk — and our reactive mask (+ auto-generate) already covers the common three.js transparency case (example 05's whole point). Shipping it as a second channel that behaves identically to reactive would be misleading; shipping the *real* distinct behavior needs core tuning we shouldn't ride onto other work. Revisit if a user actually authors T&C masks and the reactive path proves insufficient. - - ~~**Drop-in driver**~~ — **done.** Two public surfaces, both GPU-verified: - - **`UpscalePass`** (`src/UpscalePass.ts`) — the imperative drop-in (graduated from `UpscalePresenter`, which is now a re-export shim). Bakes in the MRT/jitter/velocity/present recipe. Covers renderer-agnostic / non-graph use. - - **TSL nodes** (`src/UpscalerNode.ts`, native TSL, no pmndrs dep) — "the future" surface. **One node, one code path** — modelled on three's own `FSR1Node` / `TAAUNode`: - - **`upscale(color, depth, velocity, camera, options)`** (`UpscalerNode`) — the composable node, consumes reduced-res texture nodes and outputs the upscaled result. GPU-verified. **The key mechanism** (this is what a first attempt got wrong and rendered black): the inputs must be *graph dependencies* so three renders them in-graph, in dependency order, before this node's `updateBefore`. three discovers child nodes by walking the node's **own non-`_`-prefixed** properties (`Node._getChildren`) — our fields are `_`-prefixed, so `setup()` registers the inputs explicitly into `builder.getNodeProperties(this)` (exactly as `FSR1Node` does with `properties.textureNode`). With that, jitter (applied via the `onBeforeRenderPipeline` hook, like `TAAUNode`) lands because the inputs render *inside* the post render. The factory `convertToTexture`s the color (a no-op for texture/pass nodes, which is why reduced-res pass outputs keep their size — the caller controls input resolution). `UpscalerConfig` gained `renderWidth`/`renderHeight` so the node matches an externally-sized input exactly. - - **`upscaleScene(scene, camera, options)`** — a thin convenience, **not** a separate class: it builds `pass(scene, camera)` with a `{ output, velocity }` MRT at `1/ratio` and hands the texture nodes to `upscale(...)` — the same shape as three's `taau(pass.getTextureNode('output'), …)`. So the scene renders in-graph as an FSR3 input, jitter and all. `post.outputNode = upscaleScene(scene, camera)`. Examples `07-tsl-node`, `08-tsl-compose` (`.mul(vignette)`); the full SSGI/SSR stack is `09-kitchen-sink`. - - **`upscaleSpatial(color, options)`** — color-only **spatial** (FSR1/EASU) node for inputs with no motion data: no depth/velocity/camera, no history, no reconstruction. A thin facade over `path: 'spatial'` with a stand-in camera (EASU reprojects nothing; `_writeConstants` still stages `near`/`far`, which the shader ignores, so any finite values keep NaN out of the UBO). It exists so the "I only have a color texture" case has a clean door instead of `upscale(color, null, null, camera, { path: 'spatial' })` — and so defaulting the *temporal* node to jitter-on isn't a trap. - - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity now `console.warn`s once (was a silent no-op). `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. - - Color path (both): the boot renderer's `NoToneMapping` + `LinearSRGBColorSpace` makes the PostProcessing output transform identity, so the node passes its already-sRGB texture straight through. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). - - **Resolved** (was an open question): the composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. - - ~~MSAA-input support~~ — **dropped by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). -- **Phase 5 — performance & the one remaining quality item:** ~~merge dilate+depth-clip into one pass~~ **done** (`reconstruct.ts` — one render-res dispatch, one fewer intermediate round-trip; GPU-verified disocclusion unchanged). Remaining, all deferred as of 2026-07-10 (the rename + node-parity + exposure work above was the safe, high-value batch; these were held back so a core-path change wouldn't ride onto the rename merge): - - **True SPD luminance mip chain + shading-change coarse mip** — *the one remaining quality gain.* A real Single-Pass-Downsample mip chain would let the shading-change detector sample a coarse mip instead of the current 3×3 neighborhood mean, which steadies it on high-frequency content under heavy motion (fewer false positives → less needless history aging). But it's a **core-accumulate refactor with genuine tuning risk** (allocate the mip chain, bind the coarse mip, rewire + re-tune the detector, GPU-A/B it) — the kind of thing where an untuned constant silently regresses. Deserves its own focused session with heavy GPU tuning, **not** a tail-of-session add. This is the next thing to pick up. - - **Perf-only (no quality gain, correctness risk):** `textureGather` tap packing (EASU/RCAS currently use per-tap `textureLoad` — but these are the AMD-faithful ports; changing their sampling risks subtle artifacts); f16 arithmetic (`shader-f16`, needs feature detection + fallback, precision risk); bind-group caching (rebuilt per dispatch — "fine but wasteful," but caching adds stale-view-on-resize risk to the core); half-res luma analysis. None gain image quality; each adds risk to a core path the project deliberately protects — do them only when perf is the actual bottleneck. - -- **Future project (deferred) — fused GI/denoise temporal path.** Denoising very noisy screen-space inputs (SSGI especially) can't be solved by stacking a *separate* temporal denoiser before FSR3: any second temporal resolver reprojects by velocity, which is jitter-free, so it can't see FSR3's sub-pixel jitter — it rejects the misaligned history (noise survives) and cancels the jitter variance FSR3 needs (aliasing returns). Verified 2026-07-10 with three r185's `recurrentDenoise`/`temporalReproject` in `examples/10-ssgi-denoise` (kept as **experimental documentation**, not a library feature). Spatial-only denoise + FSR3-owns-temporal avoids the conflict but inherits the third-party à-trous kernel's halos/step-lines/update-cadence skipping. The real fix is to **fuse GI history into FSR3's own accumulation** — reprojected with *our* motion vectors, sampled at *our* jitter, with GI-appropriate variance handling (not the AA-tuned clip). This is genuine R&D that touches the core accumulate pass, so it's a deliberate future effort, not a quick add. Priority remains the FSR3 upscaler itself; don't spend core complexity bending upstream nodes to it. - -Frame generation (the other half of "FSR3") is **out of scope** — it needs swapchain frame pacing browsers don't expose. +## Feature status (all shipped & GPU-verified) + +The pipeline is feature-complete. This section records each feature's mechanism and its +traps — the "why" a future change must not break. The examples double as the validation +harness: locks on `04-aliasing-torture`, the reactive mask on `05-transparency` (its +explicit acceptance test), RCAS denoise on `06-screenspace-gi`. + +**Temporal fidelity:** +- **Luminance-stability locks.** Persistent display-res lock buffer in `accumulate.ts` (r = lifetime, g = locked luma), reprojected through motion; detects thin luminance outliers, grows a lock while present, breaks on disocclusion/shading change, then widens the rectification AABB + boosts history for locked pixels. Toggle `settings.lockThinFeatures` (`FLAG_LOCKS`); inspect via `DebugView.Locks`. Tuning constants (top of `accumulate.ts`) are defaults, not final — tighten if thin features ghost, loosen if they still dim. +- **Auto-exposure.** `luminancePyramid.ts` reduces the scene to a single log-average luminance (one-workgroup 32×32-tap reduction; no mip chain — nothing consumes intermediate mips) → a pre-exposure eased over time (eye-adaptation). `accumulate.ts` pre-exposes the input before the invertible tonemap; `rcas.ts`/`blit.ts` divide it back out before display — so HDR scenes of very different brightness accumulate in the same well-conditioned range **without changing final brightness**. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); inspect via `DebugView.Exposure`. Constants (top of `luminancePyramid.ts`: key/min/max/adapt-speed) are defaults. + - **External exposure input.** An app that meters its own exposure feeds it via `dispatch({ exposureTexture })` (value in the red texel, any float format); it overrides both auto and fixed exposure and is still divided back out before display (conditions accumulation, not brightness). Wired as binding 5 of the pyramid pass behind `FLAG_EXTERNAL_EXPOSURE` (the 1024 bit) and funnelled through the same single `select`, so downstream passes are untouched and `avgLum` stays our own measurement for the shading detector. Bound to the reactive dummy as a placeholder when absent. Also on the composable node as `options.exposureTexture` (mirrors FSR3's `exposure` dispatch resource). + - **Host pre-exposure (`preExposureTexture`).** DeltaPreExposure history correction + host-invariant auto-exposure metering (auto-exposure must not chase a step the app already metered — skipping this reads as a ~2s full-screen false shading change). Validated on the Q11 bench scenario; byte-identical output when the input is absent. +- **Shading-change detector** (multi-scale form, 2026-07-21). `shadingChange.ts` (one fused half-res dispatch) compares jitter-aligned block-mean luma at 4×4/8×8 render scales against a 1-frame luma history, with contrast-adaptive noise floors and disocclusion neutralization; the response ages **non-locked** history via accumulate's `SHADING_AGE` path so changed surfaces re-converge. Costs 0.044 ms at ratio 2; zero when off (pass not dispatched). A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with its block mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`); inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). +- **Reactive-mask input.** Optional `dispatch({ reactive })` render-res mask (red = reactivity); flagged pixels suppress locks, keep near-zero accumulation, and snap to the current frame (`REACTIVE_STRENGTH` in `accumulate.ts`). No mask → a 1×1 zero texture is bound and `FLAG_REACTIVE` stays off (zero cost). `UpscalePresenter.setReactiveMask()` threads it through; `examples/05-transparency` authors one by rendering the transparents' coverage and is the acceptance demo. Inspect via `DebugView.Reactivity`. **Node parity:** `upscale()` / `UpscalerNode` take `options.reactive` and `options.reactiveOpaqueColor` texture nodes (registered as graph deps so the opaque buffer renders in-pipeline, jittered, aligned with color). No dedicated node demo — it reuses the GPU-verified reactive dispatch (example 05, imperative) + the proven graph-dep mechanism (examples 07/09), so worst case on a plumbing miss is a silent no-op, not a crash. +- **Reactive-mask authoring helper.** `dispatch({ reactiveOpaqueColor })` auto-generates the mask from the opaque-vs-final color diff (`generateReactive.ts`, FSR2's `GenerateReactiveMask`); no explicit `reactive` mask needed. `UpscalePresenter.setReactiveOpaqueColor()` threads it; `examples/05-transparency` offers manual-coverage vs auto-diff. Caveat: jitter the opaque pass like the final or high-contrast edges leave faint reactivity (sub-pixel misalignment). +- **RCAS denoise variant.** `rcas.ts` has FSR1's `FSR_RCAS_DENOISE` path (attenuate the sharpening lobe on lone luma outliers so grain from noisy inputs isn't amplified), gated by `settings.rcasDenoise` (`FLAG_RCAS_DENOISE`, off by default). Pairs with an upstream spatial denoiser; `examples/06-screenspace-gi` toggles it on for the reduced-res SSR/GI. + +**Public surfaces** (both GPU-verified): +- **`UpscalePass`** (`src/UpscalePass.ts`) — the imperative drop-in (graduated from `UpscalePresenter`, which is now a re-export shim). Bakes in the MRT/jitter/velocity/present recipe. Covers renderer-agnostic / non-graph use. +- **TSL nodes** (`src/UpscalerNode.ts`, native TSL, no pmndrs dep) — "the future" surface. **One node, one code path** — modelled on three's own `FSR1Node` / `TAAUNode`: + - **`upscale(color, depth, velocity, camera, options)`** (`UpscalerNode`) — the composable node, consumes reduced-res texture nodes and outputs the upscaled result. **The key mechanism** (this is what a first attempt got wrong and rendered black): the inputs must be *graph dependencies* so three renders them in-graph, in dependency order, before this node's `updateBefore`. three discovers child nodes by walking the node's **own non-`_`-prefixed** properties (`Node._getChildren`) — our fields are `_`-prefixed, so `setup()` registers the inputs explicitly into `builder.getNodeProperties(this)` (exactly as `FSR1Node` does with `properties.textureNode`). With that, jitter (applied via the `onBeforeRenderPipeline` hook, like `TAAUNode`) lands because the inputs render *inside* the post render. The factory `convertToTexture`s the color (a no-op for texture/pass nodes, which is why reduced-res pass outputs keep their size — the caller controls input resolution). `UpscalerConfig` has `renderWidth`/`renderHeight` so the node matches an externally-sized input exactly. + - **`upscaleScene(scene, camera, options)`** — a thin convenience, **not** a separate class: it builds `pass(scene, camera)` with a `{ output, velocity }` MRT at `1/ratio` and hands the texture nodes to `upscale(...)` — the same shape as three's `taau(pass.getTextureNode('output'), …)`. So the scene renders in-graph as an FSR3 input, jitter and all. `post.outputNode = upscaleScene(scene, camera)`. Examples `07-tsl-node`, `08-tsl-compose` (`.mul(vignette)`); the full SSGI/SSR stack is `09-kitchen-sink`. + - **`upscaleSpatial(color, options)`** — color-only **spatial** (FSR1/EASU) node for inputs with no motion data: no depth/velocity/camera, no history, no reconstruction. A thin facade over `path: 'spatial'` with a stand-in camera (EASU reprojects nothing; `_writeConstants` still stages `near`/`far`, which the shader ignores, so any finite values keep NaN out of the UBO). It exists so the "I only have a color texture" case has a clean door instead of `upscale(color, null, null, camera, { path: 'spatial' })` — and so defaulting the *temporal* node to jitter-on isn't a trap. + - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity `console.warn`s once. `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. + - Color path (both): the node emits linear/HDR color. When it is the final graph node, three's RenderPipeline applies the renderer's configured tone mapping and output color space; otherwise it can feed later linear post-processing. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). + - The composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. +- **Temporal guides (contract accepted — M6 PASS 2026-07-24; only the TSL node + stays `@experimental`).** The production + working set is published as `upscaler.guides` (`TemporalGuides` — dilated + motion/depth, disocclusion, reactive, shading change, exposure, locks, history) + and the frame can be driven split: `dispatchGuides({depth, velocity})` right + after the G-buffer (geometry guides only — reconstruct is the whole early + stage), then `dispatchUpscale({color, …})`; `path: 'guides'` runs the early + stage alone with no output texture. Contracts + program plan: + `TEMPORAL-GUIDES-SPEC.md` (root); consumer M0 review: `GUIDES-SPEC-RESPONSE.md`. + **Mechanisms a change must not break:** (1) guide textures are allocated via + `_createSharedTexture` — a three `StorageTexture` + `initTexture()`, with the + raw handle fetched back through `getGPUTexture()`; passes bind the raw handle, + consumers sample the three texture, and the two must stay the same allocation. + r32float products are pinned `NearestFilter` (non-filterable format — a linear + sampler on them is a WebGPU validation error). (2) Ping-ponged products resolve + through `_latestDepthWrite`/`_latestHistoryWrite` (set at encode time), NOT the + frame-end-flipped `_depthIndex`/`_historyIndex` — the getters must be correct + both mid-frame (between the split dispatches) and after the frame. (3) The + monolithic `dispatch()` stays one submit (`_encodeGuides` + `_encodeLate` on one + encoder) — GPU-verified byte-identical (Q0 captures) and perf-neutral (−2.7%, + within noise) against the pre-split pipeline; keep it that way. (4) A split + frame is two submits, so `GpuTimer` merges per-label results instead of + replacing the map. (5) Frame-end bookkeeping (index flips, `_frameIndex`, + `_pendingReset`) happens exactly once per frame: in `dispatch()`, in + `dispatchUpscale()`, or — guides path only — in `dispatchGuides()`. + `examples/12-temporal-guides` is the live reference + headless-verification + target (it exposes `window.__guidesExample` — including `MomentsPass` and + `THREE` — for the CDP harness). **TSL surface (M4):** + `temporalGuides(depth, velocity, camera)` (`TemporalGuidesNode.ts`) + publishes the bundle as texture nodes (`getTextureNode(name)` — stable + node identity, ping-ponged products re-pointed per frame; a 1×1 nearest + placeholder pre-configure so r32float format inference never sees a + filterable stand-in). Standalone = node owns a guides-only upscaler sized + to its depth input; linked = `upscale(..., { guides })` adopts the node's + upscaler via `_acquireUpscaler` and runs the split frame in-graph, with + the upscale node falling back to monolithic `dispatch()` whenever the + early stage didn't run this frame (`Upscaler.guidesPending` is the + branch). The guides node must be registered as a graph dep BEFORE the + color chain so its dispatch precedes effect renders. + `examples/13-guides-node` is its live reference (exposes + `window.__guidesNodeExample` + tsl handles for the CDP harness; the + dispatch-spy probe there proves the pure split path steady-state). Also in this program: reactive is + merge-not-overwrite (`generateReactive` max-merges an incoming mask; + passing `guides.reactive` back while `reactiveOpaqueColor` is set throws — + the generator writes that texture), and `MomentsPass`/`shaders/moments.ts` + is a standalone signal-agnostic statistics primitive with **zero coupling** + to the pipeline (its `FLAG_MOMENTS_YCOCG` bit is declared locally in + moments.ts, deliberately NOT in `WGSL_CONSTANTS` — adding anything to the + shared chunk re-fingerprints every shader). +- **MSAA input — rejected by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). + +**Performance structure:** dilate + depth-clip are fused into the single `reconstruct.ts` dispatch (GPU-verified disocclusion unchanged); the shading detector is one fused workgroup-local reduction instead of the source's SPD mip chain + resolve pair. The measured story of these divergences from FSR 3.1.5 — and the four upstream behaviors adopted in re-derived form — is `PARITY.md` (root) with evidence in `bench/docs/NEXT-STEPS.md`. + +**Paper material:** findings that clear the "surprised us + measured + others would hit it" bar are tracked in `PAPER-NOTES.md` (root) — claim, evidence pointers, and what a publication-grade version still needs. Add new entries there as they land; don't let them live only in commit messages. + +## Deferred / out of scope + +- **Transparency & Composition (T&C) mask** — deliberately deferred (assessed 2026-07-10). FSR2/3 takes a second render-res mask alongside reactive, but it is *not* a clean parallel: in FSR2 the T&C mask has a distinct-but-overlapping effect (a softer history-distrust than reactive, it widens the rectification AABB and interacts with locks) that is genuinely tuned. Adding it means new tuning constants and touching the accumulate blend/lock path, and our reactive mask (+ auto-generate) already covers the common three.js transparency case (example 05's whole point). Shipping it as a second channel that behaves identically to reactive would be misleading; shipping the *real* distinct behavior needs core tuning we shouldn't ride onto other work. Revisit if a user actually authors T&C masks and the reactive path proves insufficient. +- **Perf-only micro-optimizations (no quality gain, correctness risk):** `textureGather` tap packing (EASU/RCAS currently use per-tap `textureLoad` — but these are the AMD-faithful ports; changing their sampling risks subtle artifacts); f16 arithmetic (`shader-f16`, needs feature detection + fallback, precision risk); bind-group caching (rebuilt per dispatch — "fine but wasteful," but caching adds stale-view-on-resize risk to the core); half-res luma analysis. None gain image quality; each adds risk to a core path the project deliberately protects — do them only when perf is the actual bottleneck. +- **Future project — fused GI/denoise temporal path.** Denoising very noisy screen-space inputs (SSGI especially) can't be solved by stacking a *separate* temporal denoiser before FSR3: any second temporal resolver reprojects by velocity, which is jitter-free, so it can't see FSR3's sub-pixel jitter — it rejects the misaligned history (noise survives) and cancels the jitter variance FSR3 needs (aliasing returns). Verified 2026-07-10 with three r185's `recurrentDenoise`/`temporalReproject` in `examples/10-ssgi-denoise` (kept as **experimental documentation**, not a library feature). Spatial-only denoise + FSR3-owns-temporal avoids the conflict but inherits the third-party à-trous kernel's halos/step-lines/update-cadence skipping. The real fix is to **fuse GI history into FSR3's own accumulation** — reprojected with *our* motion vectors, sampled at *our* jitter, with GI-appropriate variance handling (not the AA-tuned clip). This is genuine R&D that touches the core accumulate pass, so it's a deliberate future effort, not a quick add. Priority remains the FSR3 upscaler itself; don't spend core complexity bending upstream nodes to it. +- **Frame generation** (the other half of "FSR3") — needs swapchain frame pacing browsers don't expose. --- @@ -205,4 +293,4 @@ Follow the existing style: ## Provenance / license -MIT (`LICENSE`). The EASU/RCAS WGSL derives from AMD's MIT-licensed FidelityFX (`ffx_fsr1.h`); AMD's copyright notice is in `LICENSE`. Preserve it. When porting more FidelityFX stages (Phase 3), keep the "faithful port vs. simplification" table in `src/shaders/README.md` honest. +MIT (`LICENSE`). The EASU/RCAS WGSL derives from AMD's MIT-licensed FidelityFX (`ffx_fsr1.h`); AMD's copyright notice is in `LICENSE`. Preserve it. If more FidelityFX stages are ever ported, keep the "faithful port vs. simplification" table in `src/shaders/README.md` honest. diff --git a/GUIDES-HANDOFF-RESPONSE.md b/GUIDES-HANDOFF-RESPONSE.md new file mode 100644 index 0000000..23d8b7f --- /dev/null +++ b/GUIDES-HANDOFF-RESPONSE.md @@ -0,0 +1,263 @@ +# GUIDES-HANDOFF response — consumer integration report 1 + +Consumer: ssgiDev bench, demo `16-fsr` (first full-pipeline consumer). +Against: `feat-temporal-guides` @ `34f784d`. Format per the M0 feedback loop: +accepted / friction / blocked per item. + +> **Status 2026-07-22 (their `3e374fd`): both friction items RESOLVED.** +> Dead `_renderer` deleted (their src now passes `--noUnusedLocals`); our +> vite block is the canonical linked-build recipe in GUIDES-HANDOFF.md; +> `paths` → `dist/index.d.ts` is the documented recommendation with their +> commitment to rebuild dist whenever the public surface moves (dist rebuilt +> at that commit — verified fresh, bench tsc clean against it). No contract +> changes; sequencing unaffected. Items below kept for the record. + +## Linked build (handoff option a) — ACCEPTED, one real hazard + +The vite alias works, but the handoff's "no duplicate-three hazard" claim is +**wrong for out-of-root consumers**: your source imports bare `three` while +our app imports `three/webgpu`, and vite resolved them to two copies of core +(from two different node_modules — your 0.185.0, our 0.185.1) → the +"Multiple instances of Three.js" warning. All three of these were needed in +the consumer's vite config: + +```ts +resolve: { + alias: [ + { find: '@pmndrs/upscaler', replacement: '/Users/dex/Developer/fsr3/src/index.ts' }, + { find: /^three$/, replacement: 'three/webgpu' }, // collapse dual-entry core + ], + dedupe: ['three'], // pin to the CONSUMER's copy, not fsr3/node_modules +}, +optimizeDeps: { exclude: ['three'] }, // vite pre-alias shortcut otherwise + // beats the bare->bare alias +``` + +Suggest folding this block into GUIDES-HANDOFF.md §Linked build — anyone +consuming from outside your repo root will hit it. + +## TypeScript surface — FRICTION (worked around) + +- Aliasing tsconfig `paths` at `src/index.ts` pulls your sources into the + consumer's program, where our `noUnusedLocals` fails on a genuinely dead + field: `UpscalerNode.ts:141` `_renderer` is assigned (line 175) and never + read. We now point `paths` at `dist/index.d.ts` instead — cleaner + separation anyway — but the field is worth deleting. +- Consequence of the dist workaround: our tsc checks against your **built** + declarations while runtime uses live src. Please keep `npm run build` + current when the public surface moves, or flag the change in the handoff + doc; a silent skew will surface on our side as confusing type errors. + +## Guides bundle + split dispatch (M2) — ACCEPTED, verified live + +Demo 16 drives the exact contract frame shape (beginFrame → MRT +color+velocity+float-depth at render res → endFrame → dispatchGuides → +dispatchUpscale → present). Headed verification on cornell + sponza, WGSL +console clean: + +- temporal 2× reconstructs far-field detail bilinear 2× destroys (sponza + hall: lion relief, banner fringes, floor tiling) — no ghosting observed + on static + slow-orbit content; +- ratio 1 temporal (Native AA) — crisp; this bench renders + `antialias: false`, so this is its first true AA, as planned; +- `DebugView.Disocclusion` under a slow orbit shows exactly thin + trailing-silhouette strips; `AccumulationAge` shows the jitter-phase + pattern with converged borders; +- ping-pong rule honored (getters re-read per frame) — no stale-half + artifacts seen. + +Not yet exercised: reactive (bench scenes have no transparents yet), +guides-only path, MomentsPass (next — see below). + +## Sequencing on our side (your M6 dependency) + +1. **D1 alignment next**: our private `TemporalGuidesPass` consumers still + speak NDC-delta motion; we convert them to your UV-delta (`prevUV = uv − + motion`) convention, then run the guides A/B (guides-fed SSGI temporal + vs private logic) — your M6 exit criterion. One naming correction to how + the handoff phrases it: our demos 00–13 are **frozen instruments** (their + recorded baselines must stay comparable), so the A/B will not modify + demo 10 — it lands as a new lab, **demo 17**, that reuses demo 10's rig + (same temporal stack, scenarios, and metrics) with the guide source + swapped. Functionally identical to "the demo-10 A/B" in your docs. +2. `14-svgf` per SVGF-SPEC consumes `upscaler.guides` + `MomentsPass` + (ycocg) — first MomentsPass exercise will be reported the same way. + +Nothing blocked. + +--- + +# Report 2 — M6 recorded: PASS. SVGF/MomentsPass field report. + +Consumer demos: `17-guides-ab` (M6 rig), `14-svgf` (MomentsPass consumer). +Against: `feat-temporal-guides` @ `3e374fd`. Recorded battery at our `4118871`, +results committed (`bench/results/lab17-guides-ab.*`, `lab14-svgf.*`, +`lab14-cost.*`). + +## M6 — PASS. Drop `@experimental`. + +Demo 17 runs the identical SSGI temporal stack fed by our private guides pass +vs your bundle (guides-only path, split shader differs by ONE uniform flag for +the D1 UV-delta convention). Recorded: + +- **Still-camera stability: bit-identical** (both arms 1.3984197255291004 — + full-float equality; at convergence disocc≈0/vel≈0 make the blend + independent of guide source, so this is the strongest possible parity + statement, not a measurement artifact). +- **Teleport reconvergence: 1.275 s vs 1.288 s** (yours) — inside one 500 ms + sampling interval. Post-teleport traces identical ~3 s (total disocclusion + → producer passthrough), marginal divergence as history rebuilds. +- Functional pre-check: across-arm meanAbsDiff 0.84 < within-arm temporal + noise 1.40 on stills. + +Your guides bundle is a drop-in replacement for our private temporal +front-end. By GUIDES-HANDOFF.md's own criterion, the guides API can drop +`@experimental`. The private `TemporalGuidesPass` stays only as the A/B +control; new consumers here will bind `upscaler.guides`. + +## MomentsPass field report (demo 14 SVGF) + +- Integrated exactly per your §MomentsPass docs (`space: 'ycocg'`, dispatch + per frame on demodulated GI irradiance; `coarseMoments` as the short-history + spatial variance fallback, smooth-ramped over history length instead of the + spec's hard branch). No contract friction: formats, sizes, and the + rgba16float `.rg` deviation all behaved as documented. +- Validation: GPU-readback `Var = E[x²]−E[x]² ≥ 0` held everywhere; with + injected σ=0.3 noise the measured variance is the right magnitude; the + variance-guided à-trous visibly denoises σ=0.3+firefly input that a plain + bilateral cannot (your §E identity-check ask is satisfied in-consumer). +- Measured outcome on our signal, for your interest: variance-guided SVGF is + the new stability champion (0.035–0.24 across all grid cells, 7–88× better + than our incumbent) at 2.4–2.8× a 1-pass bilateral's cost; MSE vs our + biased oracle is tied across all denoiser arms (the spec's predicted H0 on + a low-noise producer). MomentsPass is real and load-bearing, not cosmetic. + +## Open on our side + +15-ptref (unbiased PT ground truth) remains deferred — it is the only way to +turn the MSE tie into a real quality ranking. No asks on your side. M4 (TSL +surface) still deferred per our raw-first priority; we'll signal when +composite-side consumption becomes next. + +--- + +# Report 3 — upscale-path convergence defect (human-observed, then quantified) + +Dennis observed the demo-16 output visibly jittering on a STILL camera, the +`Disocclusion` debug view flickering silhouette outlines, and `AccumulationAge` +never resolving (rolling ring/moiré patterns forever). We quantified: + +- **Still-camera output never converges.** Consecutive-frame meanAbsDiff + (0–255 scale, 250 ms apart, after 6–8 s settle), cornell C-wide pose: + sustained **0.19–0.76** at defaults. Reference point: our SVGF demo after + its own jitter fix sits at **0.039** flat on the same pose. Sponza S-hall: + steady 0.15–0.18 — better, still far from converged. +- **AccumulationAge drifts ~5.5–6.2** meanAbsDiff over any 5 s window — the + age pattern rolls forever; history is being continuously re-aged. +- **No exposed knob stops it**: `detectShadingChanges` off (no change — and + output slightly worse), `autoExposure` off (no change), `sharpness=0` + (RCAS off — no improvement), `lockThinFeatures` off (no change), + **ratio 1 NativeAA** (no upscaling at all): still 0.15–0.42. +- **Scene-dependent aggravator, not cause**: cornell is worse than sponza. + Our cornell point light uses three's IGN-dithered Vogel shadow filter, + which is screen-anchored — under your camera jitter the penumbra dither + re-rolls every frame, i.e. we feed genuinely unstable luminance in + penumbrae. That explains cornell>sponza but not the sponza floor, nor + NativeAA churn on flat walls. + +Repro on our side: demo 16, `?demo=16-fsr&scene=cornell&cam=0,2.4,8.8,0,2.6,0`, +any consecutive-frame differ (our meter script is +`scratchpad/jitter-meter.mjs` pattern — 5 grabs at 250 ms, meanAbsDiff). + +Expectation check: FSR2/3-class temporal accumulation on a static scene + +still camera should converge to a supersampled stable image (frame diff +→ ~0). If your reference example shows the same on a frozen camera, this is +in the accumulation/rectification core; if not, tell us what input contract +we're violating — depth/velocity/deltaTime all follow GUIDES-HANDOFF.md and +demo 16 passed its M2 field verification for image quality (report 1). + +**Program impact on our side**: the upscale+AA acceptance work (FSR3-BRIEF +Req 2; jitterAA default-on plans) is BLOCKED on this. **M6/guides are NOT +implicated** — demo 17 consumes the guides-only path (no upscale +accumulation), its A/B numbers stand, and report 2's drop-`@experimental` +verdict for the guides API is unaffected. + +## Report 3 addendum (2026-07-24) — converging counter-example + likely mechanism + +We have since built our own sub-pixel-jittered temporal accumulator (demo +`18-temporal`, commit `67e924c`) over the same scenes, camera poses, and +churn meter. It **converges**: still-camera consecutive-frame meanAbsDiff +0.447 → 0.242 over 12 s and still falling (α = 1/N arm), teleport recovery +0.57 → 0.18 over 10 s, no ghosting. So jitter + temporal accumulation over +this exact content is convergent — the demo-16 churn is not something our +scenes force. + +Two implementation facts were REQUIRED to get there; offered as debugging +hints because the failure mode without them reproduces demo 16's symptom set +exactly (rolling `AccumulationAge`, silhouette `Disocclusion` flicker, +permanent output shimmer): + +1. **Velocity includes the jitter delta** (three's velocity node tracks the + previous *projection* matrix). True per-pixel motion is + `velocityUV − jitterDeltaUV`; any disocclusion/rectification test that + consumes raw velocity sees every pixel "moving" every frame. +2. **Sub-pixel-motion pixels must accept same-texel history unconditionally.** + At geometry edges the per-frame depth flip under jitter IS the coverage + being integrated; running a depth/world-delta reject there resets history + age every frame. We also had to widen the reject threshold for genuinely + moving pixels under jitter (nearest-texel prev-depth carries up to a texel + of sub-pixel reconstruction error). + +If the upscaler's rectification or disocclusion logic under-compensates +jitter in either of these two ways, that would produce precisely the +observed non-convergence, including at NativeAA ratio 1. Converging repro +for comparison: `?demo=18-temporal`, defaults, still camera. + +--- + +# Upscaler-side response to report 3 (2026-07-24) — CONFIRMED, RESOLVED + +Your report was correct, the defect was ours, and your addendum's hint 2 was +the right neighborhood. Reproduced in our own bench with zero consumer code +(scenario Q1, still camera, capture mode): sustained consecutive-frame +meanAbsDiff **0.211** — and, decisively, **0.182 between frames at the SAME +jitter phase** one period apart, i.e. genuinely aperiodic history churn, not +the benign per-phase pattern. Your input contract was never violated; no +demo-16 change is needed. + +Three stacked core defects, all fixed on `feat-temporal-guides`: + +1. **Depth-clip vote starvation** (reconstruct). Only positive-separation + taps voted or carried weight, so at a still silhouette one bilinear tap + straddling the previous frame's texel-quantized dilation boundary became + the sole voter → disocclusion 1.0 per phase — your flickering + `Disocclusion` outlines. Now every valid tap votes (agreement = full + confidence) and the best tap wins; genuine trails still read ~1. +2. **Clip-magnitude history aging** (accumulate, removed). Aging the sample + count by clip amount pinned equilibrium age low wherever the converged + mean sat outside one jitter phase's variance box — your rolling + `AccumulationAge` moiré, at every ratio including NativeAA. +3. **Clip write-back** (accumulate). The blend stores the *clipped* history, + so each phase's box re-snapped the buffer regardless of blend weight. Now + still + converged + quiet pixels (no motion/disocclusion/shading-change/ + reactivity) get a ×9-widened rectification box; any of those signals + restores full rectification, so motion behavior is unchanged. + +Post-fix, same meter definition as yours (0–255 meanAbsDiff, consecutive +frames after settle): Q1 torture scene 2× **0.112** (was 0.211), NativeAA +ratio 1 **0.081** (phase-locked 0.003), and a new **Q12 +cornell-still-convergence** scenario built to your repro recipe (enclosed +box, point light with three's IGN-dithered Vogel shadows, your camera pose): +**0.024** — below your SVGF reference's 0.039, with the disocclusion view +fully black and age saturated. No-regression: Q3 object-motion disocclusion +shows only the documented thin trailing crescents, finals ghost-free; Q4 +camera orbit clean. Full evidence ladder: `bench/docs/NEXT-STEPS.md` §5; +churn meter: `scripts/measure-convergence.mjs`. + +One nuance for your acceptance criteria: a bounded-memory (EMA) accumulator +never reaches your α = 1/N arm's asymptotic zero — the floor is +~(current-frame phase variation)/maxAccumulation. The numbers above are at +`maxAccumulation` 24; your knob raises it to 64 if you want a lower floor at +the cost of slower response. The FSR3-BRIEF Req 2 / jitterAA-default-on work +should be unblocked; we'd welcome a re-run of your demo-16 meter against the +updated branch. diff --git a/GUIDES-HANDOFF.md b/GUIDES-HANDOFF.md new file mode 100644 index 0000000..da6f351 --- /dev/null +++ b/GUIDES-HANDOFF.md @@ -0,0 +1,177 @@ +# Temporal guides — consumer handoff + +Audience: the agent building the guides-fed labs (demo-10 SSGI temporal A/B, +demo-14/15 SVGF) as a **linked-build consumer** of this repo. +State: branch `feat-temporal-guides`, all raw-path milestones landed and +GPU-verified. This doc is the integration entry point; the normative +contract stays [TEMPORAL-GUIDES-SPEC.md](TEMPORAL-GUIDES-SPEC.md) (your M0 +review [GUIDES-SPEC-RESPONSE.md](GUIDES-SPEC-RESPONSE.md) is folded in as +§10). + +## What is ready for you + +| Capability | Status | Where | +|---|---|---| +| Guides bundle (`upscaler.guides`) — dilated motion/depth, prev depth, disocclusion, reactive, shading change, exposure, locks, history as ordinary three textures | ✅ M2 | `Upscaler.guides`, contracts on the `TemporalGuides` type | +| Split frame: `dispatchGuides({depth, velocity})` post-G-buffer → effects → `dispatchUpscale({color})` | ✅ M2 | `Upscaler` | +| Guides-only mode (no upscale at all) | ✅ M2 | `configure({ path: 'guides' })` | +| Reactive merge-not-overwrite (+ effect-writable `guides.reactive`) | ✅ M3 | `DispatchInputs.reactive` docs | +| `MomentsPass` — signal-agnostic (E[x], E[x²]) + one coarse level, linear or YCoCg-Y | ✅ M5 | `MomentsPass` export | +| Grazing-angle disocclusion stability fix | ✅ | commit `b16274a` — **baseline your lab on this or later**; earlier disocclusion flickered on grazing planes | +| TSL node surface: `temporalGuides()` publishes the bundle as texture nodes; `upscale({guides})` shares one computation (split frame in-graph) | ✅ M4 | `temporalGuides` / `TemporalGuidesNode` exports; live reference `examples/13-guides-node`. Built post-handoff so it's ready when your composite-side consumption lands — nothing in your raw path depends on it | + +Everything guides-related is `@experimental`: the contract is frozen (M0) +but may still shift until your integration (M6) accepts. Flag friction in a +response doc rather than working around it. + +## Linked build + +Two options; (a) is what this repo's own bench/examples do and gives you +shader hot-reload while we iterate. + +**(a) Vite alias straight at the source (recommended for the labs):** + +```ts +// vite.config.ts in your repo (block verified by the demo-16 integration) +resolve: { + alias: [ + { find: '@pmndrs/upscaler', replacement: '/Users/dex/Developer/fsr3/src/index.ts' }, + { find: /^three$/, replacement: 'three/webgpu' }, // collapse dual-entry core + ], + dedupe: ['three'], // pin to YOUR copy, not fsr3/node_modules +}, +optimizeDeps: { exclude: ['three'] }, // else vite's pre-bundle shortcut beats the alias +build: { target: 'esnext' }, // the lib uses modern syntax; examples use TLA +``` + +The bare-imports design means the library *can* use your three instance — +but an out-of-root consumer does NOT get that for free: vite resolves our +`three` and your `three/webgpu` from two different `node_modules` (two core +copies → the "Multiple instances of Three.js" warning, and worse, two +backend maps). All three extra lines above are load-bearing; they came out +of the first real integration (GUIDES-HANDOFF-RESPONSE.md). + +TypeScript path (for editor types): point `paths` at the **built +declarations**, not the source — `"paths": { "@pmndrs/upscaler": +["/Users/dex/Developer/fsr3/dist/index.d.ts"] }`. Aliasing at `src/` +pulls our sources into your program and subjects them to your compiler +flags (the first integration hit `noUnusedLocals` on our code). The trade: +your tsc sees `dist/` while runtime uses live `src/` — **we keep `npm run +build` current whenever the public surface moves** (it's part of our commit +gate for API changes); if you hit a type error that looks stale, re-run the +build here first. + +**(b) Packed dependency (for anything that shouldn't track our working tree):** + +```bash +cd /Users/dex/Developer/fsr3 && npm run build # dist/ + declarations +# your repo: +npm install /Users/dex/Developer/fsr3 # file: dependency +``` + +Peer requirement: `three >= 0.184`. We verify internals against r184; your +r185 harness uses the same `renderer.backend.get(...)` shape, and our bridge +(`internal/threeWebGPU.ts`) throws loudly if a three bump ever changes it. + +## Consuming the bundle (raw path) + +```ts +import { Upscaler } from '@pmndrs/upscaler'; + +const upscaler = new Upscaler({ renderer }); +upscaler.init(); +upscaler.configure({ displayWidth, displayHeight, customUpscaleRatio: 2, path: 'temporal' }); +// jitter-free velocity (only relevant when you jitter — temporal path): +velocity.setProjectionMatrix(upscaler.unjitteredProjectionMatrix); + +// per frame +upscaler.beginFrame(camera); +/* render G-buffer / depth+velocity MRT */ +upscaler.endFrame(camera); +upscaler.dispatchGuides({ depth, velocity, deltaTime }, camera); +/* your effects: bind upscaler.guides.* — valid from here */ +upscaler.dispatchUpscale({ color, deltaTime }, camera); +``` + +- Guides are three `Texture`s. For raw WGSL passes, resolve the `GPUTexture` + with your own bridge (`renderer.backend.get(tex).texture`) — same shape as + your harness's `three-internals.ts`. +- **Re-read the getters every frame.** Ping-ponged products (`dilatedDepth` + / `previousDepth`, `exposure`, `lockStatus`, `history`) resolve to the + most-recently-written half; caching a texture reference across frames + gives you a stale half every other frame. +- `r32float` products (`dilatedDepth`, `previousDepth`, `shadingChange`) + are non-filterable: `textureLoad` or a nearest sampler only. They ship + with `NearestFilter` set for TSL use. +- Motion convention (your accepted D1): `.xy` is a **UV delta**, y-flip + applied — `prevUV = uv - motion`. Align your harness pass before the swap, + as planned. +- Late products consumed by render-stage effects are **frame N−1 priors** + (your accepted D2). Early products are same-frame after `dispatchGuides`. +- Undocumented channels are reserved (you acknowledged this) — e.g. locks + `.b` already carries shading age. +- Guides-only apps: `path: 'guides'`, then `dispatchGuides` is the whole + frame (~0.04 ms at 1080p render). No color, no output texture (the + accessor throws with an explanation), no jitter. + +Reactive is bidirectional: write reactivity into `guides.reactive` between +the two dispatches and pass that same texture as `dispatchUpscale`'s +`reactive` input — or, when also using `reactiveOpaqueColor`, pass a +*different* texture and it max-merges with the generated diff (passing +`guides.reactive` itself in that combination throws: the generator writes +that texture). + +## MomentsPass (SVGF statistics half) + +```ts +import { MomentsPass } from '@pmndrs/upscaler'; +const moments = new MomentsPass({ renderer }); +moments.configure({ width, height, space: 'ycocg' }); // your §8 decision 1 +moments.dispatch({ source: giIrradiance }); // any float texture, per frame +// moments.moments rgba16float, source size: .r = E[x], .g = E[x²] +// moments.coarseMoments rgba16float, ceil(size/4): 4×4 block means +``` + +- Signal-agnostic by construction: no exposure/tonemap/albedo assumption; + a single-channel source (r32float GI luma) reads through unchanged in + `'linear'` space. +- One deviation from the spec's sketch: outputs are **rgba16float with + `.rg` used** — `rg16float` is not a core WebGPU storage format. `.ba` + reserved. +- One coarse level only (your answer 4: nothing reads deeper). +- GPU-verified against a CPU reference in both spaces (<0.1% rel. error, + f16); the `Var = E[x²] − E[x]²` identity check your spec §E asks for is + the same harness — rerun it in your lab if you want it in your evidence + chain (`scripts` equivalent lives in this repo's session notes; the pass + itself is deterministic). + +## Verification expectations on your side + +- Your demo-10 A/B (guides-fed SSGI temporal vs private logic) is **M6 — + the program's exit criterion**. The guides API drops `@experimental` when + it passes. +- `examples/12-temporal-guides` (npm run examples → :5300) is the live + reference: split dispatch + the guide views, and it exposes + `window.__guidesExample` (upscaler, renderer, camera, `Upscaler`, + `MomentsPass`, `THREE`, `getRenderTarget`) for headless CDP harnesses. +- Headless WebGPU on this machine works with + `--headless=new --enable-unsafe-webgpu --use-angle=metal` (our harness + drives it over CDP; screenshots + `Log.entryAdded` catch WGSL errors). +- If disocclusion misbehaves in your scenes: check `DebugView.Disocclusion` + expectations in CLAUDE.md first, and note the grazing-angle fix landed + `b16274a` — re-pull before debugging on your side. + +## Feedback loop + +Respond the way M0 worked: a doc in this repo (or a note pointing at one in +yours) with accepted/friction/blocked per item. Known-open items on our +side: the bench merged-mask capture scenario (deferred to your lab +exercising the real merge) and the `@experimental` freeze pending your M6. +(M4, deferred at handoff, has since landed — see the readiness table.) + +**Report 1 received** ([GUIDES-HANDOFF-RESPONSE.md](GUIDES-HANDOFF-RESPONSE.md), +demo-16, against `34f784d`): linked build + M2 contract accepted and verified +live on cornell/sponza; nothing blocked. Both friction items are resolved +here — the vite block above is theirs, the dead `UpscalerNode._renderer` +field is deleted, and the tsconfig-`paths` guidance now points at `dist/`. +Next on their side: D1 motion-convention alignment, then the demo-10 A/B (M6). diff --git a/GUIDES-SPEC-RESPONSE.md b/GUIDES-SPEC-RESPONSE.md new file mode 100644 index 0000000..ceabdd4 --- /dev/null +++ b/GUIDES-SPEC-RESPONSE.md @@ -0,0 +1,113 @@ +# GUIDES-SPEC-RESPONSE.md — consumer-side M0 review (bench → FSR port) + +Response to `TEMPORAL-GUIDES-SPEC.md` §7 (deviations) and §9 (open +questions), plus the pre-build decisions `SVGF-SPEC.md` §8 asks for. +Written 2026-07-21 from the bench/consumer side. Nothing here is code; per +M0, contract agreement precedes build on both sides. + +## Verdict up front + +The spec's central move — **publish products, not pass boundaries (D5)** — +is accepted and is better than what FSR3-BRIEF asked for. The brief wanted +the split because it wanted (a) the products and (b) early-stage-without- +upscale; `dispatchGuides()` / `path: 'guides'` delivers both while keeping +the port's measured fusions. Our own harness prototype went through the same +reasoning at smaller scale (one class, three dispatches, texture getters — +consumers bind textures, never passes), so the contract shapes already match +in spirit. FSR3-BRIEF's "Consumption requirements v2" section (commit +c7ba382) should be read with its pass-boundary language superseded by D5; +its Req-1 acceptance (the live demo 10 rig) stands unchanged and maps to +your M6. + +## §9 answers + +**1. D1 — UV-delta motion: ACCEPTED, and preferred.** Publishing the +directly-usable form (`prevUV = uv − motion`, y-flip pre-applied) deletes +the convention seam that cost us the demo 03 speckle hunt. Raw NDC is not +needed — every consumer that wants it owns the velocity MRT it came from. +Consumer-side consequence we own: our harness `TemporalGuidesPass` currently +publishes the raw-velocity convention (consumers apply `· (0.5, −0.5)` +themselves, e.g. `ssrt3-guides-temporal.wgsl`). Before the swap we will +align the harness pass to publish YOUR convention, so the port's bundle is +drop-in against unmodified consumer shaders. That alignment is bench-side +work, scheduled with the swap, not before. + +**2. D2 — frame N−1 locks/age: SUFFICIENT.** A prior is previous-frame by +definition. Our only shipping consumer of history age (demo 13 `adaptive`) +already reads the *previous* frame's N — that is how it works today, at +3.17× — so N−1 latency is the semantics we validated, not a compromise. +Same for the SVGF history-rejection prior. + +**3. D3 — display-res age sampling: fine for day one.** Accepted with one +recorded risk: our adaptive march reads N per march-res pixel (currently +half render-res); sampling display-res age at march UV is a filtered +approximation and may dilate age across silhouettes → transient over-effort +at edges (safe direction: over-effort, never under-quality). The guides lab +measures exactly that. Fallback exists and costs you nothing: demo 03/12/13 +keep their own render-res counter (the brief always allowed the consumer to +produce `historyLength`); we only retire it if the lab shows display-res +sampling is clean. + +**4. Moment pyramid: 3 levels.** SVGF's short-history fallback is one +coarse-neighborhood read (7×7-equivalent ≈ mip 2); nothing in the SVGF-SPEC +chain reads deeper. Allocate mip 0 + 2. + +**5. Consumption mechanism: raw bind groups first, TSL second.** Every +hot-path consumer (march, temporal, à-trous) is raw WGSL; TSL consumption is +wanted only composite-side (display/debug and the bent-normal/spec-occlusion +env terms). Per-frame texture re-pointing is fine — our `RawComputePass` +re-binds per execute, and our TSL usage follows the same `updateBefore` +re-point pattern you cite — so the stable-identity copy can stay spec'd-only. +If M4 needs trimming, defer the TSL example, not the raw path. + +**D4 (formats), D6 (environment):** accepted; format is read from the +texture at bind time on our side, channel conventions (.r reads) noted. +Reserved-channel rule acknowledged — we will never read unlisted fields. + +**Producer-of-record confirmed:** the SVGF addendum (§A/§B/§E, +`MomentPyramid`, the signal-&-space rule) is present in this repo's +`FSR3-BRIEF.md` and matches what both specs cite. (An earlier draft of this +response flagged a sync gap — retracted; it was a stale read on the consumer +side.) One brief-side reconciliation was applied instead: the brief's +"Consumption requirements v2" and bundle table now note that the accepted +deviations D1 (UV-delta motion) and D4 (port-native formats) supersede the +brief's suggested convention/formats — see the brief's "Contract resolution" +note. + +## SVGF-SPEC §8 pre-build decisions (recommendations) + +1. **Moments: scalar luminance in YCoCg-Y** (spec default). Two channels; + revisit RGB variance only if chroma noise visibly survives — the + noiseinject arm will show it cheaply. +2. **History feedback: A/B it, default to the paper convention** + (first-iteration output feeds history). The over-stabilization risk is + real but it is exactly what the stability metric + reconvergence protocol + measure — let the numbers pick. +3. **À-trous full-res first.** Clean H1/H5 read; half-res à-trous is a + follow-up arm measured against the lab-09 chain it would ride. +4. **`svgf-noiseinject` first; the path tracer only if demo 14 is + ambiguous.** Agree with the spec's own recommendation and scope-honesty: + the PT is the largest build in the spec and is justified solely by + unbiased-GT + noise knob. One addition: if the PT is built, its + 1024-spp accumulation doubles as the long-deferred *absolute* ground + truth for cornell (the Cycles stand-in) — worth stating in 15-ptref's + goals so the build buys two things. + +Also endorsed explicitly: SVGF-lite as a first-class arm (H1 isolation at +bilateral cost is the likeliest shippable outcome if H0 holds on the SSGI +signal), and the honest H0 framing itself — a measured "no" on our +already-temporally-filtered signal would match our lab-10 experience and +would still leave demo 15 standing as an independent artifact. + +## Sequencing from the consumer side + +- Now: this M0 response goes to the port side; both sides resolve the + addendum sync flag. +- Your M1+M2 unblock our guides lab (demo 10 rig, already live) — that A/B + is the exit criterion and needs nothing new built here. +- Demo 14/15 build starts only after M2 (guides bundle) + M5 + (`MomentPyramid`) exist to consume — per SVGF-SPEC's own thesis, building + it standalone would duplicate the front-end and invalidate the "small + build" premise. +- Bench-side jitter/AA consumption (FSR3-BRIEF Req 2) remains post-pass-3; + unaffected by this review. diff --git a/PAPER-NOTES.md b/PAPER-NOTES.md new file mode 100644 index 0000000..ba1ec0e --- /dev/null +++ b/PAPER-NOTES.md @@ -0,0 +1,163 @@ +# Paper notes — findings worth writing up + +A running tracker of results from this project that are novel or non-obvious +enough to publish (blog series, talk, or a short paper). Each entry: the +claim, where the evidence lives, and what a publication-grade version still +needs. Add to this file whenever a finding clears the bar: *it surprised us, +we measured it, and someone else would hit it too.* + +Consumer-facing prose for several of these already exists in +[PARITY.md](PARITY.md); this file tracks them *as paper material* — evidence +pointers and gaps, not exposition. + +--- + +## 1. Per-texel relative-difference metrics are biased under sub-pixel jitter + +**Claim:** any temporal change detector built on per-texel relative +differences (`1 − min/max`, signed ratios) carries a *coherent* bias on +high-frequency content under sub-pixel jitter: the darker side of any alias +residue always yields the larger ratio, so the mean of per-texel ratios +floors at ~0.07–0.10 on a perfectly still scene — averaging cannot cancel a +one-sided error. Taking the ratio **of block means** (average first, compare +second) is unbiased; the still-scene floor drops to ~0. Additionally, 2×2 +block means are unrescuable at any threshold (thin features still swing +them); 4×4 is the smallest stable scale. + +**Evidence:** `src/shaders/shadingChange.ts` (inline comments record the +measured floors); five GPU tuning iterations in +`bench/docs/NEXT-STEPS.md` (item 4); PARITY.md §3. Measured on Q1/Q4/Q9/Q11. + +**Still needs:** a minimal synthetic reproduction (checkerboard + jitter, no +upscaler) showing the bias analytically and numerically; comparison against +FSR 3.1.5's own signed-difference pyramid on the same input. + +## 2. The price of skipping the scatter — and geometry-derived repairs + +**Claim:** FSR2/3's "reconstruct previous depth" scatter is not just a +performance choice — it is what makes disocclusion testing *self-referencing* +(a visible surface compares against its own same-frame depth). Replacing it +with a cross-frame gather (−22–30% pass cost) silently converts the test +into a cross-frame comparison that inherits sub-texel sampling error, which +on steep depth gradients (grazing-incidence planes) exceeds the +viewport/depth-scaled tolerance by an order of magnitude → per-jitter-phase +disocclusion flicker (measured: 12–14% of mask pixels flipping >32/255 per +frame). Three compensations restore stability at zero cost and with zero +scene-tuned constants: (a) reference per-tap skip semantics (no veto), (b) +jitter-delta-compensated reprojection, (c) a separation tolerance widened by +the 3×3 dilation ring's own depth relief — data the fused pass already holds +in-register. The gather + repairs form retains the scatter's stability at +the gather's cost. + +**Evidence:** commit `b16274a`; `src/shaders/reconstruct.ts` (inline); +PARITY.md §1 ("the price of skipping the scatter"); flicker metrology in the +commit message (example-12 disocclusion quadrant, before/after); +`bench/results/raw/GUIDES-M1/capture-depthfix`. + +**Still needs:** an A/B against the true scatter form (the structural +candidate bundle still implements it) on the same grazing-plane scene — +does the repaired gather match the scatter's mask exactly, or only its +stability? Cross-vendor timing for the cost claim. + +**Amended 2026-07-24:** repair (a) was itself insufficient — skipping +agreement taps leaves lone straddling taps as the only voters, which +re-disoccludes every *still* silhouette per jitter phase (the previous +dilated-depth field's boundary is texel-quantized, so one bilinear tap +routinely reads the old occluder). The complete repair is (a′): every valid +tap votes, agreement = full confidence, and the **best tap wins** (max +aggregation) — "any tap that recognizes the current surface means same +surface." Genuine trails keep reading ~1 (all taps on the old occluder). +Evidence: NEXT-STEPS §5, Q1 phase-locked churn + Q12 disocclusion-black. + +## 6. EMA temporal AA cannot converge if rectification writes back + +**Claim:** In a bounded-memory (EMA) temporal accumulator with per-frame +neighborhood rectification, still-scene convergence is impossible whenever +(i) history age is reduced as a function of clip magnitude, or (ii) the +*clipped* history is what gets stored — because the variance box is built +from ONE jitter phase's taps, and on high-frequency content the converged +supersampled mean falls outside some phases' boxes. (i) pins equilibrium +age low (alpha never shrinks); (ii) re-snaps the stored history to each +phase's box regardless of alpha. Measured signature that separates the two +from benign per-phase shimmer: the **same-jitter-phase frame diff** one +period apart (0.182 with both defects, 0.183 with only (ii), 0.005 with +rectification off). The fix that keeps anti-ghosting: gate box width on +stillness × convergence × absence of disocclusion/shading-change/reactivity +signals — rectify fully the moment any signal fires (FSR2's lock relaxation +generalized from thin features to everywhere). Shipped ×9 relax: 0.018 +phase-locked, motion scenarios unchanged. + +**Evidence:** `bench/docs/NEXT-STEPS.md` §5 (full measurement ladder); +`scripts/measure-convergence.mjs` (the phase-locked metric); +`bench/results/raw/convergence/*`; consumer cross-validation in +GUIDES-HANDOFF-RESPONSE.md report 3 (independent repro + their converging +α=1/N counter-example). Cornell + IGN-dithered Vogel shadow (screen-anchored +dither = adversarially unstable input luminance): 0.024 consecutive. + +**Still needs:** a formal fixed-point argument (under what box statistics is +the converged mean a fixed point of clip∘blend?); comparison against FSR2's +actual still behavior on the same scene; sensitivity of the ghosting +trade-off to the relax factor on a scene with sub-detector lighting drift. + +## 3. Source-faithful pass graphs measured against fused re-derivations + +**Claim:** porting FSR 3.1.5's pass graph faithfully to WebGPU costs ++36% / +43% / +76% GPU compute (filter / structural / SPD-resolver bundles) +over a fused re-derivation with no measurable visual win on torture scenes — +because the source graph's structure pays for generality (intermediate +textures, atomic scatters, SPD mip chains) that a renderer-integrated +upscaler can fuse away. Includes the negative results: which upstream +behaviors *were* worth adopting (conditioned-space RCAS −34%, AMD's +disocclusion tolerance, DeltaPreExposure) and which were not. + +**Evidence:** the whole parity program — `bench/docs/PARITY-DECISIONS.md`, +`bench/docs/PARITY-CANDIDATES.md`, `bench/docs/NEXT-STEPS.md`, PARITY.md. +Candidate bundles remain runnable +(`node scripts/run-benchmark.mjs --smoke --variant --comparison `). + +**Still needs:** cross-device timings (all numbers are one Apple Metal +adapter family); blinded-review grading of the capture pairs (168 pairs +exist under `bench/results/raw/CANDIDATES/`, ungraded). + +## 4. Temporal guides: frame properties vs upscaler properties + +**Claim (systems/architecture):** dilated motion, dilated depth, +disocclusion, and history validity are *frame* properties that every +temporal consumer (upscaler, SSGI temporal pass, SVGF-class denoiser, TAA) +re-derives privately today. Publishing them as a contracted bundle — with +the split the data actually dictates (early = signal-agnostic geometry, +late = beauty-color-dependent) — lets one computation feed all consumers. +The interesting boundary result: lock/instability state *cannot* be an +early product (it derives from final color by construction), so the correct +contract is previous-frame priors, which is also exactly what +history-rejection consumers want. + +**Evidence:** `TEMPORAL-GUIDES-SPEC.md` (+ `GUIDES-SPEC-RESPONSE.md`, the +consumer-side review); implementation on branch `feat-temporal-guides` +(M1/M2 commits); the acceptance A/B (guides-fed SSGI temporal vs private +logic) will live in the consumer repo's demo-10 lab. + +**Still needs:** the consumer lab's measured win (quality and/or ms) — +without it this is a design essay, not a result. + +## 5. Methodology: GPU power-state contamination in headless benchmarking + +**Claim (smaller, methods note):** headless-Chrome WebGPU timing runs are +valid A/B *within* an environment but absolute numbers are hostage to GPU +DVFS: the same workload read a uniform ~3× slower launched from a cold +scratchpad worktree (CPU-bound frame delivery keeps the GPU at low clocks). +Uniform inflation across all passes is the diagnostic signature separating +environment from code. Complements the existing finding (recorded in +PARITY-DECISIONS) that long ABBA sequences show monotonic drift that +forbids fine-margin claims. + +**Evidence:** `bench/results/raw/GUIDES-M1/` (timing vs timing-pre/pre2 vs +timing-post-wt); CLAUDE.md bench caveat; TEMPORAL-GUIDES-SPEC.md M1 notes. + +**Still needs:** nothing much — this is a workshop/appendix note, but worth +a paragraph wherever the timing methodology is described. + +--- + +*Maintenance: link new entries from the relevant commit messages; when an +entry ships in a writeup, note where.* diff --git a/PARITY.md b/PARITY.md new file mode 100644 index 0000000..ba60ca5 --- /dev/null +++ b/PARITY.md @@ -0,0 +1,222 @@ +# @pmndrs/upscaler vs FSR 3.1.5 — design notes + +`@pmndrs/upscaler` derives from AMD FidelityFX FSR — FSR1's EASU/RCAS directly, and an +FSR2/3-style temporal resolver written for WebGPU compute. It is **not** byte-for-byte +FSR 3.1.5, and that is a deliberate, *measured* position, not an unfinished port. This +document explains where we match upstream, where we diverge, what we changed outright, +and the evidence behind each choice. + +Reference: FidelityFX SDK commit `60f4ea8` (FSR Upscaler 3.1.5). The full per-pass audit +lives in [`src/shaders/README.md`](src/shaders/README.md); raw benchmark evidence in +`bench/results/`; the adoption record in +[`bench/docs/NEXT-STEPS.md`](bench/docs/NEXT-STEPS.md). + +## The short version + +We implemented the source-style pipeline — the full FSR 3.1.5 pass graph, including +Lanczos2 reconstruction, deringed bicubic history, atomic depth scatter, motion +divergence, SPD-style mip chains, luma instability, and the coordinated source resolver — +as three cumulative candidate graphs inside this repo, GPU-validated them, and A/B +benchmarked them against the production path on deterministic scenes. + +**The source-style graphs cost +36% to +76% more GPU compute and produced no visible +quality improvement on our test scenarios.** The differences that exist are sub-4% RMSE +spread across edge detail, with no artifacts, ghosting, or convergence failures on either +side. On a library whose priority order is **performance > quality > realism**, that +result decides the question: the simplified production path ships; the source-parity +graphs remain in-repo as benchmark candidates. + +| Comparison (ratio 2, 1920×1080 display, Apple Metal, ABBA timing) | GPU compute | Δ | +| --- | --- | --- | +| production → source filter/reconstruction graph | 0.63 → 0.85 ms | **+36%** | +| … → + structural inputs/reactivity | 0.85 → 0.91 ms | **+6.5%** | +| production → full source SPD/resolver graph | 0.63 → 1.10 ms | **+76%** | + +Every delta was repeatable across four interleaved A/B blocks with noise floors ≤ 2%. +Reproduce with: +`node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant rcas-fsr315-limiter --comparison `. + +The program then fed back: four upstream behaviors *were* worth having, and each was +adopted — but re-derived into cheaper forms rather than transplanted (next section). + +## What we changed — enhancements beyond a port + +These are the places where this implementation deliberately does something *different* +from FSR 3.1.5 and has measurements showing the difference is an improvement on this +platform. Each was validated on GPU with the deterministic capture harness (byte-level +RMSE gates) in addition to timing. + +### 1. Fused single-pass depth reconstruction & disocclusion + +**Upstream:** three stages — "reconstruct previous depth" (an atomic floating-point +scatter into a previous-depth buffer), "dilate depth & motion", and "depth clip" +(disocclusion), with intermediate textures between them. Core WebGPU has no +floating-point storage atomics, so the scatter must be emulated through `u32` +storage-buffer atomics. + +**Ours:** one render-resolution dispatch (`reconstruct.ts`). The key observation: depth +clip only ever reads the current pixel's own dilated depth and motion — both already +in-register after the dilate step — plus last frame's dilated depth, which a gather +(read the previous frame's output) provides without any scatter. We kept AMD's +*math* — the viewport/depth-scaled disocclusion tolerance +(`1.37e-5 · halfViewportWidth · maxDepth`) with per-bilinear-tap confidence voting from +`ffx_fsr2_depth_clip.h` — inside our fused structure. + +**Measured:** the source-style scatter + separate passes cost +30% (prepareInputs) and ++22% (depthClip) in the structural candidate with no visual difference; the fused pass +runs in 0.035 ms at ratio 2 with disocclusion output validated identical in behavior +(thin stable silhouette outlines, quiet still scenes, age resets confined to +disocclusion trails). + +**The price of skipping the scatter, and its fix (2026-07-22):** upstream's scatter +compares each pixel against *same-frame* depth values relocated to their previous +positions, so a continuously-visible surface effectively compares against itself. Our +gather form compares against *last frame's* depth texture, which carries sub-texel +sampling mismatch (bilinear taps, jitter phase). On steep depth gradients — a ground +plane at grazing incidence — neighboring taps differ by tens of view units, far beyond +the ~3-unit tolerance, and the pass shipped with a full-flicker artifact there +(12–14% of disocclusion pixels flipping per jitter phase, found via the temporal-guides +example). Three compensations restore stability at zero measurable cost: reference +per-tap skip semantics (a tap at/behind the current surface contributes nothing and +must not veto the pixel — the original port's running-AND veto was itself a +misreading of upstream), jitter-delta-compensated reprojection, and a separation +tolerance widened by the 3×3 ring's own depth relief (available free from the dilation +loop). All are geometry-derived; no scene-tuned constants were added. Genuine +disocclusion (Q3's fence trails and silhouettes) is unchanged. + +### 2. RCAS in conditioned tonemap space + +**Upstream:** RCAS sharpens exposed linear texels; each of the 5 taps is loaded in the +working color domain. + +**Ours:** the temporal path's history is already stored in invertible-tonemap space +(`c/(1+max(c))`, FSR2's own conditioning). Production RCAS sharpens those bounded +conditioned texels directly and inverts the conditioning + exposure **once on the +result** instead of per tap. Tonemap inversion and exposure division are exactly the +per-tap ALU that made the flag-on form expensive; hoisting them out of the tap loop is +free because RCAS's ratio-based limiter is scale-invariant enough that the sharpening +decision is unchanged in practice. + +**Measured:** −34% on the RCAS pass (0.103 → 0.068 ms), −5.7% total pipeline compute; +captures across Q0/Q1/Q3 plus an HDR-bulb stress scenario show full-frame RMSE +≤ 1.8/255 and HDR ROI maxima ≤ 9/255 — visually identical, no overshoot. The per-tap +form is frozen in the bench registry (`rcas-fsr315-limiter`) so the comparison stays +reproducible. + +### 3. Fused multi-scale shading-change detector + +The most substantial re-derivation, and the one with a genuinely new result. + +**Upstream:** a two-pass design — an SPD (single-pass downsampler) builds a +signed luma-difference mip chain, then the resolve reads multiple mips to detect +shading changes. The per-texel metric is a relative difference +(`1 − min/max`) averaged over each mip footprint. + +**Ours:** one fused half-resolution dispatch (`shadingChange.ts`). An 8×8 workgroup with +a 2×2 render block per thread covers exactly one 16×16 render tile, so every reduction +scale (4×4, 8×8) is workgroup-shared-memory-local — no mip textures, no second pass, no +per-tap re-loads of the 1×1 frame-info texels. + +Two findings from GPU tuning (five documented iterations): + +- **Per-texel relative differences carry a coherent bias under jitter.** On + high-frequency content, sub-pixel jitter leaves alias residue between the current + frame and the reprojected previous frame. The relative-difference metric is + asymmetric — the darker side of any residue always yields the larger ratio — so the + *mean of per-texel ratios* floors at ~0.10 on a completely still scene and no amount + of averaging cancels it. Averaging the *luma first* and taking the ratio of block + means is unbiased: block-mean luma is stable under jitter, so genuine shading changes + move the means while alias flicker does not. +- **The finest (2×2) scale is unrescuable.** 2×2 means of a thin feature still swing + under sub-pixel jitter regardless of the noise floor; a genuinely changing small + feature still moves its containing 4×4 mean. The response therefore uses only the + coarse scales, each gated by a base + contrast-adaptive floor (scaled by the block's + coefficient of variation), with disoccluded texels neutralized. + +**Measured:** 0.044 ms at ratio 2 vs 0.231 ms for the source-style two-pass candidate +(5× cheaper; zero when disabled — the pass isn't dispatched). Quality beats both +alternatives: still-scene response at the old inline heuristic's baseline, *fewer* +false positives than that heuristic under camera motion on high-frequency content +(worst-case 3.6 vs 4.9 on the torture scene), and light steps register as clean +single-frame spikes (137/255) where the old detector produced a weaker response (84) +with a ~20-frame decay tail. + +### 4. Single-workgroup exposure reduction with host-invariant metering + +**Upstream:** auto-exposure reads the coarsest mip of the SPD luminance pyramid. + +**Ours:** no consumer needs the intermediate mips (the shading-change detector above +does its own fused reduction), so exposure is a single 8×8-workgroup log-average +reduction (`luminancePyramid.ts`) — one tiny dispatch instead of a device-wide +pyramid. Two upstream behaviors are preserved exactly: `DeltaPreExposure` history +correction (reprojected history is ratio-corrected across a host pre-exposure change), +and host-invariant metering — auto-exposure divides the host's pre-exposure out of the +scene luma before adapting, so it never chases a step the application already metered. +Skipping that second part reads as a full-screen false shading change for ~2 s after a +host exposure step; we found this on GPU and it is now covered by a dedicated +step+ramp scenario (Q11), with byte-identical output when no pre-exposure input is +supplied. + +## Where we match upstream (adopted parity) + +- **RCAS numeric math.** The production sharpener uses FSR 3.1.5's lower limiter and the + corrected denoise luma/range math, adopted after A/B measurement showed parity was + free (E01). Denoise is opt-in pending evidence on representative noisy content. + (The load domain diverges — see enhancement 2 above.) +- **Host pre-exposure (`DeltaPreExposure`).** The `preExposureTexture` dispatch input is + honored end-to-end with upstream's contract (see enhancement 4). +- **Viewport/depth-scaled disocclusion.** AMD's threshold formulation, kept inside our + fused reconstruction pass (see enhancement 1). +- **Color and exposure domains.** Like upstream, the upscaler applies no tone mapping or + output encoding — input and output are the caller's linear/HDR domain, and internal + conditioning exposure is divided back out before output. An earlier internal ACES/sRGB + transform was removed for source alignment (E03). +- **Core temporal semantics.** Jittered projection (Halton), jitter-free motion vectors, + `prevUV = uv − motion` reprojection, invertible-tonemap accumulation with FSR2's + firefly guard, YCoCg variance clipping, disocclusion-driven history rejection, + luminance-stability locks, auto-exposure conditioning, reactive masks (explicit and + auto-generated from opaque-vs-final diff, FSR2-style) — the algorithmic lineage is + FSR's throughout. +- **EASU (spatial path).** The 12-tap edge analysis, anisotropic Lanczos kernel, tap + placement, and deringing follow `ffx_fsr1.h`; only language-level details differ + (native WGSL division/`inverseSqrt` instead of AMD's approximation helpers). + +## Where we diverge, and why + +**1. Platform constraints (WebGPU is not Vulkan/DX12).** Core WebGPU has no +floating-point storage-texture atomics (source depth scatter), no device-wide atomic +counter for single-pass downsampling (source SPD), no guaranteed f16 arithmetic, and no +swapchain pacing control (which rules out FSR3 frame generation entirely). The candidate +graphs prove these can be *emulated* — storage-buffer atomics, direct mip re-reads — but +the emulations are part of why the source graphs measure slower here. + +**2. Measured cost without measured benefit.** The compact accumulate pass (bilinear-free +Lanczos2 upsample + Catmull-Rom history, no deringed bicubic) survived because the +source alternative cost +47% on accumulate and the deterministic quality scenarios +(static convergence, camera motion, object-motion disocclusion) could not distinguish +them visually. A divergence is kept only while that remains true — the candidates stay +in-repo precisely so this can be re-tested as scenes, devices, or the library change. + +**3. Scope decisions.** Frame generation is out of scope (browser swapchain limits). +MSAA input is rejected by design — FSR's temporal path *is* the anti-aliaser. The +Transparency & Composition mask is accepted as a dispatch input for API compatibility +but currently maps to the reactive path; upstream's distinct softer T&C channel is +implemented in the structural candidate and will only be promoted with evidence that +the reactive path is insufficient for real content. + +## Honest limits of the evidence + +Current measurements are one adapter family (Apple Metal), one upscale ratio class, and +synthetic torture scenes over short deterministic sequences. The source graphs' +theoretical advantages target harder content — exposure ramps, transparency-heavy +scenes, noisy GI inputs, extreme motion — that the decisive runs did not exercise. The +benchmark harness (`npm run bench`, `scripts/run-benchmark.mjs`) exists so any of these +claims can be re-tested; a repeatable ≥5% result is treated as actionable, <3% as noise. + +## Status + +The parity program is concluded. Every adoption-worthy behavior it identified landed on +2026-07-21 — the four items above. Nothing from the program remains open; what remains +deferred (perf-only micro-optimizations, the distinct T&C channel, a fused GI/denoise +temporal path) is listed with rationale in the project README and +`bench/docs/NEXT-STEPS.md`. diff --git a/README.md b/README.md index f7d0742..412fcc6 100644 --- a/README.md +++ b/README.md @@ -26,10 +26,9 @@ The recommended integration is the **TSL node** — drop it in as the output of import * as THREE from 'three/webgpu'; import { upscaleScene, QualityMode } from '@pmndrs/upscaler'; -// The upscaler outputs display-ready sRGB, so make the post output transform -// identity: boot the renderer with NoToneMapping + LinearSRGBColorSpace. -renderer.toneMapping = THREE.NoToneMapping; -renderer.outputColorSpace = THREE.LinearSRGBColorSpace; +// The upscaler remains linear/HDR. Choose presentation independently. +renderer.toneMapping = THREE.ACESFilmicToneMapping; +renderer.outputColorSpace = THREE.SRGBColorSpace; const post = new THREE.PostProcessing(renderer); post.outputNode = upscaleScene(scene, camera, { quality: QualityMode.Quality }); @@ -101,9 +100,8 @@ upscaler.dispatch( { color: rt.textures[0], depth: rt.depthTexture, velocity: rt.textures[1], deltaTime }, camera, ); -// upscaler.outputTexture is a display-resolution three texture — present it on -// a fullscreen quad (already tonemapped + sRGB, so keep renderer tone mapping / -// output encoding off for that draw). +// upscaler.outputTexture is a display-resolution linear/HDR texture. Present it +// through the renderer's normal output transform or continue post-processing it. ``` Runtime knobs live on `upscaler.settings` (`sharpness`, `maxAccumulation`, `exposure`, `debugView`) and take effect next frame. `upscaler.resetHistory()` drops accumulation on camera cuts. @@ -137,24 +135,37 @@ Each frame the projection is offset by a sub-pixel **jitter** (Halton(2,3) seque The **spatial path** (`path: 'spatial'`) is a faithful FSR1 port: EASU's edge-direction-rotated, anisotropically-stretched 12-tap Lanczos kernel, then RCAS. No history, no motion vectors — also the fallback story for content that can't produce velocity. -Full per-pass details and deviations from the FidelityFX reference: [`src/shaders/README.md`](./src/shaders/README.md). +Full per-pass details and deviations from the FidelityFX reference: [`src/shaders/README.md`](./src/shaders/README.md). For the measured story of how this implementation relates to real FSR 3.1.5 — what matches, what was re-derived into cheaper forms, and the benchmark evidence — see [`PARITY.md`](./PARITY.md). ### Integration approach Three doesn't expose its WebGPU internals publicly, so the upscaler grabs `renderer.backend.device` and the `GPUTexture` handles behind render-target attachments (`internal/threeWebGPU.ts` documents exactly which internals we touch and throws loudly if a three upgrade changes them). Compute passes are encoded on our own `GPUCommandEncoder` and submitted between three's scene render and the presentation draw — queue order guarantees correctness with zero synchronization code. The final image lands in a three `StorageTexture` so presenting it is ordinary three code. -## Phases +### Temporal guides -| Phase | Scope | Status | -| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| **0** | Package scaffold, raw-WebGPU pass infra on three's device, bench app, unit tests | ✅ | -| **1** | Spatial baseline: faithful FSR1 EASU + RCAS ports | ✅ | -| **2** | Temporal pipeline: Halton jitter, unjittered velocity, dilation, depth-clip disocclusion, Lanczos2 accumulate w/ YCoCg variance clipping, per-pass GPU timings | ✅ | -| **3** | Fidelity: luminance-stability **locks**, luminance-average auto-exposure (+ external exposure input), shading-change detection, reactive mask input for transparents/particles | ✅ | -| **4** | API & ecosystem: reactive-mask authoring helpers, imperative `UpscalePass` + composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`), RCAS denoise toggle (MSAA input intentionally excluded — FSR's temporal path _is_ the anti-aliaser) | ✅ | -| **5** | Performance: merged dilate+clip pass ✅; remaining — `textureGather` tap packing, f16 (`shader-f16`), bind-group caching, a true SPD luminance mip chain (steadier shading-change), half-res luma analysis | 🚧 | +Dilated motion, dilated depth, and disocclusion are **frame properties, not upscaler properties** — every temporal effect upstream (SSGI/SSR temporal reprojection, denoisers, any TAA-class pass) needs them and usually re-derives worse versions privately. The upscaler publishes its internal working set as the **temporal guides** bundle (`upscaler.guides`, ordinary three textures), and the frame can be driven split so the geometry guides exist *before* the final color does: -Frame generation (the other half of "FSR3") needs swapchain-level frame pacing that browsers don't expose; if we ever want it, the realistic shape is interpolating between presented frames ourselves — out of scope here. +```ts +upscaler.dispatchGuides({ depth, velocity }, camera); // right after the G-buffer +// … effects sample upscaler.guides.dilatedMotion / .disocclusion / .dilatedDepth … +upscaler.dispatchUpscale({ color, deltaTime }, camera); // finish the frame +``` + +An app that never upscales can run `path: 'guides'` for the geometry products alone. Reactivity is bidirectional: an explicit mask **merges** (per-pixel `max`) with the auto-generated one, and effects can write into `guides.reactive` mid-frame. The standalone `MomentsPass` rounds out the bundle — per-pixel `(E[x], E[x²])` of a configurable scalar (linear luma or YCoCg Y) over *any* float texture plus one coarse level, the statistics half an SVGF-class denoiser needs, with no beauty/exposure assumption baked in. + +The same surface exists declaratively for `THREE.PostProcessing` graphs: `temporalGuides(depth, velocity, camera)` publishes the bundle as texture nodes (`guides.getTextureNode('disocclusion')`), and `upscale(color, depth, velocity, camera, { guides })` shares one computation — the guides dispatch runs as soon as the G-buffer has rendered, in-graph effects consume the products, and the upscale finishes the split frame. + +Per-product contracts (format, space, resolution, latency) are documented on the `TemporalGuides` type and in [`TEMPORAL-GUIDES-SPEC.md`](./TEMPORAL-GUIDES-SPEC.md); `examples/12-temporal-guides` (raw) and `examples/13-guides-node` (TSL) are the live references. The contract is **accepted**: an external SSGI/SVGF consumer swapped its private temporal front-end for this bundle and measured bit-identical still-camera stability (spec M6). The TSL surface stays marked `@experimental` — the same contract, but its graph wiring has only our own verification so far. + +## Status + +The pipeline is **feature-complete and GPU-verified**: spatial (FSR1) and temporal paths, luminance-stability locks, auto-exposure (+ external and host pre-exposure inputs), multi-scale shading-change detection, reactive masks (explicit + auto-generated), RCAS with opt-in denoise, imperative `UpscalePass`, and the composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`). The **temporal guides** surface (above) has passed its cross-repo acceptance; only its TSL node stays experimental. A benchmarking program A/B-compared this implementation against source-style FSR 3.1.5 pass graphs on-GPU; the adopted results and remaining divergences — with measurements — are written up in [`PARITY.md`](./PARITY.md). + +Deliberately **not** planned: + +- **Frame generation** (the other half of "FSR3") — needs swapchain-level frame pacing browsers don't expose. +- **MSAA input** — FSR's temporal path _is_ the anti-aliaser; a multisampled input is redundant and can't bind to the compute passes. +- **Perf-only micro-optimizations** (`textureGather` tap packing, f16 arithmetic, bind-group caching) — each adds correctness risk to a core path with no image-quality gain; deferred until performance is an actual bottleneck on real content. ## Package layout diff --git a/TEMPORAL-GUIDES-SPEC.md b/TEMPORAL-GUIDES-SPEC.md new file mode 100644 index 0000000..1cceedd --- /dev/null +++ b/TEMPORAL-GUIDES-SPEC.md @@ -0,0 +1,346 @@ +# Temporal Guides — opening the upscaler's internals (spec) + +Status: **program complete** (M6 PASS, 2026-07-24). M1–M5 landed + +GPU-verified (M4 followed post-handoff); M6, the consumer's cross-repo A/B, +passed — their SSGI temporal stack fed by this bundle measured bit-identical +still-camera stability against their private front-end, so the +`@experimental` tag came off the raw guides surface and `MomentsPass` +(consumer reports 1–2 in +[GUIDES-HANDOFF-RESPONSE.md](GUIDES-HANDOFF-RESPONSE.md)). The TSL surface +(M4) keeps the tag: same contract, but no external consumer has wired the +node yet. Integration entry point: +[GUIDES-HANDOFF.md](GUIDES-HANDOFF.md). Contract frozen at M0 +(consumer review in [GUIDES-SPEC-RESPONSE.md](GUIDES-SPEC-RESPONSE.md), +resolution in §10). +Request: [bench/docs/FSR3-BRIEF.md](bench/docs/FSR3-BRIEF.md) — the consuming +pipeline (SSGI temporal pass, an SVGF-style denoiser, any TAA-class effect) +wants the upscaler's early data products as first-class outputs instead of +re-deriving worse versions privately. This spec maps that request onto what +this codebase actually is, states where we deviate from the brief and why, and +lays out the delivery plan. + +Non-negotiable constraint: **nothing on `feat-match-fsr3` regresses.** Every +change here is additive; the existing `dispatch()` path must stay +byte-identical and perf-identical (bench A/B gated) when no guide is consumed. + +--- + +## 1. The good news: the products already exist + +The brief assumes the port is a monolith that must be split. It isn't — the +pipeline is already discrete compute passes over named internal textures, and +**every texture the brief's "temporal guides" bundle names is already +allocated and written every temporal frame** (`Upscaler._allocateTextures`): + +| Brief asks for | We already have | Where it's produced | +|---|---|---| +| `dilatedVelocity` | `_dilatedMotion` (rgba16float, **UV** delta in .xy) | `reconstruct.ts` (fused dilate + depth-clip) | +| `dilatedDepth` | `_dilatedDepth[cur]` (r32float, linear view depth) | `reconstruct.ts` | +| `prevDepth` | `_dilatedDepth[prev]` — the ping-pong's other half | free (kept for depth-clip) | +| `disocclusion` (graded 0..1) | `_masks.r` (rgba8unorm) — AMD's confidence-voted grade | `reconstruct.ts` | +| `reactive` | `_reactiveGenerated` / caller's mask | `generateReactive.ts` or app | +| `lockStatus` | `_locks[cur]` (rgba16float: r = lifetime, g = locked luma, b = shading) | `accumulate.ts` | +| `historyLength` | history `.a` (accumulation age, display res) | `accumulate.ts` | +| luminance statistics | *(does not exist — see §5)* | — | + +So the deliverable is **contracts and plumbing, not new algorithms**: make +these textures reachable from outside (raw-WebGPU and TSL), make the +geometry-only subset runnable without the upscale, and document each +product's space, resolution, and latency so a consumer can't mis-apply it. + +## 2. Design principle: publish products, not pass boundaries + +The brief prescribes a 4-pass decomposition ("pass boundaries, not a +monolith"). We decline the *dispatch* shape and keep the *data* shape: + +- Our fusions are measured wins from the parity program (see `PARITY.md`): + dilate + depth-clip fused is 0.035 ms where the source's split form costs + ~3×; the shading detector is one fused dispatch at 0.044 ms vs the + two-pass candidate's 0.231 ms. Splitting dispatches to mirror the brief's + diagram would regress performance for zero consumer benefit — a consumer + binds textures, not passes. +- What the brief *actually needs* from the split is (a) the products, and + (b) the ability to produce the early ones **without running the upscale**. + Both are satisfiable with the fusions intact. + +The contract is therefore: **named textures with documented format, space, +resolution, and production stage.** Internally we stay free to fuse, reorder, +or re-derive as long as the contracts hold. + +## 3. The two-stage frame contract + +The real seam in the pipeline is not the brief's pass 1/2/3/4 — it's the line +the brief's own addendum draws ("signal & space"): **geometry products are +signal-agnostic and need only depth + velocity; everything else needs the +final beauty color.** + +``` +frame start + │ (previousDepth, previous lockStatus, previous history already valid — + │ last frame's products are readable before anything runs this frame) + ├─ scene G-buffer/depth/velocity available + ├─ ► dispatchGuides() EARLY stage — geometry guides + │ reconstruct.ts only: dilatedMotion, dilatedDepth, disocclusion + ├─ effects run (SSGI march, SSGI temporal, denoiser…) consuming guides + ├─ final beauty color available + ├─ ► dispatchUpscale() LATE stage — luma-dependent products + upscale + │ exposure → shadingChange → generateReactive → accumulate → RCAS + └─ present +``` + +- **`dispatchGuides(inputs)`** takes `{ depth, velocity }` (+ optional + `reset`/`deltaTime`) and encodes *only* the reconstruct pass. Its three + outputs plus `previousDepth` are then valid for every downstream effect — + this is the brief's "pass 1 runnable without 3–4", at 0.035 ms. +- **`dispatchUpscale(inputs)`** encodes the rest. The existing monolithic + `dispatch()` becomes exactly `dispatchGuides(); dispatchUpscale();` on one + command encoder / one submit — the split must be observable only when the + caller opts into calling the halves. +- **`path: 'guides'`** in `UpscalerConfig`: allocates only the early working + set and never runs the late stage — the "app that never upscales" case. + `outputTexture` throws on this path (there is no output). +- Queue ordering on three's shared device gives correctness for free, same + as today: effects submitted after `dispatchGuides` see its writes. + +**Late products have one-frame latency for render-stage consumers.** Locks, +history age, and shading change are computed *after* the effects that would +consume them have already run. This is not a defect to engineer away — FSR3's +own locks derive from the final color's luminance, so no implementation can +provide same-frame locks to a pass that runs before final color exists. The +brief's SVGF use ("anti-ghost / history-rejection prior") is a prior, and +frame N−1's lock state is the correct prior for frame N. The contract +publishes them as **previous-frame products** with that label. + +## 4. The published bundle + +Access: `upscaler.guides` — a `TemporalGuides` object of three `Texture`s +(consumable as TSL `texture()` nodes and by raw bind groups alike). Getters +resolve ping-pongs to the correct half for "current" vs "previous". + +| product | resolution | format | space / convention | stage | notes | +|---|---|---|---|---|---| +| `dilatedMotion` | render | rgba16float (.xy) | **UV delta**, `prevUV = uv − motion`; y-flip (0.5, −0.5) already applied | early | see deviation D1 | +| `dilatedDepth` | render | r32float | linear view depth (eye-Z), reversed-depth already resolved | early | | +| `previousDepth` | render | r32float | linear view depth, frame N−1 | frame start | retires the consumer's ≥3 private depth copies | +| `disocclusion` | render | rgba8unorm (.r) | graded 0 (stable) → 1 (fresh), AMD confidence-voted | early | | +| `reactive` | render | rgba8unorm (.r) | 0..1 reactivity, post-merge (§6) | late | valid same frame *after* `dispatchUpscale` | +| `shadingChange` | ceil(render/2) | r32float | 0..1 response, block-mean metric | late | | +| `exposure` | 1×1 | rgba16float | r = conditioning pre-exposure, g = avg luma (**exposed beauty luma — not for GI**), b = host pre-exposure | late | space-labeled per the brief's one rule | +| `lockStatus` | **display** | rgba16float | r = lock lifetime, g = locked luma (conditioned tonemap space), b = shading age | late, frame N−1 | consumer downsamples or samples at display UV | +| `historyAge` | **display** | (history `.a`) | accumulated frame count, 0..maxAccumulation | late, frame N−1 | see deviation D3 | + +Consumers MUST treat every field not listed here (e.g. `dilatedMotion.zw`, +`masks.gba`) as reserved — we keep the right to pack new signals into spare +channels, as we already did with locks `.b`. + +### Implementation note: three-visible textures + +Guide textures are currently raw `GPUTexture`s. To publish them we flip the +allocation to the same mechanism as `outputTexture`: allocate as three +`StorageTexture`s (with `generateMipmaps = false`), `renderer.initTexture()`, +and keep using the raw handle internally via `getGPUTexture()`. Zero shader +changes; the pipeline can't tell the difference. Ping-ponged products +(`dilatedDepth`, `locks`, exposure, history) allocate both halves this way +and the `TemporalGuides` getters return the right half per frame; the TSL +guide nodes update their texture reference in `updateBefore` (the same +per-frame re-point pattern `UpscalerNode` already uses internally). If a +consumer needs *stable* texture identity across frames (some graph setups +cache hard), the fallback is an opt-in copy into a stable target — spec'd but +not built until the lab shows it's needed. + +## 5. The statistics primitive (SVGF addendum B) — new code, not a refactor + +The brief assumes FSR3-style per-pixel luminance moments exist to refactor. +**They don't, here.** Our exposure pass is a single 1×1 log-average (no +pyramid, no per-pixel moments — `luminancePyramid.ts` deliberately computes +only what's consumed), and our variance clip computes its 3×3 YCoCg moments +inline in `accumulate.ts` in conditioned tonemap space — exactly the space +the brief forbids for GI variance. + +So `MomentPyramid` is a **new, standalone, signal-agnostic pass**: + +- in: any texture + `{ space: 'linear' | 'ycocg', channels }` — hardcodes no + exposure/tonemap/albedo assumption, per the brief's contract. +- out: `moments` rg16float `(E[x], E[x²])` per pixel, plus one coarse level + (mip-2-equivalent, 4× reduction) — the consumer's short-history fallback + reads exactly one coarse neighborhood and nothing deeper (§10, answer 4). + No full chain. +- Lives in its own files (`shaders/moments.ts` + a small `MomentsPass` + driver / `moments()` node), exported `@experimental`. It touches nothing + in the core pipeline — zero regression surface. +- Our own pipeline does **not** adopt it initially (accumulate's inline 3×3 + is fused and cheap; swapping it for a consumed pyramid is a perf/quality + trade to measure separately, if ever). One primitive, external consumers + first. +- Verification split: the CPU-reference `Var = E[x²]−E[x]²` check the brief + asks for cannot run in our GPU-free CI; we ship the structural shader test + + a bench-side scripted GPU check, and the consumer's SVGF lab is the + acceptance test (their own stated criterion). + +## 6. Reactive becomes bidirectional (merge, not overwrite) + +Today: explicit `reactive` input wins, else auto-generate from +`reactiveOpaqueColor`, else zero dummy — mutually exclusive. New contract per +the brief: + +- `generateReactive.ts` gains an optional incoming-mask binding; response = + `max(generated, incoming)`. Supplying both inputs now composes instead of + the explicit mask silencing the generator. +- The published `guides.reactive` texture is storage-writable: an effect + (SVGF flagging high-variance GI) can write into it between `dispatchGuides` + and `dispatchUpscale`, and the late stage consumes the merged result. +- This is the only production-shader change in the whole program + (`generateReactive` hash pin updates; GPU re-verify on example 05, which is + the reactive acceptance demo). + +## 7. Deviations from the brief (flagged for the consumer's review) + +- **D1 — motion is a UV delta, not an NDC delta.** We publish the + already-converted form (`velocity.xy · (0.5, −0.5)`, `prevUV = uv − m`). + Rationale: the brief's own trap list says the y-mirror seam "cost us the + demo 03 speckle hunt" — publishing the convention-free, directly-usable + form deletes that seam for every consumer. If the raw NDC delta is truly + needed, the consumer already owns the velocity MRT it came from. +- **D2 — locks/reactivity are late-stage, not "early-mid".** See §3; locks + need final color by construction. Published as frame N−1 priors. +- **D3 — `historyLength` is display-res history `.a`, not a render-res + r16float.** The brief itself allows the consumer to keep producing its own + (demo 03 already does). We publish what exists; a dedicated render-res + count texture is added only if the guides lab shows sampling display-res + age is insufficient. +- **D4 — formats are ours, not the brief's suggestions** (rgba16float + motion vs rg16float, rgba8unorm disocclusion vs r16float). The suggested + formats save memory, not correctness; we won't fork allocations for it. + Revisit under real memory pressure. +- **D5 — no dispatch-level 4-pass split** (§2). Contract is textures. +- **D6 — environment notes in the brief that don't apply here:** we bridge + three internals via `internal/threeWebGPU.ts` (not the bench harness path), + RCAS is already tonemap-agnostic (the ACES-in-RCAS bug is fixed and E03 in + `bench/docs/PARITY-DECISIONS.md` is its record), and `RenderPipeline` + naming is already handled in the node docs. + +## 8. Delivery plan + +Branch `feat-temporal-guides` off `feat-match-fsr3`; each milestone lands +green (165 unit tests / typecheck / lint), GPU-verified per the CLAUDE.md +headless-CDP protocol, and bench-A/B'd where perf could move. Merge back only +when the consumer lab has accepted. + +- **M0 — contract review (this document).** Send to the SSGI/SVGF side; + resolve D1–D5 and the open questions (§9) before code. +- **M1 — internal seam. DONE (`3603a14`, 2026-07-21).** Split + `_encodeTemporal` into `_encodeGuides` / `_encodeLate` private halves + composed on one encoder; no public API change. Gates met (evidence: + `bench/results/raw/GUIDES-M1/`): worktree-vs-worktree smoke A/B vs + `005de6d` — compute-sum −2.7%, within the <3% noise policy; Q0 captures + (3 frames × final + motion-vectors/disocclusion/accumulation-age) + byte-identical except frame-119 `final`, where the *pre-M1 run's own two + arms already disagree (harness nondeterminism at deep accumulation, + 0.098% px, max 23/255) while the M1 run's arms agree. Bench caveat + learned: runs launched from a scratchpad worktree read ~3× slower + absolute (GPU stays in a low power state; likely cold-vite frame + delivery) — uniform across passes, so A/B *within* that environment is + valid, but never compare worktree absolutes against repo-run records. +- **M2 — publish the bundle. DONE (2026-07-22).** Three-visible allocation + flip (`_createSharedTexture`: three `StorageTexture` + raw handle, one + allocation), the `guides` accessor, `dispatchGuides`/`dispatchUpscale` + public split, `path: 'guides'`, `@experimental` tags. New example + `12-temporal-guides` (11 was taken) renders final + three geometry guides + from the split frame — the GPU acceptance harness. Gates met: Q0 captures + byte-identical vs M1 (24/24, incl. frame 119 and all debug views); example + 12 headless-verified (no validation errors, guides sampled as TSL nodes, + split-frame timings read out); guides-only path CDP-driven under a + validation error scope (clean across both ping-pong halves, API guards + throw, late products null). Evidence: `bench/results/raw/GUIDES-M1/capture-m2`. +- **M3 — reactive merge (§6). DONE (`64a4e65`, 2026-07-22).** The generator + max-merges an incoming mask (new binding, inert dummy when absent); the + aliasing hazard (passing `guides.reactive` back while the generator writes + it) throws. Gates met: fingerprint updated, 165 tests green, example 05 + GPU-verified across manual/auto/off modes with no validation errors. The + bench merged-mask capture scenario is deferred to M6 (the consumer lab + exercises the merge for real). +- **M4 — TSL surface. DONE (2026-07-22, post-handoff).** + `temporalGuides(depth, velocity, camera)` (`TemporalGuidesNode`) publishes + the bundle as texture nodes via `getTextureNode(name)` — stable node + identity, ping-ponged products re-pointed per frame. Two modes decided by + wiring: **standalone** (node owns a guides-only upscaler sized to its + depth input; late products null + a one-shot guidance warning) and + **linked** (`upscale(..., { guides })` adopts the node's upscaler and the + frame runs split in-graph: guides dispatch → effects render → late + upscale — the node falls back to the monolithic dispatch on frames where + the early stage couldn't run, e.g. inputs not yet GPU-backed or a + mid-frame reconfigure). Enabled by a `guidesPending` getter on + `Upscaler`. Gates met: examples 07/09 GPU-verified unchanged; new + `13-guides-node` demo consumes `disocclusion` in a toy effect (orange + trailing-silhouette tint, pre-upscale) — dispatch-spy probe shows the + pure split path steady-state (120 guides + 120 late, 0 monolithic over + 1 s) on one shared upscaler; standalone mode CDP-driven under a + validation error scope (clean, early products live, late products null, + warning fires). Originally deferred per §10 answer 5; built post-handoff + so the surface is ready when composite-side consumption lands. Was: gate + design sketched at handoff. +- **M5 — `MomentPyramid` (§5). DONE (`52c3b12`, 2026-07-22).** Shipped as + `MomentsPass` + `shaders/moments.ts`, `@experimental`, zero coupling to + the upscaling pipeline. Deviations recorded: outputs are rgba16float + (`.rg` used — rg16float is not a core WebGPU storage format); the coarse + level is the single 4×-reduction the consumer asked for (§10 answer 4). + Gates met: structural tests + fingerprint (172 tests green); scripted GPU + check vs CPU reference on a seeded DataTexture in BOTH linear and ycocg + spaces — validation-clean, max rel. error <0.1% (f16 tol 1%), coarse + variance non-negative. +- **M6 — cross-repo acceptance. PASS (consumer report 2, 2026-07-24).** Their + demo 17 ran the identical SSGI temporal stack fed by their private guides + pass vs this bundle (guides-only path; the split shaders differ by one + uniform flag for the D1 UV-delta convention). Recorded: **still-camera + stability bit-identical** (both arms 1.3984197255291004 — at convergence + disocc≈0/vel≈0 make the blend independent of guide source, the strongest + parity statement available); teleport reconvergence 1.275 s vs 1.288 s + (inside one 500 ms sampling interval); across-arm meanAbsDiff 0.84 below + the 1.40 within-arm temporal noise. Their verdict: drop-in replacement for + their private temporal front-end. `MomentsPass` was separately + field-verified in their demo-14 SVGF (variance identity held on GPU + readback; variance-guided à-trous denoises σ=0.3 + fireflies a bilateral + can't). Their 15-ptref (unbiased PT ground truth) stays deferred on their + side — no asks here. + +Sequencing note: M1+M2 unblocked the consumer's guides lab; M3–M5 proceeded +in parallel with their integration. The guides API shipped marked +`@experimental` until M6 passed, so `main` never carried a frozen contract we +hadn't seen consumed — the tag came off the raw surface + `MomentsPass` with +M6's PASS, and stays only on the M4 TSL node until someone wires it. + +## 9. Open questions for the consumer side + +1. Is the UV-delta motion convention (D1) acceptable, or is raw NDC needed? +2. Is frame N−1 lock/age latency (D2) sufficient for the SVGF prior? +3. Does the SSGI temporal pass need `historyLength` at render res on day + one, or does display-res age sampling suffice (D3)? +4. Moment pyramid mip chain: how many levels does the short-history spatial + fallback actually read? (We'd rather allocate 3 than "a full chain".) +5. Consumption mechanism: TSL `texture()` nodes, raw bind groups, or both? + (Both are spec'd; if only one is consumed we defer the other's example.) + +## 10. M0 resolution (2026-07-21) + +Consumer review: [GUIDES-SPEC-RESPONSE.md](GUIDES-SPEC-RESPONSE.md). Every +deviation D1–D6 accepted; the brief's pass-boundary language and suggested +formats are superseded by D5/D1/D4 (reconciled on the brief side). §9 +answers, now binding: + +1. **UV-delta motion accepted and preferred.** The consumer harness will + align its own guides pass to *our* convention before the swap (their + work, scheduled with the swap) so the bundle is drop-in. +2. **Frame N−1 locks/age sufficient** — it is the semantics their shipping + consumers already validated, not a compromise. +3. **Display-res age sampling accepted for day one**, with one recorded + risk their guides lab measures: filtered age may dilate across + silhouettes → transient over-effort at edges (fails safe). Their + render-res counter remains as fallback; we build nothing extra unless + the lab rejects display-res sampling. +4. **Moments: mip 0 + one coarse level (mip-2-equivalent) only.** Nothing + reads deeper. +5. **Raw bind groups first; TSL second.** Per-frame texture re-pointing is + fine on their side; the stable-identity copy (§4) stays spec'd-only. + +Standing acceptance criteria unchanged: their live demo-10 guides rig is +M6's exit A/B; their SVGF/moments lab builds only after our M2 + M5 exist. diff --git a/bench/README.md b/bench/README.md index 2e43908..d351f33 100644 --- a/bench/README.md +++ b/bench/README.md @@ -19,7 +19,7 @@ Suggested tour: 1. Start on **FSR3 temporal / Performance (2×)** with auto-orbit on — the scene renders 1/4 the pixels of native. Pause the orbit and watch thin lines converge. 2. Flip to **Bilinear** at the same quality to see what the temporal pass is reconstructing. -3. Try **Ultra Performance (3×)** — 1/9 the pixels; edges hold up, fine texture detail softens (this is where FSR3's locks, Phase 3, would help). +3. Try **Ultra Performance (3×)** — 1/9 the pixels; watch the luminance-stability locks hold thin features that would otherwise dim (**Debug ▸ Locks** shows where they form). 4. Set quality to **Native AA (1.0×)** on the temporal path — that's pure TAA mode, the fair comparison against `Native`'s shimmer. 5. Open **Debug ▸ Motion vectors / Disocclusion** while the spheres orbit to sanity-check the inputs. diff --git a/bench/docs/DENOISING-DIRECTION.md b/bench/docs/DENOISING-DIRECTION.md new file mode 100644 index 0000000..590b8b9 --- /dev/null +++ b/bench/docs/DENOISING-DIRECTION.md @@ -0,0 +1,526 @@ +# Denoising direction + +## Status + +This document captures the current denoising discussion and a possible direction +for future experiments. It is an outline, not an approved implementation plan. + +The immediate recommendation is: + +- keep the upscaler as the only final-image temporal resolver; +- keep noisy SSGI and SSR signals separate long enough to denoise them with + effect-specific information; +- make any effect-level temporal reprojection aware of the exact projection + jitter used by the upscaler; +- apply spatial denoising primarily where temporal history is weak; +- consider a learned filter only after a conventional implementation provides a + measured baseline. + + +## The terms that are easy to conflate + +### Projection jitter + +Projection jitter moves the camera projection by a fraction of a pixel. It lets +the final temporal resolver collect different subpixel samples over multiple +frames. + +Exactly one component should own projection jitter for a render. In this +repository, that should normally be the upscaler. + +### Effect sampling noise + +SSGI, GTAO, stochastic SSR, and denoisers can rotate or shift their own sampling +patterns each frame. This changes ray or filter-tap locations without moving the +camera. + +These patterns do not need to use the upscaler's Halton sequence. They only need +predictable frame identity, reset behavior, and rendered cadence. + +### Spatial denoising + +A spatial denoiser combines neighboring pixels from the current frame. Depth, +normals, roughness, albedo, and ray metadata can prevent filtering across +unrelated surfaces. + +Spatial denoising helps immediately, including on the first frame, but excessive +filtering blurs detail and can create halos. + +### Temporal denoising + +A temporal denoiser reprojects an effect's previous result into the current +frame and accumulates it over time. + +It needs motion, depth, history rejection, and reset handling. In a jittered +pipeline it must also account for the current and previous projection offsets. + +### Final temporal AA/upscaling + +TRAA, TAAU, and this library's temporal path accumulate the final image. They +reduce aliasing and can average moderate effect noise, but they do not have the +effect-specific inputs needed to be ideal GI or reflection denoisers. + + +## What the repository currently uses + +### Examples 06 and 09 + +`examples/06-screenspace-gi` and `examples/09-kitchen-sink` use three's ordinary +`DenoiseNode` for SSGI and SSR. + +That node is spatial-only: + +- it filters the current frame; +- it uses depth and normals for edge stopping; +- it does not reproject history; +- it does not consume velocity; +- it does not jitter the camera. + +The resulting signal is composited into scene color and then accumulated by the +upscaler. + +### Example 10 + +`examples/10-ssgi-denoise` is explicitly experimental and compares three SSGI +paths: + +#### `builtin` + +Raw, temporally varying SSGI is composited into scene color. The upscaler owns +all temporal accumulation. + +This is the default path. + +#### `spatial` + +`RecurrentDenoiseNode` is used with: + +`accumulate: false` + +There is no `TemporalReprojectNode`, so the class runs as a spatial denoiser +despite its name. The upscaler remains the only temporal stage. + +#### `recurrent` + +The full chain is: + +```text +Raw SSGI + -> TemporalReprojectNode + -> RecurrentDenoiseNode { accumulate: true } + -> denoised output fed back as effect history + -> scene composite + -> temporal upscaler +``` + +This is genuinely temporal. + +The experiment produced the worst result under the upscaler's projection +jitter. Its velocity-only reprojection did not account for the changing +subpixel projection offset, so effect history could be misaligned before the +final temporal resolve. + +The experiment also exposes a separate integration risk: `UpscalerNode` and +`TemporalReprojectNode` assign the same scalar render-pipeline hooks and touch +shared velocity state. + +The current result does not prove that effect-level temporal denoising is +inherently incompatible with temporal upscaling. It shows that a second resolver +must share the projection-jitter and lifecycle contract. + + +## How SSGI temporal filtering relates to the upscaler + +`SSGINode.useTemporalFiltering` does not maintain SSGI history. It changes the +SSGI ray pattern over time so a downstream temporal resolver can average the +samples. + +The flow is: + +```text +Jittered scene render + -> SSGI produces a different noisy estimate + -> SSGI is composited into scene color + -> final temporal resolver reprojects previous scene color + -> noisy GI contribution is averaged with the rest of the pixel +``` + +SSGI's sample generation does not require velocity. The downstream temporal +resolver uses velocity to align the composited result. + +This can reduce the amount of spatial filtering required, but it does not +guarantee that spatial denoising becomes unnecessary. Newly revealed areas, +moving surfaces, rejected history, and the first frame still have little or no +usable temporal history. + + +## What `RecurrentDenoiseNode` contributes + +The current three.js implementation contains useful effect-specific ideas: + +- separate diffuse and specular modes; +- depth-, normal-, roughness-, and albedo-aware edge stopping; +- SSR ray-length handling; +- spatial radius based on history confidence; +- stronger filtering for young or unreliable history; +- firefly suppression; +- disocclusion smoothing; +- temporally varying analytic R² kernel rotation; +- optional temporal blending. + +Its changing R² kernel is effect sampling noise, not camera projection jitter. + +For temporal operation, the node relies on a reprojected input and feedback +history. The important transferable idea is not to insert this node unchanged, +but to coordinate effect history with the same jitter, motion, reset, and frame +identity used by the upscaler. + + +## What n8AO's neural denoiser does + +Source: + +- [NeuralDenoise.js](https://github.com/N8python/n8ao/blob/master/src/NeuralDenoise.js) +- [PoissionBlur.js](https://github.com/N8python/n8ao/blob/master/src/PoissionBlur.js) + +n8AO's neural stage is not a temporal neural denoiser. It is a compact learned +spatial correction inside the second Poisson blur iteration. + +At a high level it: + +1. gathers 4, 8, or 16 neighboring AO samples; +2. encodes local position, normal, occlusion, and distance features; +3. runs a small int8 attention model; +4. predicts a scalar residual; +5. adds that residual to the conventionally filtered AO result. + +The model does not use: + +- motion vectors; +- previous-frame textures; +- camera projection jitter; +- temporal reprojection. + +n8AO has a separate accumulation path, but it accumulates only while camera +matrices remain unchanged. Camera movement resets that history instead of +reprojecting it. + +The bundled neural weights are specific to n8AO's AO estimator, feature layout, +sample counts, and training distribution. Reusing those weights for SSGI, SSR, +or final scene color would not be valid. + +The implementation also does not receive special neural hardware acceleration. +Its int8 weights are emitted into a generated GLSL program and evaluated as +ordinary shader math. Actual cost depends on shader compilation, register +pressure, generated instruction count, and GPU scheduling. + + +## Where denoising could improve this project + +### Final scene color + +A general denoiser over final scene color is not the recommended first step. + +After SSGI or SSR has been composited into RGB, the upscaler cannot reliably +distinguish stochastic noise from: + +- texture detail; +- foliage; +- thin geometry; +- specular highlights; +- particles; +- intentional film grain. + +A generic filter would risk blur, haloing, and lost material detail across every +application, even when no noisy effect is present. + +### Separate noisy effect signals + +Effect-specific denoising has a clearer potential gain. + +Keeping SSGI or SSR separate allows the resolver to use information that final +RGB no longer contains: + +- normal; +- depth; +- roughness and metalness; +- albedo; +- AO value; +- reflection ray length; +- effect-specific variance; +- effect history age. + +That can improve quality or permit fewer expensive SSGI/SSR samples. + +### Low-confidence pixels + +The most promising spatial-denoise policy is to spend filtering work where +temporal history is weak: + +- newly disoccluded pixels; +- rejected history; +- low accumulation age; +- high local variance; +- reactive or rapidly changing pixels; +- pixels whose history was heavily clipped. + +Stable pixels with strong history should need less spatial filtering and a +smaller radius. + + +## Recommended experimental architecture + +The preferred experimental flow is: + +```text +Raw SSGI or SSR + -> jitter-aware effect reprojection + -> effect-specific temporal history and rejection + -> confidence-driven spatial denoise + -> composite with scene color + -> final temporal AA/upscale +``` + +This keeps effect denoising and final image reconstruction as separate concerns +while placing both on the same temporal timeline. + +### Required effect inputs + +Common: + +- raw effect color or scalar; +- depth; +- normal; +- velocity; +- current and previous projection jitter; +- reset generation; +- render dimensions; +- frame/sample identity. + +For SSGI: + +- albedo or diffuse color; +- optional direct/indirect separation; +- effect variance or confidence. + +For SSR: + +- roughness; +- metalness; +- hit distance or ray length; +- optional environment-hit classification. + +### Required temporal state + +Each independently denoised effect would need: + +- effect history; +- history age or confidence; +- previous depth; +- current and previous camera transforms; +- current and previous projection offsets. + +The effect resolver must use the same motion convention and reset cadence as the +final upscaler. + +### Spatial fallback + +Start with a small conventional kernel: + +- 5 to 8 taps; +- depth/plane-distance rejection; +- normal rejection; +- effect-specific material rejection; +- radius controlled by history confidence; +- stronger firefly suppression for young history; +- no filtering across disocclusions. + +This establishes a readable and tunable baseline before considering a learned +filter. + + +## Integration options + +### Option A: separate jitter-aware effect resolver + +Build a small reusable effect-history stage that consumes the application's +shared temporal context. + +**Advantages** + +- Clear separation from final AA/upscaling. +- Can use effect-specific inputs. +- Easier to compare or disable. +- Matches how dedicated GI/reflection denoisers are commonly structured. + +**Risks** + +- Additional history textures and passes. +- Requires careful scheduling. +- Can still create double-history lag if both stages are overly conservative. + +### Option B: fuse effect history into the upscaler + +Add dedicated SSGI or SSR inputs and resolve their histories inside the +upscaler's temporal pipeline. + +**Advantages** + +- One jitter and motion implementation. +- One reset lifecycle. +- Can share disocclusion and confidence signals. + +**Risks** + +- Couples the general upscaler to specific rendering effects. +- Adds inputs, textures, passes, and tuning to the core path. +- Makes the API harder to use for applications without those effects. +- Diffuse GI and specular reflection need materially different filtering. + +This should be treated as a focused research path, not a default extension. + +### Option C: final-color denoising + +Filter the accumulated or reconstructed final scene color. + +**Advantages** + +- Simple integration. +- No additional effect buffers. + +**Risks** + +- Cannot distinguish noise from detail. +- Applies cost and blur to applications that do not need denoising. +- Lacks normal, roughness, ray, and effect-confidence information. + +This is not recommended as the initial direction. + + +## Neural filtering considerations + +A neural filter becomes interesting if a conventional filter cannot provide the +desired quality at an acceptable tap count. + +Before pursuing it, the project would need: + +- a narrowly defined signal, such as diffuse GI or AO; +- representative training scenes and camera motion; +- noisy input and high-sample reference pairs; +- HDR-aware feature normalization; +- jitter-aware temporal examples; +- disocclusion and transparency coverage; +- separate training or features for diffuse and specular signals; +- a WebGPU implementation with measured shader size and register pressure. + +Potential benefits: + +- better edge preservation at a fixed tap count; +- learned rejection of recurring noise patterns; +- fewer SSGI/SSR rays for similar output quality. + +Potential costs: + +- model-training and dataset maintenance; +- content-dependent failure modes; +- difficult debugging; +- shader compilation and register pressure; +- no guarantee of being cheaper than a small hand-written kernel; +- new provenance, packaging, and model-version responsibilities. + +The n8AO implementation is evidence that a compact shader-resident model is +possible. It is not evidence that its model or architecture will generalize to +this upscaler. + + +## Benchmark outline + +Compare one change at a time. + +### Baselines + +1. Raw SSGI/SSR into the temporal upscaler. +2. Existing spatial `DenoiseNode` into the temporal upscaler. +3. `RecurrentDenoiseNode` with `accumulate: false`. +4. Current full recurrent experiment. + +### Proposed variants + +5. Jitter-aware temporal effect reprojection without spatial filtering. +6. Jitter-aware reprojection plus confidence-driven spatial filtering. +7. Reduced effect sample count plus the proposed denoiser. +8. Learned spatial correction only after the conventional path is understood. + +### Required scenes + +- static convergence; +- slow camera movement; +- fast translation; +- rotating camera; +- independently moving objects; +- disocclusion; +- thin geometry; +- glossy and rough reflections; +- diffuse GI around depth and normal edges; +- bright fireflies; +- camera cuts and reset; +- transparency where relevant. + +### Measurements + +- effect pass GPU time; +- denoiser GPU time; +- final temporal-upscale GPU time; +- combined frame GPU cost; +- temporal variance after convergence; +- disocclusion recovery time; +- visible trail length; +- retained edge/detail contrast; +- required SSGI/SSR sample count; +- memory footprint; +- compile and first-frame cost. + +Quality comparisons should use deterministic camera paths, fixed random seeds, +matched effect settings, clean history, and identical output transforms. + + +## Suggested next steps + +1. Preserve example 10 as the documented baseline for the current recurrent + conflict. + +2. Make a bench-only effect reprojection experiment that consumes exact current + and previous projection jitter. + +3. Verify reprojection in isolation before adding temporal blending or spatial + denoising. + +4. Add effect-specific history age, rejection, and reset behavior. + +5. Add a small confidence-driven spatial kernel. + +6. Compare raw, spatial-only, recurrent, and jitter-aware variants at equal + SSGI/SSR sample counts. + +7. Reduce effect samples and determine whether denoising produces a net GPU + saving rather than only an image-quality improvement. + +8. Decide between a separate reusable effect resolver and a fused experimental + path. + +9. Investigate a learned filter only if the conventional kernel is a measured + quality or performance bottleneck. + + +## Current recommendation + +There is a plausible quality and performance gain in actual denoising, especially +if it permits lower SSGI or stochastic SSR sample counts. + +The highest-value target is not a general denoiser over the upscaler's final +color. It is a jitter-aware, effect-specific temporal resolver with spatial +filtering concentrated on low-confidence pixels. + +The RecurrentDenoise implementation provides useful filtering ideas. The n8AO +model provides a useful example of compact learned spatial correction. Neither +should be inserted into the current temporal path unchanged. diff --git a/bench/docs/FSR3-BRIEF.md b/bench/docs/FSR3-BRIEF.md new file mode 100644 index 0000000..d66ebd5 --- /dev/null +++ b/bench/docs/FSR3-BRIEF.md @@ -0,0 +1,176 @@ +# FSR3-BRIEF.md — how the FSR3 port should decompose to flow with this pipeline + +Audience: the agents working on Dennis's separate FSR3-for-three.js port. That +repo stays self-contained; this brief defines the *decomposition and contracts* +so its internals can be produced early and consumed by effects (SSGI temporal, +denoisers) well before the upscale — instead of shipping as one monolith that +only runs last. + +## The architectural thesis (why decompose) + +Disocclusion, dilated motion vectors, reactivity, and history validity are +**frame properties, not upscaler properties**. FSR3 computes them internally +and hides them; meanwhile every temporal effect upstream (our SSRT3 temporal +pass, bilateral denoisers, any TAA) re-derives worse versions privately. The +port should expose FSR3's early data products as standalone passes with clean +texture contracts, so one computation feeds everything. See +`advantagous-concepts.md` ("temporal guides") for the consumer-side motivation. + +## Required decomposition (pass boundaries, not a monolith) + +1. **Reconstruct & dilate** (early — immediately after G-buffer): + - in: depth, motion vectors, prev depth + - out: `dilatedVelocity` (closest-depth 3×3 dilation), `dilatedDepth`, + `disocclusionMask` (graded 0..1, depth-reprojection based — NOT binary) +2. **Locks / reactivity** (early-mid): + - in: luma, history, app-provided reactive hints + - out: `reactiveMask`, `lockStatus` (or nearest FSR3 equivalents) +3. **Accumulate + upsample** (late): consumes 1+2 plus color; owns history. +4. **RCAS sharpen** (last): consumes 3 only. + +Each pass = explicit input/output textures with documented formats and a +uniform block; no pass reaches into another's internals. Passes 1–2 must be +runnable WITHOUT 3–4 (that is the whole point): an app that never upscales can +still run them as its temporal-guides provider. + +## Contracts this bench expects (the "temporal guides" bundle) + +Produced once per frame, post-G-buffer, pre-effects: + +| texture | format (suggested) | contents | +|---|---|---| +| `dilatedVelocity` | rg16float | closest-depth-dilated NDC delta | +| `disocclusion` | r16float | graded disocclusion 0 (stable) .. 1 (fresh) | +| `historyLength` | r16float | per-pixel valid-history frame count N | +| `reactive` | r8unorm | app/effect-flagged fast-changing pixels | + +Notes: +- `historyLength` may be produced by the consumer instead (demo 03 already + maintains one for frame-count accumulation); the contract just names it. +- `reactive` flows the other way too: GI/multibounce passes can WRITE into it + (fast-changing GI ⇒ don't lock/ghost) — design the pass to accept an + optional pre-populated mask and merge rather than overwrite. + +## three.js r185 / WebGPU environment facts (traps we already paid for) + +- `PostProcessing` is deprecated → `RenderPipeline`; after reassigning + `outputNode` you MUST set `needsUpdate = true` or the graph silently keeps + rendering the old output. +- Velocity convention (three MRT `velocity`): ndc delta; UV reprojection is + `prevUV = uv - velocity.xy * vec2(0.5, -0.5)` (TRAANode convention). +- Compute/raw passes: top-left UV origin, y-down. Anything derived from Unity + or Shadertoy (y-up) needs the y-mirror at the screen-step seam — this bug + cost us the demo 03 "box-top speckle" hunt; check every direction that is + used in BOTH view space and screen space. +- Raw WGSL passes never allocate GPU textures: allocate as three + RenderTarget/StorageTexture, bridge via `renderer.backend.get(...)` + (isolated in one file, see `bench/src/harness/three-internals.ts`), so + outputs stay consumable as TSL `texture()` nodes. +- Camera jitter: `setViewOffset` (TRAANode convention); harness helpers exist + (`bench/src/harness/temporal.ts`, Halton(2,3)). +- Known port bug from earlier work: **ACES tonemapping baked into RCAS** — + RCAS must be tonemap-agnostic (sharpen in the same space FSR3 specifies, + don't embed a display transform; three applies output encoding itself). + Shadertoy-derived passes similarly must not carry a trailing `pow(x, 0.45)`. + +## Verification expectations (same discipline as this bench) + +- Headed browser only for WebGPU verification (headless chromium = software + adapter = black canvas). `--enable-unsafe-webgpu`, no Vulkan flag on macOS. +- Each pass verified standalone with a scripted capture before integration; + numbers recorded with config + commit (see TESTING.md methodology). +- The bench will consume passes 1–2 in a lab (demo 10 extension) A/B-ing + guides-fed SSGI temporal vs its private logic — that lab is the acceptance + test for the decomposition being real and not cosmetic. + +## Addendum — data products SVGF wants (denoiser as first-class consumer) + +Motivation: the bench is adding an SVGF-style GI denoiser (variance-guided +à-trous). Its temporal front-end is *identical* to FSR3's, and its missing +half (per-pixel variance) reuses FSR3's luminance-statistics machinery. This +addendum expands the decomposition so the port serves the denoiser directly. +See `advantagous-concepts.md` (SVGF-style variance moments) for the +consumer-side motivation; the bench SVGF lab is the acceptance test for the +statistics primitive below being real and not cosmetic. + +**The one rule that governs this addendum — signal & space.** FSR3's +frame-property outputs (motion, disocclusion, depth, history length) are +*signal-agnostic*: correct for any temporal consumer. Its luminance-derived +products are NOT — they live on tonemapped display-luminance of the full +beauty, in FSR3's exposure/YCoCg conventions, sometimes at display res. SVGF +variance must live on **GI irradiance: pre-albedo, linear HDR, render res**. +So the denoiser consumes FSR3's *geometry/frame data* directly, but must NOT +consume its *luminance buffers* — it reuses the luminance *pass* on a +different input. Every luma-derived output MUST carry a space label so a +consumer cannot mis-apply display-luma to a linear-HDR denoiser. + +### A. Promote three internals into the published contract + +These already exist inside passes 1–2 but aren't in the consumed bundle. SVGF, +the SSGI march, and the temporal pass each re-derive them privately today; +publish once. + +| texture | format (suggested) | contents | space | +|---|---|---|---| +| `dilatedDepth` | r32float | closest-depth 3×3 dilated depth (from pass 1) | linear eye-Z | +| `prevDepth` | r32float | previous frame's depth (pass-1 owns the copy) | linear eye-Z | +| `lockStatus` | rg16float | FSR3 lock state / trust (or nearest equivalent) | unitless | + +- `prevDepth`: the bench maintains ≥3 private copies right now + (`ssrt-copydepth.wgsl`, `ssrt12-copydepth.wgsl`, guides-commit). One + published copy retires all of them. +- `lockStatus` feeds SVGF as an anti-ghost / history-rejection prior; if the + port has no clean lock equivalent, omit rather than approximate. + +### B. The reusable statistics primitive (SVGF's missing half) + +FSR3 already computes luminance moments/pyramids for its own stability and +neighborhood-clamp logic. SVGF's variance stages want the SAME computation on +a DIFFERENT signal. Factor it as a signal-agnostic pass, not a beauty-hardcoded +internal: + + Pass: MomentPyramid (or nearest FSR3 luma-pyramid refactor) + in: + source : any texture (FSR3 passes beauty-luma; SVGF passes GI) + space : enum { linear-irradiance, tonemapped-display, ycocg } + channel : which channel(s) form the scalar "luminance" + out: + moments : rg16float — (E[x], E[x^2]) per pixel → Var = E[x^2]-E[x]^2 + pyramid : optional mip chain of the above (spatial fallback, + short-history pixels borrow a coarser level) + + Contract: the pass reads `source`/`space` as parameters and hardcodes + NO exposure/tonemap/albedo assumption. FSR3 instantiates it on beauty-luma + for its stability term; the bench instantiates the same code on pre-albedo + GI irradiance for SVGF variance. One primitive, two consumers — the guides + thesis applied to statistics instead of geometry. + +If the accumulate stage already tracks a per-pixel accumulation weight / +confidence distinct from `historyLength`, expose it too (`r16float`) — SVGF +consumes it as a variance prior for freshly-accumulated pixels. + +### C. What SVGF does NOT need from the port (avoid over-serving) + +- **The à-trous / spatial wavelet filter** — SVGF builds its own; RCAS is a + display-res sharpen at the wrong stage and is not a substitute. +- **Reconstructed-from-depth normals** — the bench has real G-buffer view + normals (`normalView`); FSR3's depth-reconstructed normals are strictly + worse. Do not consume them. +- **Any display-res or post-tonemap luminance buffer** — see the space rule; + wrong signal, wrong space for GI denoising. + +### D. Reactive is bidirectional here too + +SVGF is a `reactive` PRODUCER as well as a consumer: high GI variance ⇒ flag +reactive (fast-changing GI shouldn't lock/ghost in the upscaler). Same +merge-not-overwrite contract as the base `reactive` note above. + +### E. Verification + +- The `MomentPyramid` primitive is verified standalone: instantiate on a known + synthetic input, assert `Var = E[x^2]-E[x]^2` matches a CPU reference within + tolerance, in BOTH a linear and a ycocg space, before any denoiser consumes + it. Numbers recorded with config + commit per TESTING.md. +- The SVGF lab (bench, demo-13 candidate) A/Bs variance-guided denoise vs the + current fixed-radius bilateral — that lab is the acceptance test for this + addendum's statistics primitive being real and not cosmetic. diff --git a/bench/docs/NEXT-STEPS.md b/bench/docs/NEXT-STEPS.md new file mode 100644 index 0000000..f424e1d --- /dev/null +++ b/bench/docs/NEXT-STEPS.md @@ -0,0 +1,149 @@ +# Post-parity adoption record (2026-07-21) — all items landed + +Outcome of the parity program: no candidate bundle adopted wholesale (see +[PARITY-DECISIONS.md](PARITY-DECISIONS.md) and the consumer-facing +[/PARITY.md](../../PARITY.md)). Four items survived as adoption-worthy, and +**all four landed on 2026-07-21** — this document is the evidence record for +each. Nothing from the parity program remains open. + +Every item follows the same gate: `npm test && npm run typecheck && npm run lint`, +then an A/B timing + capture run +(`node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant --comparison `, +plus `--mode capture --scenarios Q0,Q1,Q3 --reloads 1 --allow-differences --review-all`). +≥5% repeatable = actionable, <3% = noise; any visual regression rejects. + +## 1. RCAS input-range investigation — DONE (adopted: conditioned-space sharpening) + +The measured "resolver history made RCAS 47% cheaper" was **not** a value-range +effect — production history texels are bounded [0,1). Reading the wiring showed the +cost: with `FLAG_INPUT_REINHARD`, production RCAS paid a 1×1 exposure load + a +`tonemapInvert` division + an exposure division **per tap** (5 taps/pixel); the +resolver ran with the flag off (plain loads). + +- Two isolating variants were built and ABBA-timed (warm blocks, ratio 2): + `rcas-hoisted-exposure-v1` (identical math, hoisted exposure) → **−20% RCAS**; + `rcas-tonemap-space-v1` (sharpen the bounded tonemapped texels, invert once) + → **−34% RCAS** (0.103 → 0.068 ms), −5.7% total pipeline compute. +- Captures Q0/Q1/Q3 + Q9 HDR stress: full-frame RMSE ≤ 1.8/255, HDR-bulb ROI + ≤ 9/255 max — visually indistinguishable, no overshoot. +- **Adopted** as production `RCAS_SHADER`. The per-tap form is frozen as + `RCAS_PER_TAP_SHADER` behind the `rcas-fsr315-limiter` / `rcas-fsr315-numeric` + bench identities; the timing variants remain in the registry for re-testing. + +## 2. Host pre-exposure correction — DONE (DeltaPreExposure semantics) + +- Pyramid publishes host pre-exposure in the exposure texel's `.b` (1.0 when no + `preExposureTexture` is supplied) and **meters host-invariantly** — auto-exposure + must not chase a step the app already metered (found on GPU: without this, the + conditioning re-adapts for ~2s after a host step and the drift reads as a + full-screen shading change on flat regions). +- Accumulate ratio-corrects reprojected history in linear space when the host value + changed (binding 11 = previous frame's exposure texel). Self-gating: identity + without the input. +- Validated on the new **Q11 host-pre-exposure scenario** (bench drives the scene + MRT color and `preExposureTexture` together; manifest updated): 2.5× step + ramp + leaves the shading detector at baseline, never resets accumulation age, and output + brightness tracks the drive. No-input captures are **byte-identical** pre/post. + +## 3. AMD disocclusion constant — DONE (in the fused reconstruct pass) + +- `DEPTH_SEPARATION_SCALE`/`DEPTH_SIMILARITY_FLOOR` guesses replaced by AMD's + per-bilinear-tap confidence voting with the viewport/depth-scaled tolerance + (`1.37e-5 · halfViewportWidth · max(depth)`), lifted from the GPU-verified + candidate port. Fused single-pass structure kept (the source's atomic scatter + + separate pass measured +30%/+22% with no visual win). +- Q3 validation: thin stable silhouette outlines, still scenes near-black, age + resets confined to trails; finals shift RMSE ≤ 1.1/255; reconstruct pass time + unchanged (0.035 ms at ratio 2). + +## 4. Multi-scale shading-change detector — DONE + +The long-standing roadmap item (the source's SPD coarse-mip detector concept) +landed as `src/shaders/shadingChange.ts`: one fused half-resolution +dispatch (an 8×8 workgroup covers a 16×16 render tile, so the 4×4/8×8 reductions are +workgroup-local) that maintains a 1-frame luma history, compares jitter-aligned +block-mean luma per scale with base + contrast-scaled noise floors, neutralizes +disoccluded texels, and feeds accumulate's `FLAG_SHADING_CHANGE` aging path (binding +12). Locks kept their self-referential break — untouched, per the documented trap. + +Five GPU tuning iterations were needed (all evidence in +`bench/results/raw/E00/pre-spd-reference` + `post-spd-v*`): +1. The candidate's mean-of-per-texel-signed-ratios floored at ~0.10 still-scene + response — the relative-difference metric weights the darker side of alias + residue, a coherent bias signed averaging cannot cancel. +2. Jitter-delta-aligned bilinear reprojection helped but did not fix it. +3. Ratio-of-block-means (average first) collapsed the floor. +4. Disocclusion neutralization + coefficient-of-variation-scaled floors fixed + moving-silhouette false fires. +5. Dropping the 2×2 scale (thin features flicker at that scale regardless) hit the + full acceptance matrix: still scene at the old detector's baseline (Q1 ≈ 2 vs + 1.1), **fewer** false positives under camera motion on high-frequency content + (Q4 worst 3.6 vs old 4.9), light steps fire as clean single-frame spikes (Q9: + 137/255 vs old 84 with a 20-frame decay tail), host pre-exposure steps quiet + (Q11), finals within 1.6/255 RMSE of the old detector. + +Cost: **0.044 ms** at ratio 2 (the candidate's two-pass form measured 0.231 ms; +5× cheaper), zero when `settings.detectShadingChanges` is off. Slow ramps +deliberately do not fire (the 1-frame comparison sees only the per-frame delta; +blend + variance clip track ramps — verified no lag/ghosting on Q9 ramp finals). + +## 5. Still-scene convergence defect — DONE (2026-07-24, consumer report 3) + +The first full-pipeline consumer (ssgiDev demo 16, GUIDES-HANDOFF-RESPONSE +report 3) observed visible still-camera output jitter, flickering +`Disocclusion` silhouettes, and a never-settling rolling `AccumulationAge` — +at every ratio including NativeAA, immune to every exposed knob. Reproduced +in OUR bench (Q1, capture mode — no consumer code): sustained +consecutive-frame meanAbsDiff **0.211** after 3 s settle, and — decisive — +**same-jitter-phase** diff 0.182 one period apart, so history itself churned +aperiodically, not just the benign per-phase pattern. Metrology tool: +`scripts/measure-convergence.mjs` (CDP, deterministic capture API, +consecutive + phase-locked diffs, debug-view PNGs). Three stacked defects: + +1. **Reconstruct depth-clip vote starved of agreement** (`reconstruct.ts`). + Only positive-separation taps voted OR carried weight, so at a still + silhouette one bilinear tap straddling the previous frame's dilated-depth + quantization (boundary lands up to a texel away per phase) became the sole + voter → disocclusion 1.0 at edges, re-flipping with jitter phase. Fix: + every valid tap votes (taps at/behind the surface = confidence 1) and the + **best tap wins** (max, not weighted mean) — any tap recognizing the + current surface means same surface; a genuine trail has every tap on the + old occluder and still reads ~1. This supersedes the 2026-07-22 "skip, + never veto" semantics (skipping still let lone outliers decide). +2. **Clip-magnitude history aging** (`accumulate.ts`, removed). Aging + `sampleCount` by `clipAmount` put convergence out of reach wherever the + converged mean sat outside one jitter phase's variance box (any contrasty + edge; normalized by `extents` it also fired on numerically-tiny deviations + on flat walls) — equilibrium age stayed low, alpha stayed high, the age + view rolled forever. FSR2 never ages on rectification strength; stale + shading is the clip's + shading detector's job. +3. **Clip write-back re-snapping converged history** (`accumulate.ts`). + With 1+2 fixed, phase-locked diff was STILL 0.183: the blend stores the + *clipped* history, so each phase's box re-snaps the buffer regardless of + alpha (clip fully disabled: 0.005). Fix: `STILL_CLAMP_RELAX` — widen the + box ×9 only at full stillness (<0.05 render-texel motion) × converged + history × no disocclusion/shading-change/reactivity; any signal restores + full rectification. The locks mechanism generalized softly to everywhere. + +Q1 ratio 2 ladder (consecutive / phase-locked meanAbsDiff, 0–255): pre +0.211/0.182 → fix 1+2: 0.182/0.183 → +relax ×4: 0.116/0.038 → **+relax ×8: +0.112/0.018** (shipped) → rectification off (floor): 0.109/0.005. NativeAA +ratio 1: 0.081/0.003. New **Q12 cornell-still-convergence** (enclosed box, +IGN-dithered Vogel point-light shadows — the consumer's screen-anchored- +dither aggravator, camera per their repro pose): **0.024/0.012**, disocclusion +view fully black, age saturated (consumer's cornell measured 0.19–0.76; their +converging SVGF reference is 0.039). No-regression: Q3 disocclusion shows the +documented thin trailing crescents only, final ghost-free; Q4 mid-orbit final +clean (the relax fades out above 0.5 texel/frame motion). Runs under +`bench/results/raw/convergence/` (pre-fix / post-fix / exp-noclip / +exp-still8 / post-fix2 labels). + +## Explicitly not planned (measured against) + +- Lanczos2/bicubic history filtering (+47% accumulate, no visible win). +- Farthest depth / motion divergence signals (+30% prepareInputs, outputs unconsumed). +- Atomic depth scatter as a wholesale replacement for the fused reconstruct pass. +- T&C as a distinct softer channel — revisit only on user demand with real content. +- Conditioning-exposure history correction (beyond host pre-exposure): eased + adaptation keeps the per-frame mismatch under the shading detector's threshold; + correcting it changes output for every auto-exposure user. Revisit with evidence. diff --git a/bench/docs/PARITY-CANDIDATES.md b/bench/docs/PARITY-CANDIDATES.md new file mode 100644 index 0000000..683a890 --- /dev/null +++ b/bench/docs/PARITY-CANDIDATES.md @@ -0,0 +1,302 @@ +# Parity Candidate Test Guide + +## Current status + +**Update 2026-07-18:** all three bundles are now **GPU-compiled, validated, rendered, +timed, and captured** (after fixing five defects that had prevented any of them from +surviving `init()` — see PARITY-DECISIONS.md). Measured results and the reproduce +command live in `PARITY-DECISIONS.md`; raw evidence in `bench/results/raw/CANDIDATES/`. +The candidate A/B path is simply `run-benchmark.mjs --smoke --variant +--comparison ` — no candidate-specific manifest was needed (the cross-variant +`analyzeAbba` label crash was fixed to compare only shared pass labels). + +The production fallback is unchanged and remains the default. No bundle is adopted: +all three cost more compute than production (+36%, +43%, +76% cumulative) without a +demonstrated visual win on the deterministic scenarios. + +**Program conclusion (2026-07-21):** four individual behaviors identified through +these bundles were extracted and landed in production in re-derived, cheaper forms — +conditioned-space RCAS, host pre-exposure correction, AMD's disocclusion threshold, +and the multi-scale shading-change detector. Evidence: [NEXT-STEPS.md](NEXT-STEPS.md). +The bundles themselves stay registered as bench variants for future re-testing. + +## What is already production behavior + +These changes are adopted; they are not part of the unmeasured candidate decision: + +- **Linear/HDR output:** internal ACES and sRGB presentation transforms were removed from + EASU, RCAS, blit, final output, and debug output. The upscaler returns caller-domain + `rgba16float`; the consuming renderer owns tone mapping and output encoding. +- **RCAS numeric parity:** production RCAS now uses the FSR 3.1.5 lower limiter and the + corrected denoise luma/range math. RCAS denoise remains opt-in because enabling it by + default changed high-contrast detail without a demonstrated noisy-input benefit. + +The production graph, adopted RCAS math, linear/HDR output contract, and opt-in denoise +policy are the fallback for every candidate below. + +## What was authored + +The candidates are cumulative and must be tested in this order: + +1. `source-filter-bundle-v1` +2. `source-structural-bundle-v1` = filter bundle + structural inputs/reactivity +3. `source-spd-resolver-bundle-v1` = structural bundle + SPD signals/coordinated resolver + +This ordering matters. Comparing each bundle only with production would show the total +effect, but adjacent comparisons are needed to attribute the incremental cost and image +change. Later adoption does not have to be all-or-nothing if evidence supports extracting +a compatible stage, but no stage should be transplanted before its dependencies and +state semantics are understood. + +### `source-filter-bundle-v1` + +**Changes** + +- Separates host `preExposure` from internal conditioning exposure, tracks previous and + current values, and moves reprojected history into the current domain before filtering. + Internal conditioning is removed at output while host pre-exposure remains. +- Replaces current-frame reconstruction with radial approximate Lanczos2 and adaptive + kernel bias. +- Replaces history sampling with a deringed 4×4 bicubic Lanczos reconstruction. +- Replaces fused local depth reconstruction with an atomic `u32` nearest-depth scatter, + an explicit pass boundary, and viewport/depth-scaled disocclusion. +- Selects an FSR1-style EASU shader using approximate reciprocal and reciprocal-square-root + helpers while preserving the existing 12-load topology. + +**Likely problems addressed** + +- Exposure adaptation or app pre-exposure changes can otherwise compare current and + history color in different domains, causing pumping, trails, or incorrect clipping. +- Source-style current/history filters may improve sub-pixel reconstruction, moving-edge + stability, and history deringing. +- Atomic previous-depth scatter more closely represents reprojected geometry and may + reduce incorrect disocclusion around moving silhouettes. + +**Why these changes are grouped** + +They form the smallest source-style reconstruction path that can retain the local +accumulation/lock model. Exposure correction must occur before history filtering, and the +depth result supplies the disocclusion signal used by that filter path. + +**Expected tradeoff and fallback** + +The 4×4 history filter, atomic scatter, and extra depth pass can cost more GPU time and +bandwidth. Approximate EASU math may reduce ALU cost, but introduces numeric error that +could soften or destabilize spatial edges. Production exact EASU, local reconstruction, +and local exposure/history behavior remain available by selecting no candidate. + +EASU is exercised only by the spatial path. The registered bundle metadata currently +selects the temporal path, so a normal temporal candidate run will compile the EASU +pipeline but will not measure its dispatch. A focused spatial manifest/runner is required +before making an EASU claim. + +**Evidence needed** + +- WebGPU compilation/validation for all supported ratios and odd render sizes. +- Per-pass and compute-sum timing against production, then visual review of convergence, + camera/object motion, silhouettes, and reset behavior. +- Controlled exposure steps and ramps that distinguish internal conditioning changes + from host pre-exposure changes. +- A separate spatial EASU comparison using edge/detail captures and numeric differences. + +### `source-structural-bundle-v1` + +**Changes** + +- Includes all of `source-filter-bundle-v1`. +- Prepares farthest depth and current luma alongside nearest depth/motion. +- Adds motion divergence to identify unreliable reprojection. +- Adds configurable source-style opaque-versus-final reactive generation. +- Max-dilates application reactivity into the aggressive history-reset/shading channel. +- Keeps Transparency & Composition (T&C) plus motion divergence in a distinct, softer + rectification channel instead of treating them as fully reactive. +- Adds render-resolution accumulation/reset state and atomic transient new-lock + preparation. + +**Likely problems addressed** + +- The current generated reactive mask has fixed policy and one aggressive meaning; + source-style controls should better classify transparent or composition changes. +- Treating T&C exactly like reactive content would discard too much history. The softer + channel should reduce ghosting without forcing every transparent pixel to current color. +- Farthest depth and motion divergence provide context for depth boundaries and + inconsistent motion that simple nearest-depth disocclusion misses. +- Prepared accumulation and new-lock state coordinate resets before display-resolution + accumulation. + +**Why these changes are grouped** + +These signals share the prepare-inputs and prepare-reactivity resource graph. Testing one +without its consumers would pay structural cost without evaluating the behavior it was +added to drive, while mixing them into the local mask channels would make attribution and +fallback semantics ambiguous. + +**Expected tradeoff and fallback** + +Expect extra textures/buffers, an additional prepare-reactivity dispatch, atomic lock +scatter, and more source reads. The likely quality upside is less transparency ghosting, +better silhouette handling, and more selective history distrust; the main quality risks +are over-resetting, noisy masks, flicker, and locks appearing on ordinary edges. The +filter candidate and production graph remain lower-cost fallbacks. + +**Evidence needed** + +- Incremental timing versus `source-filter-bundle-v1`, including `prepareInputs`, + `depthClip`, and `prepareReactivity`. +- Q5 transparency captures with manual/application reactive input, generated reactive + input, and T&C input reviewed separately. +- Reactivity, disocclusion, accumulation-age, locks, and motion-vector debug views to + verify channel meaning and alignment. +- Moving-geometry and camera-motion review for farthest-depth and motion-divergence + behavior, including false positives on stable opaque surfaces. + +### `source-spd-resolver-bundle-v1` + +**Changes** + +- Includes all of `source-structural-bundle-v1`. +- Adds a luma mip chain carrying spatial luma/depth information and frame exposure state. +- Adds a signed current/history luma-difference mip chain and three-mip shading-change + resolve. +- Adds persistent render-resolution four-frame luma history and instability. +- Replaces the local resolver with coordinated source-style current/history + reconstruction, dynamic rectification, accumulation weighting, ridge locks, and state + packing. +- Stores lock lifetime in history alpha. Production stores normalized sample age there, + so the two histories are intentionally incompatible and must reset when switching. + +**Likely problems addressed** + +- Coarse luma context should make shading-change detection less sensitive to local + high-frequency detail than the current 3×3 heuristic. +- Four-frame instability can preserve valid recurring detail that a one-frame comparison + might clip, while still rejecting unstable history. +- Coordinating accumulation, rectification, locks, and prepared state avoids combining + signals designed for different state models. + +**Why these changes are grouped** + +The mip signals, instability history, lock lifetime, accumulation value, and rectification +weights are mutually dependent. Testing them as independent toggles would produce invalid +state combinations. This bundle is the only candidate whose history alpha has source-style +lock semantics. + +**Expected tradeoff and fallback** + +This is the highest-cost and highest-risk candidate: it adds luma/shading dispatches, +history resources, and coarse reductions before a more complex resolver. The WebGPU-safe +mip implementation directly rereads the prepared source at higher levels instead of using +AMD's device-wide atomic SPD counter; that is portable but may duplicate substantial work. +Potential benefits are steadier lighting transitions, better detail retention, and more +coherent locks. Risks include false shading changes, excess history retention, flicker, +ghosting, incorrect reset state, and enough cost to outweigh fidelity gains. The +structural bundle and production graph remain fallbacks. + +**Evidence needed** + +- Incremental timing versus `source-structural-bundle-v1` for `lumaSpd`, `shadingSpd`, + `shadingResolve`, `lumaInstability`, and `accumulate`, plus total compute. +- Exposure-transition, lighting-change, static-convergence, motion, cut, reset, and resize + captures with history reset confirmed at every boundary. +- Shading-change, locks, accumulation-age, exposure, and final debug review that checks + signal meaning rather than only final-image similarity. +- Multi-frame metrics for convergence, temporal variance/flicker, disocclusion trails, + thin-feature retention, and ghost persistence. + +## Static audit corrections + +The post-authoring audit changed candidate-only code to correct likely defects before GPU +work: + +- restored accumulation-state growth in the structural path; +- seeded all four luma-history slots after reset or offscreen reprojection; +- computed reconstruction weights from unclamped tap positions at image borders; +- rounded odd-size SPD allocations so every written mip has sufficient extent; +- compile-time-specialized farthest-depth/current-luma preparation and motion divergence + so `source-filter-bundle-v1` does not silently execute structural work; +- kept benchmark-only constructor hooks out of the public constructor declaration; +- aligned T&C documentation and benchmark metadata with the implemented dispatch inputs. + +GPU-free checks previously passed after those fixes, but static checks cannot prove WGSL +device compilation, binding validity, image quality, timing, or cross-adapter behavior. + +## Later test matrix + +Test in project priority order: **performance > quality > realism**. + +### 1. Performance gate + +Measure both total and incremental cost: + +- production → `source-filter-bundle-v1`; +- `source-filter-bundle-v1` → `source-structural-bundle-v1`; +- `source-structural-bundle-v1` → `source-spd-resolver-bundle-v1`; +- production → each candidate as a total-cost check. + +Use ratios `1`, `1.5`, `2`, and `3`; fixed dimensions, timestep, seeded scene state, +camera path, reset state, and presentation domain; and fresh per-frame timestamp-query +samples. Record every pass median/p95, compute-sum median/p95, missing samples, adapter, +browser/backend, shader keys, and resource graph. + +Interpret repeated measurements using the established practical thresholds: + +- **below 3%:** tied/noise; +- **3–5%:** uncertain; rerun before a decision; +- **at least 5%:** actionable when repeatable. + +A visual regression rejects a candidate regardless of speed. A performance loss of at +least 5% requires a clear, repeatable quality benefit to remain under consideration. + +### 2. Quality gate + +Prioritize the existing deterministic scenarios: + +- **Q0 input/debug validation:** compile/binding sanity and every candidate debug view. +- **Q1 static convergence:** thin-feature retention, lock growth, convergence, flicker. +- **Q2 slow aliasing dolly / Q4 camera-motion hold:** reconstruction stability and trails. +- **Q3 object-motion disocclusion:** atomic depth, motion divergence, silhouette ghosts. +- **Q5 seeded transparency/reactivity:** generated/application reactive masks and softer + T&C behavior. +- **Q9 exposure transition:** conditioning/pre-exposure correction, pumping, and shading + response. +- **Q10 reset/cut/resize:** state packing, reset seeding, and resource recreation. + +Review final output plus motion vectors, disocclusion, accumulation age, locks, exposure, +shading change, and reactivity where each scenario defines them. Preserve blinded captures, +ROI max-absolute error and RMSE, frame-exact comparisons, and reviewer notes. Numerical +similarity is supporting evidence; temporal artifacts and visible regressions decide the +quality gate. + +Add focused odd-width/odd-height cases to validate candidate mip allocation; no existing +Q0–Q10 scenario supplies that coverage. + +Run a separate spatial EASU matrix because the temporal matrix does not dispatch EASU. +Compare edge sharpness, ringing, flat-field stability, small text/grid detail, and GPU cost +between exact production math and the approximate candidate. + +### 3. Realism/coverage gate + +Only after performance and deterministic quality survive, expand to Q6–Q8 screen-space +effects, representative HDR scenes, noisy GI/reflections, transparency-heavy content, +multiple adapters/browsers, and long camera paths. This gate checks whether a source-style +gain generalizes; it must not rescue a candidate that already failed deterministic +performance or visual review. + +## Existing harness entry points and missing work + +Verified existing package entry points: + +- `npm run bench` starts the interactive Vite bench. +- `npm run bench:run` and `npm run bench:capture` execute the immutable E00 harness. +- `npm run bench:compare:rcas` is the focused E01 RCAS comparison. + +The browser configuration recognizes `variant` and `comparison` selectors with these exact +candidate IDs, ratios `1`, `1.5`, `2`, and `3`, and scenarios `Q0` through `Q10`. +Candidate timing labels and resource identities are registered. + +There is **no focused candidate command or immutable candidate manifest yet**. +`bench:run`/`bench:capture` enforce E00's baseline roles for authoritative non-smoke runs, +and `bench:compare:rcas` covers RCAS only. Before collecting decision evidence, author +candidate manifests plus runner/report orchestration for cumulative and adjacent A/B +comparisons, including a separate spatial EASU comparison. Do not treat an interactive +selection or smoke run as adoption evidence. diff --git a/bench/docs/PARITY-DECISIONS.md b/bench/docs/PARITY-DECISIONS.md new file mode 100644 index 0000000..913f5b6 --- /dev/null +++ b/bench/docs/PARITY-DECISIONS.md @@ -0,0 +1,61 @@ +# FSR 3.1.5 Parity Decisions + +This is the concise decision record for the (concluded) parity experiment program. +Raw evidence remains under `bench/results/raw/`; the post-parity adoptions that came +out of these decisions are recorded with evidence in [NEXT-STEPS.md](NEXT-STEPS.md). + +Program facts: pinned FidelityFX SDK source `60f4ea81909200d8542eca14dccb2628b763a9a3` +(FSR Upscaler 3.1.5); local baseline commit `5d6a65e` on `feat-match-fsr3`. The E00 +benchmark harness is adopted as a first-pass engineering tool: repeatable changes +≥5% are actionable, <3% is treated as noise, 3–5% is uncertain, and visual +regressions reject a candidate regardless of speed. (Publication-grade fine-margin +acceptance was deferred — long ABBA sequences showed monotonic timing drift on the +test machine that prevents claims near 1.5–2.5% noise limits.) + +| Experiment | Candidate | Measured result | Recommendation | User decision | Action | +| --- | --- | --- | --- | --- | --- | +| E01 | FSR 3.1.5 lower limiter | Linear/HDR rerun: total compute median −0.62% (tie); sparse differences remained difficult to see without a heatmap. | Adopt: parity improves without a demonstrated total-cost or quality regression. | Adopt source behavior — 2026-07-17. | Integrated and GPU-verified in the production RCAS shader. | +| E01 | Source denoise luma and center-inclusive range | Linear/HDR rerun found no stable total-compute cost. Corrects green-only luma and the center-excluded range. | Adopt the source math whenever RCAS denoise is enabled. | Adopt source behavior — 2026-07-17. | Integrated and GPU-verified in the production RCAS shader. | +| E01 | Enable temporal RCAS denoise by default | Linear/HDR Q0/Q1 captures changed contrast edges by RMSE 0.55–0.60/255 with maxima 17–33/255; total compute median +2.87% (tie). | Test representative noisy temporal inputs before changing the default; this remains quality policy rather than math-only parity. | Pending explicit default-policy decision. | Keep current opt-in default until the noisy-input review. | +| E03 | Remove internal ACES/sRGB presentation | All output paths now write caller-domain `rgba16float`; temporal, spatial, bilinear, and depth-debug paths compiled and rendered on WebGPU. | Adopt unconditionally: FSR must not own tone mapping or output encoding. | Must remove — 2026-07-17. | Integrated. Bench/examples apply renderer presentation after the upscaler. | +| E05–E07, E15 | `source-filter-bundle-v1` | GPU-validated + measured 2026-07-18 (after 2 compile fixes): **+35–36% total compute** vs production at ratio 2 (accumulate +47%; exposure/RCAS tied; noise floor ≤1.7%). Q0/Q1/Q3 captures artifact-free; steady-state diff vs production grows to RMSE ~10/255. | Do not adopt wholesale — the source reconstruction/history filtering costs ~⅓ more compute with no demonstrated visual win on the bench scenes. Cherry-pick pieces (exposure-domain correction, AMD depth-separation constant) individually. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | +| E04, E08–E10 | `source-structural-bundle-v1` | GPU-validated + measured 2026-07-18: **+6.2–6.9% over the filter bundle** (prepareInputs +30%, depthClip +22%, accumulate/RCAS tied; noise floor ≤2%). Output near-identical to filter bundle (its prepared accumulation/newLocks outputs are currently unconsumed). | Cheap increment, but mostly inert until a consumer exists; evaluate only together with the resolver or after wiring its prepared state into an accumulate path. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | +| E11–E14 | `source-spd-resolver-bundle-v1` | GPU-validated + measured 2026-07-18 (after 3 compile/validation fixes + a `/0.5`→`/20.0` velocity-normalization correction): **+75–78% total compute** vs production (accumulate +38%; RCAS **−47%**, see note). Q0/Q1/Q3 artifact-free, no disocclusion trails; steady-state diff vs production RMSE ~4.6/255 — closer than the filter bundle's ~10/255. | Too expensive to adopt as-is. Two leads worth extracting: (1) why its history makes RCAS 47% cheaper (production history may contain value ranges that are ALU-hostile); (2) its coarse-mip shading detector is the long-planned SPD design, now GPU-proven (since landed in re-derived fused form — NEXT-STEPS item 4). | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | + +## E01 evidence + +- Reproducible comparison: `npm run bench:compare:rcas` +- Verified linear/HDR report: `bench/results/raw/E01/rcas-comparison-linear-hdr/index.html` +- Reopen report: + + ```bash + npm run bench:compare:rcas -- --reuse bench/results/raw/E01/rcas-comparison-linear-hdr + ``` + +## Candidate-bundle evidence (2026-07-18) + +Timing (ratio 2, 1920×1080 display, Apple Metal-3, headless Chrome 150, ABBA blocks, +240 warmup + 300 samples per run, `--smoke`): + +| Comparison | compute-sum A → B | Δ per block | Verdict (≥5% actionable / <3% noise) | +| --- | --- | --- | --- | +| production → filter bundle | 0.627 → 0.852 ms | +35.2…+36.1% | Actionable regression | +| filter → structural bundle | 0.850 → 0.906 ms | +6.2…+6.9% | Actionable regression (small) | +| production → resolver bundle | 0.627 → 1.104 ms | +75.2…+77.6% | Actionable regression | + +Per-pass: filter's cost is entirely `accumulate` (+47%); structural's is `prepareInputs` +(+30%) + `depthClip` (+22%) + the added `prepareReactivity` dispatch; the resolver adds +its SPD/instability passes and a heavier accumulate (+38%) while making `rcas` 47% +cheaper (input-content-dependent ALU — unexplained, worth investigation). + +Quality (Q0/Q1/Q3 captures, 168 blinded pairs per comparison, `review.html` in each +capture directory under `bench/results/raw/CANDIDATES/`): no artifacts, ghost trails, or +convergence failures spotted in any bundle; differences are sub-4% RMSE distributed over +edges/grid detail. Blinded human review remains open before any adoption decision. + +Five defects were fixed before the bundles would compile/run at all (they had never +touched a GPU): WGSL has no `isInf()`; `external` is a reserved keyword; `r8unorm` is +not a core storage-texture format (3 shaders + 4 allocations → `r32float`); the resolver +accumulate declared an unused sampler binding (auto-layout drops it → bind-group failure); +and its history velocity falloff used `/0.5` where every sibling uses `/20.0`. +Reproduce: `node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant --comparison `. diff --git a/bench/docs/THREE-TEMPORAL-COMPARISON.md b/bench/docs/THREE-TEMPORAL-COMPARISON.md new file mode 100644 index 0000000..e06c9f1 --- /dev/null +++ b/bench/docs/THREE-TEMPORAL-COMPARISON.md @@ -0,0 +1,1304 @@ +# three.js TRAA/TAAU comparison + +## 1. Scope and executive answer + +This audit compares the repository's **current** WebGPU temporal upscaler with the implementations installed in **three.js 0.185.1**: + +- `TRAANode`: same-resolution temporal reprojection anti-aliasing. +- `TAAUNode`: reduced-resolution temporal anti-aliasing plus upscaling. + +The exact installed version is pinned by `../package-lock.json:2717-2722` and `../node_modules/three/package.json:1-19`. + +The local implementation facts in this document describe the code as it exists now. They are not claims that every behavior is permanent design intent. + +### Executive answer + +- **TRAA is not the general upscaling comparator.** It resolves at the input resolution, so it compares directly only with local `QualityMode.NativeAA` at a 1× ratio. See `../src/types.ts:12-24`. + +- **TAAU is the closest reduced-resolution comparator.** It expects low-resolution beauty, depth, and velocity, then reconstructs an output-resolution image. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:10-30`. + +- **The local resolver is feature-richer and structurally heavier.** It has a separate reconstruction pass, persistent age and lock state, reactive masks, generated reactivity, shading-change handling, exposure conditioning, debug views, and RCAS. TAAU and TRAA are simpler and are therefore likely lighter, but there is no controlled GPU benchmark here that proves a winner. + +- **The local resolver can replace TRAA or TAAU for many WebGPU applications, but it is not a transparent drop-in.** The output color domain, logarithmic-depth support, render-graph hooks, resource footprint, reset controls, and APIs differ. + +- **Shared jitter is directionally correct only at the pipeline level.** The preferred design is an application- or render-pipeline-owned temporal sampling context. SSGI, SSR, GTAO, and denoisers should not depend directly on the upscaler. + +### Evidence labels used below + +- **Current implementation fact** means the behavior is directly visible in the audited source. +- **Source-level estimate** means operation or resource cost inferred from source structure, not measured GPU performance. +- **Unverified concern** means the source suggests a possible problem that still needs a focused runtime test. +- **Recommendation** means a proposed direction, not current behavior. + +The files under `node_modules` are useful evidence for the installed package, but they are addon/internal source, not a stable public API contract across future three.js releases. + + +## 2. Plain-language primer + +### Projection jitter + +**Projection jitter** moves the camera's projection by a fraction of one pixel before rendering a frame. Geometry then lands at a slightly different subpixel position each frame. + +A temporal resolver combines those differently positioned samples into a cleaner or higher-resolution result. Exactly **one final temporal resolver** should own this camera movement for a given render. + +This repository applies jitter with `camera.setViewOffset()`. The local frame lifecycle captures a stable projection first, applies jitter before rendering, and clears it afterward. See `../src/Upscaler.ts:260-297`. + +### Stochastic shader noise + +**Stochastic shader noise** changes where a shader samples within an AO, GI, reflection, or denoising kernel. It can rotate rays, shift a sampling pattern, or choose a new random direction. + +It does **not** move the camera. It also does not need to use the upscaler's Halton distribution. An SSGI ray pattern and a camera-jitter pattern solve different sampling problems. + +### Temporal history and reprojection + +**Temporal history** is a texture containing information from previous frames. + +**Reprojection** uses motion vectors and camera transforms to estimate where a previous-frame pixel belongs in the current frame. Without reprojection, camera or object motion would cause history to trail behind the scene. + +### Disocclusion + +A **disocclusion** is an area that becomes newly visible, such as background revealed when a foreground object moves away. + +There is no trustworthy history for a newly revealed surface. A temporal resolver uses depth and motion to detect that case and reject or heavily reduce the old history. + +### Rectification + +**Rectification** constrains reprojected history to colors that are plausible in the current neighborhood. + +It is a defense against stale history. If the old color lies far outside the range or variance of nearby current samples, the resolver clips it toward that neighborhood before blending. + +### Reactive mask + +A **reactive mask** identifies pixels where current color should dominate history. + +Transparent surfaces, particles, animated emissives, and additive effects often lack reliable depth or motion. Marking those pixels as reactive reduces ghost trails. The local API accepts a mask or can generate one from opaque-only versus final color. See `../src/types.ts:119-156` and `../src/Upscaler.ts:421-448`. + +### Why TRAA or TAAU must not run before the local resolver + +Stacking TRAA/TAAU before the local temporal upscaler creates two owners of temporal sampling: + +1. The upstream resolver jitters and temporally filters the scene. +2. The local resolver jitters or expects jitter again and temporally filters the already-resolved result. + +That creates **double jitter** or a mismatch between the color buffer and the final resolver's expected sample position. It also creates **double temporal filtering**, which can blur detail, retain stale history longer, and suppress the per-frame variance the final upscaler needs for reconstruction. + +Use one final AA/upscaling resolver per run: local Native AA, local temporal upscale, TRAA, or TAAU. + + +## 3. Exact implementations audited + +### Local repository + +- Low-level compute orchestration and frame lifecycle: `../src/Upscaler.ts:42-63`, `../src/Upscaler.ts:139-205`, and `../src/Upscaler.ts:260-339`. +- Composable graph node: `../src/UpscalerNode.ts:78-115` and `../src/UpscalerNode.ts:172-330`. +- High-level imperative scene driver: `../src/UpscalePass.ts:21-36` and `../src/UpscalePass.ts:88-172`. +- Jitter generation: `../src/math/halton.ts:1-41` and `../src/math/jitter.ts:3-68`. +- Temporal shaders: `../src/shaders/reconstruct.ts:4-105`, `../src/shaders/accumulate.ts:4-291`, and `../src/shaders/luminancePyramid.ts:4-95`. +- Output stages: `../src/shaders/rcas.ts:4-92`, `../src/shaders/blit.ts:4-46`, and `../src/shaders/common.ts:82-120`. +- Canonical bench integration: `./src/BenchPipeline.ts:14-22` and `./src/BenchPipeline.ts:125-162`. +- Visual fixtures: `../examples/04-aliasing-torture/main.ts:11-14`, `../examples/05-transparency/main.ts:12-18`, `../examples/06-screenspace-gi/main.ts:27-32`, `../examples/09-kitchen-sink/main.ts:25-37`, and `../examples/10-ssgi-denoise/main.ts:27-52`. + +### Installed three.js 0.185.1 + +- TRAA: `../node_modules/three/examples/jsm/tsl/display/TRAANode.js`. +- TAAU: `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js`. +- SSGI: `../node_modules/three/examples/jsm/tsl/display/SSGINode.js`. +- GTAO: `../node_modules/three/examples/jsm/tsl/display/GTAONode.js`. +- SSR: `../node_modules/three/examples/jsm/tsl/display/SSRNode.js`. +- Spatial denoise: `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js`. +- Recurrent denoise: `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js`. +- Temporal reprojection for effects: `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js`. +- Analytic R² noise: `../node_modules/three/examples/jsm/tsl/utils/RNoise.js`. +- Render-pipeline hook storage: `../node_modules/three/src/renderers/common/RenderPipeline.js`. +- Shared velocity node: `../node_modules/three/src/nodes/accessors/VelocityNode.js`. + + +## 4. Jitter comparison + +### Local schedule + +**Current implementation fact** + +The local sequence is centered Halton(2,3): each generated value subtracts `0.5`, producing offsets in render-pixel units around zero. See `../src/math/halton.ts:29-41`. + +The phase count adapts to the upscale ratio: + +`round(8 × ratio²)` + +with a minimum of one phase. See `../src/math/jitter.ts:3-17`. + +Examples: + +- 1×: 8 phases. +- 1.5×: 18 phases. +- 2×: 32 phases. +- 3×: 72 phases. + +The upscaler: + +- captures a stable unjittered projection; +- advances the sequence; +- reads the newly advanced sample; +- applies it through `camera.setViewOffset()`. + +See `../src/Upscaler.ts:270-287`. + +Because `advance()` occurs before `current` is read, the runtime order is a one-sample rotation of the generated array. This does not change the set of samples or cycle length, but it matters when comparing frame-by-frame sequences. + +### three.js schedules + +**Current implementation fact** + +Both installed nodes generate 32 uncentered Halton(2,3) offsets and subtract `0.5` when applying them. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:735-754` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:803-822` + +Both increment with: + +`index % (_haltonOffsets.length - 1)` + +That makes the effective runtime cycle **31 samples**, leaving the final generated offset unused. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:328-338` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:361-370` + +TRAA and TAAU each install their own before/after render-pipeline callbacks to apply and clear camera jitter. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:443-464` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:502-527` + +TAAU's source comments say its offset is reduced to an output-pixel footprint, but the installed implementation applies the centered `[-0.5, 0.5]` offset directly through an input-sized view offset. No input-to-output scale is visible in that code. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:318-354`. + +That makes the intended TAAU jitter footprint internally inconsistent in the audited source. The benchmark should record the behavior as installed and treat any corrected output-pixel variant as a separate diagnostic. + +### Practical conclusion + +Both implementations use the same broad method: low-discrepancy Halton projection jitter and stable motion vectors. + +Local adapts phase count to scale ratio, while installed TRAA/TAAU use an effective fixed 31-frame cycle. Installed TAAU also has the input-pixel/output-pixel inconsistency above. + +For a **stock-product comparison**, preserve each resolver's own schedule. That compares what an application actually receives. + +For a **diagnostic comparison**, a bench may inject one identical sequence into both resolvers to isolate reconstruction and accumulation behavior. That result must be labeled as a modified diagnostic configuration, not stock TRAA/TAAU behavior. + + +## 5. Feature comparison + +### 5.1 Inputs and velocity convention + +#### Local + +The temporal path requires render-resolution color, depth, and velocity. Velocity is three's NDC delta, `current - previous`. Local multiplies it by `(0.5, -0.5)` to convert to UV delta and reprojects with `previousUV = uv - motion`. See `../src/types.ts:119-129`, `../src/Upscaler.ts:620-622`, and `../src/shaders/reconstruct.ts:86-100`. + +**Pros** + +- Matches three's velocity convention. +- Supports explicit reactive and exposure inputs. +- The low-level API makes frame reset and cadence explicit. + +**Cons** + +- Requires correct raw texture access and WebGPU-only integration. +- The current composable node mutates the shared three velocity singleton. +- Public documentation is inconsistent: `UpscalerConfig.jitter` says composable `upscale()` defaults off, but the node constructor currently defaults it on. Compare `../src/types.ts:91-111` with `../src/UpscalerNode.ts:129-165`. + +#### three TRAA/TAAU + +Both consume beauty, depth, velocity, and camera. Both use the same NDC-to-UV conversion and `historyUV = uv - offset`. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-681` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:663-679` + +**Pros** + +- Natural TSL graph integration. +- Returns a graph texture that can feed later post-processing. +- TRAA can use a velocity node from builder context rather than always assuming only the exported singleton. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:466-480`. + +**Cons** + +- Camera and shared velocity state are still modified through node-owned hooks. +- TAAU's jitter setup directly uses the exported global `velocity` singleton, even though its shader accepts a velocity input node. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:327-365`. + +#### Replacement implication + +The motion convention is compatible. The ownership and lifecycle APIs are not. Replacing one resolver with another requires rewiring render ownership, reset behavior, and output transformation rather than only swapping a function call. + + +### 5.2 Depth/motion dilation and disocclusion + +#### Local + +Local fuses nearest-depth motion dilation and depth rejection into one render-resolution compute pass. It searches a 3×3 depth neighborhood, chooses the nearest sample, carries that sample's motion, and compares current versus reprojected previous linear depth. See `../src/shaders/reconstruct.ts:4-25` and `../src/shaders/reconstruct.ts:60-104`. + +Local explicitly supports: + +- standard depth; +- reversed depth; +- perspective cameras; +- orthographic cameras. + +See `../src/shaders/common.ts:122-140` and flag staging in `../src/Upscaler.ts:598-645`. + +**Pros** + +- Strong current support for reversed and orthographic depth. +- Dilation is separated from output-resolution accumulation and reused there. +- Debug views expose motion, depth, and disocclusion. + +**Cons** + +- Previous depth is stored as last frame's positive linearized view-space depth and compared with current linearized view-space depth after motion reprojection. +- It does **not** transform the previous depth sample from previous view space to world space and then into current view space. +- It has no logarithmic-depth conversion path. + +The missing cross-frame camera transform is a quality risk during camera translation or rotation, especially on large depth gradients. It is a source-level difference, not a demonstrated failure in every scene. + +#### three TRAA/TAAU + +Both installed nodes reconstruct the previous depth sample using the previous projection, transform previous view position to world space, then transform it into the current camera view before comparing depth. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:536-548` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:572-581` + +TRAA additionally handles: + +- reversed depth; +- logarithmic depth; +- orthographic depth. + +See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:466-486`, `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:498-546`. + +Installed TAAU does not show equivalent branches in its current-depth sampling, and its previous-depth conversion always calls `viewZToPerspectiveDepth()`. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:537-579`. + +**Pros** + +- Matrix-transformed previous depth better accounts for camera motion. +- TRAA has the broadest depth-mode handling of the three audited paths. + +**Cons** + +- TAAU's installed depth path appears perspective-oriented and lacks explicit reversed/logarithmic handling. +- Depth rejection uses mutable scalar properties with fixed defaults; those defaults still require scene validation. + +#### Replacement implication + +Local explicitly handles reversed-depth and orthographic WebGPU scenes in ways that are not visible in installed TAAU's depth branches. + +TRAA remains safer where logarithmic depth is required. Local should not claim log-depth replacement until that path exists and is tested. + + +### 5.3 Current reconstruction and history filtering + +#### Local + +The current frame is reconstructed with a jitter-aware separable Lanczos2 kernel over a 3×3 render-texel footprint. History uses five filtered Catmull-Rom fetches. See: + +- `../src/shaders/accumulate.ts:83-120` +- `../src/shaders/accumulate.ts:149-201` + +**Pros** + +- Explicitly accounts for where the jittered input sample landed. +- Sharper history reconstruction than a single bilinear sample. +- Current reconstruction, moments, and min/max reuse the same 3×3 color loads. + +**Cons** + +- More filtering work and more source texture operations. +- Negative Lanczos lobes require deringing. +- The exact quality/performance tradeoff has not been benchmarked against installed TAAU. + +#### three TAAU + +TAAU reconstructs current color from a 3×3 input neighborhood with `exp(-2.29 × distance²)`, described in source as a Blackman-Harris Gaussian approximation. It reads one filtered history sample. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:629-715` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:717-728`. + +TRAA is same-resolution and therefore samples current color directly rather than running a reduced-to-output reconstruction kernel. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-681`. + +**Pros** + +- Simpler history path. +- TAAU reuses its nine current samples for both reconstruction and variance moments. + +**Cons** + +- One filtered history sample may blur or under-reconstruct compared with a sharper bicubic history filter. +- The fixed 3×3 current filter may not scale optimally across every upscale ratio. + +#### Replacement implication + +TAAU is the correct comparator for local reduced-resolution reconstruction. TRAA is not. + +The visual comparison should focus on thin geometry, moiré, camera motion, and newly revealed surfaces rather than treating filter names as proof of quality. + + +### 5.4 Rectification and accumulation + +#### Local + +Local: + +- computes neighborhood moments in YCoCg; +- clips history toward a variance box; +- stores accumulation age in history alpha; +- caps age with `maxAccumulation`; +- ages history under disocclusion, heavy clipping, shading change, and reactivity; +- lets stable thin-feature locks relax clipping and favor history. + +See `../src/shaders/accumulate.ts:204-290`. + +**Pros** + +- Rich control over stale-history rejection. +- Explicit age makes reset and convergence behavior inspectable. +- YCoCg usually creates a tighter color-aligned rectification box than RGB. + +**Cons** + +- More persistent state and more interacting tuning constants. +- More ways to over-reject history, protect bad history, or trade shimmer for ghosting. +- Many constants are shader constants rather than convenient diagnostic controls. + +#### three TAAU/TRAA + +TAAU computes RGB moments, narrows variance gamma under motion, clips history, and raises current-frame weight with motion. Its baseline current weight is `0.025`. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:123-132` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:717-755`. + +TRAA uses a `0.05` baseline, adds motion weighting, and can add subpixel-motion correction before the same broad variance/flicker-reduction blend. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:613-643` and `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:683-704`. + +Neither node stores the same explicit normalized age used locally. With valid stationary history and stable luminance, their baseline current weights produce an exponential-like response; motion and flicker reduction alter the effective weights. + +**Pros** + +- Fewer interacting state channels. +- Motion-responsive current weighting is direct and easy to tune. +- TRAA's optional subpixel correction is a focused anti-blur mechanism. + +**Cons** + +- Less explicit convergence control. +- No local-equivalent reactive, shading-change, or exposure-conditioned age policy. +- RGB variance can be less color-aligned than YCoCg. + +#### Replacement implication + +Local has broader controls for difficult content. Three's motion weighting and TRAA subpixel correction are still valuable experiments for local variants; they should be tested behind internal flags rather than assumed superior. + + +### 5.5 Reactive masks, locks, shading, and exposure + +#### Local + +These are mostly local-only features: + +- explicit reactive mask; +- generated reactivity from opaque/final color difference; +- persistent luminance locks; +- shading-change detection; +- automatic, fixed, or external exposure conditioning; +- debug views for each major signal. + +See `../src/Upscaler.ts:394-553`, `../src/shaders/accumulate.ts:211-290`, and `../src/shaders/luminancePyramid.ts:4-30`. + +**Pros** + +- Better tools for transparents, particles, thin features, and HDR stability. +- Caller can explicitly reset and inspect temporal state. + +**Cons** + +- Additional passes, textures, branches, and tuning surface. +- The current scalar exposure analysis is deliberately simpler than a full luminance pyramid. + +#### three + +TRAA has no equivalent reactive-mask, exposure, lock, or shading-change input. + +TAAU contains a thin-feature lock calculation and allocates two history attachments: color and lock. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:153-163` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:736-758`. + +**Unverified concerns: TAAU lock path** + +The installed source deserves a focused GPU check: + +- history has two attachments; +- the resolve target is created with one attachment; +- the resolve material declares two logical outputs; +- the per-frame copy visibly copies only `resolve.texture` into the first history texture. +- lock history is sampled at the current UV while color history is sampled at the reprojected `historyUV`; +- thin-feature normalization divides by mean luminance without a visible zero-luminance guard. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:160-172`, `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:460-462`, and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:626-627`, `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:727-758`. + +This source structure suggests persistence, reprojection, or dark-pixel stability may not behave as intended, but it is **not evidence that the lock path is definitely broken**. three's material/output handling and backend behavior need a runtime attachment inspection and visual lock test. + +#### Replacement implication + +Local offers explicit reactive-content controls that TAAU does not expose. Whether those controls produce the better image in a given scene still requires GPU comparison. + +Do not copy or design around TAAU's current lock behavior until its second-output persistence is verified. + + +### 5.6 Sharpening and output color domain + +#### Local + +The default temporal output runs RCAS. Each tap is inverse-tonemapped, de-exposed, and +sharpened in the caller's linear/HDR domain before the pass writes `rgba16float`. +Presentation is deliberately outside the upscaler. + +**Pros** + +- Composable linear/HDR output. +- Caller-controlled tone mapping, output color space, and later post-processing. +- Built-in sharpening and optional RCAS denoise. + +**Cons** + +- Direct presentation requires the integration to configure its renderer/output transform. +- Fair resolver comparisons must apply the same presentation transform after each result. + +#### three + +TRAA and TAAU resolve into half-float render targets and return graph textures. The final `RenderPipeline` can apply tone mapping and output conversion later. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:139-172` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:153-216` +- `../node_modules/three/src/renderers/common/RenderPipeline.js:195-225` + +**Pros** + +- Better composability. +- Keeps the temporal result in the linear graph domain. +- Lets one final output transform serve the whole post stack. + +**Cons** + +- Direct presentation requires correct downstream output configuration. +- It does not include a local-equivalent RCAS stage. + +#### Replacement implication + +Linear half-float output is a clear three.js composability advantage. Local's fixed display output is simpler for direct presentation. + +A future local linear/HDR output option would remove one of the largest replacement boundaries without requiring the direct-present mode to disappear. + + +### 5.7 Reset and resize behavior + +#### Local + +Local has an explicit public `resetHistory()` and a per-dispatch `reset` input. Reconfiguration reallocates textures and schedules reset. See `../src/Upscaler.ts:173-205`, `../src/Upscaler.ts:249-258`, and `../src/types.ts:157-160`. + +**Pros** + +- Camera cuts, teleports, cadence gaps, and app-level discontinuities can be explicit. + +**Cons** + +- The caller must use the reset contract correctly. + +#### three + +TRAA and TAAU recreate or resize targets and seed history after dimension changes. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:383-400` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:413-440` + +No equivalent explicit public camera-cut reset is visible in the audited node APIs. + +**Pros** + +- Resize recovery is automatic. + +**Cons** + +- A camera cut at unchanged dimensions has no obvious public history-reset operation. + +#### Replacement implication + +Local is easier to integrate into applications with explicit scene cuts or discontinuous simulation. + + +### 5.8 Debug, tuning, and API surface + +#### Local + +The current API exposes: + +- low-level `Upscaler`; +- high-level `UpscalePass`; +- composable `UpscalerNode` factories; +- runtime settings; +- eight diagnostic views; +- per-compute-pass timing where timestamp queries exist. + +See `../src/types.ts:39-62`, `../src/Upscaler.ts:63-76`, and `../src/internal/GpuTimer.ts:1-24`. + +**Pros** + +- Strong integration diagnostics. +- Multiple ownership models. + +**Cons** + +- More API and more ways to combine incompatible ownership paths. +- Some tuning remains compile-time shader constants. +- Current jitter-default documentation conflicts with code. + +#### three + +TRAA/TAAU expose a smaller set of mutable thresholds and weights directly on node instances. TRAA also exposes subpixel correction. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:88-120` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:98-132` + +**Pros** + +- Compact API. +- Useful tuning values are directly mutable. + +**Cons** + +- No comparable built-in debug-view suite. +- No explicit reactive/exposure/reset API. + + +## 6. Replacement boundaries + +### Correct comparison matrix + +#### 1× input + +Compare: + +- local temporal with `QualityMode.NativeAA`; +- three `TRAANode`. + +Both are same-resolution temporal AA. TAAU may technically operate near 1×, but it is not the primary product comparison for this case. + +#### Reduced-resolution input + +Compare: + +- local temporal at 1.5×, 2×, and 3×; +- three `TAAUNode` at the same input and output dimensions. + +The local preset ratios are defined in `../src/math/resolution.ts:3-13`. + +### When local is a strong replacement + +Local is a strong candidate when the application: + +- is WebGPU-only; +- can provide three-compatible color, depth, and velocity; +- wants Native AA or temporal upscaling under one API; +- needs reactive-mask handling; +- benefits from thin-feature locks or shading-change aging; +- needs explicit reset; +- wants built-in RCAS and direct display output; +- needs reversed-depth or orthographic behavior beyond installed TAAU's visible branches; +- values debug views over graph simplicity. + +### When TRAA/TAAU remain preferable + +TRAA or TAAU may be preferable when the application: + +- wants a native TSL graph texture in linear half-float space; +- needs later HDR effects or caller-owned output transformation; +- wants fewer persistent resources and a simpler resolver; +- wants three's renderer-state save/restore behavior; +- uses logarithmic depth and can choose TRAA at 1×; +- does not need reactive masks, explicit reset, local exposure conditioning, or RCAS; +- wants to avoid raw WebGPU/private-backend integration. + +### What local should consider adopting + +1. **Matrix-transformed previous depth.** Reconstruct previous view position, move it through world space, and compare in current view space. + +2. **Motion-responsive current weighting.** Prototype a current-frame bias based on motion magnitude. + +3. **TRAA subpixel-motion correction.** Test it as an internal experiment, including its documented square-pattern risk. + +4. **Logarithmic-depth support if an actual integration requires it.** + +5. **A linear/HDR output option.** Keep direct display output as an explicit integration mode. + +6. **Renderer-state preservation in imperative helpers.** `UpscalePass.draw()` currently restores MRT and render target to `null`, not necessarily to the caller's previous state. See `../src/UpscalePass.ts:129-156`. + +7. **Mutable internal benchmark thresholds.** Make depth, weighting, and correction parameters adjustable in bench variants before deciding whether any deserve public API. + +### What local should not copy automatically + +- The fixed effective 31-phase schedule. +- A simpler one-sample history path without visual evidence. +- The absence of reactive, exposure, and explicit-reset features. +- TAAU's installed lock persistence behavior before it is verified. +- Source constants merely because they are in three.js; they target a different resolver structure. + + +## 7. Performance comparison + +Everything in this section is a **source-level structural estimate**, not a GPU measurement. + +Texture-operation counts below count visible source sampling/loading expressions. They are not shader ISA counts. Hardware filtering, compiler common-subexpression elimination, cache behavior, format bandwidth, occupancy, and backend scheduling can change actual cost substantially. + +### Local pass structure + +The default temporal path encodes four compute passes: + +1. exposure; +2. reconstruct; +3. accumulate; +4. RCAS, or blit when sharpening is disabled. + +Generated reactivity adds a fifth pass. See `../src/Upscaler.ts:394-553`. + +#### Exposure + +The current exposure pass runs one 8×8 workgroup but only one invocation performs a serial 32×32 loop: + +- 1,024 filtered scene-color samples; +- one previous-exposure load; +- one external-exposure load. + +The 1,024 scene samples occur before the auto/manual/external selection, so they still run when fixed or external exposure is selected. See `../src/shaders/luminancePyramid.ts:42-94`. + +This is a fixed per-frame cost, not a per-output-pixel count. + +#### Reconstruct + +Per render-resolution pixel, the visible source structure is approximately: + +- nine current depth loads; +- one velocity load; +- up to four previous-depth loads for manual bilinear filtering. + +That is about **14 source texture operations per render pixel**, except offscreen reprojections can return before previous-depth sampling. See `../src/shaders/reconstruct.ts:44-57` and `../src/shaders/reconstruct.ts:66-104`. + +#### Accumulate + +With default locks enabled and no reactive mask, the visible source structure is approximately: + +- one motion load; +- one mask load; +- one exposure load; +- nine current-color loads; +- five filtered history samples; +- one lock-history sample. + +That is about **17 source texture operations per output pixel**. An active reactive input adds one more, for about 18. See `../src/shaders/accumulate.ts:131-178`, `../src/shaders/accumulate.ts:191-201`, and `../src/shaders/accumulate.ts:221-250`. + +#### RCAS or blit + +RCAS visibly performs five input loads. Its helper also loads the same exposure texel per tap on the temporal path, giving ten source texture expressions. A compiler may hoist or merge the invariant exposure loads, so **five input plus one-to-five exposure operations** is the honest range. See `../src/shaders/rcas.ts:36-61`. + +Blit has one filtered input sample and one exposure load on the temporal path. See `../src/shaders/blit.ts:30-45`. + +#### Owned textures + +The current temporal allocation owns **13 textures**, excluding caller inputs, scene render targets, sampler, UBO, and timing buffers: + +- one output; +- two history; +- two locks; +- two dilated depth; +- one dilated motion; +- one mask; +- two exposure; +- one reactive dummy; +- one generated-reactive target. + +This count follows `../src/Upscaler.ts:665-720`. Several are tiny or render-resolution resources; counting textures alone does not represent memory cost. The two display-resolution RGBA16F history textures and two display-resolution RGBA16F lock textures dominate this inventory. + +### TAAU structure + +TAAU performs one output-resolution fullscreen resolve, then: + +- copies resolved output color into history; +- copies current input depth into previous-depth storage; +- performs an additional seed render only after resize. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:413-488`. + +The visible resolve is approximately **22 source texture operations per output pixel**: + +- nine current depth loads; +- one velocity load; +- one previous-depth sample; +- nine current beauty loads; +- one history sample; +- one lock-history sample. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:537-579` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:629-755`. + +That estimate does not resolve the lock-attachment concern described earlier. + +### TRAA structure + +At 1×, TRAA's visible resolve is approximately **21 source texture operations per output pixel**: + +- nine current depth loads; +- one velocity load; +- one previous-depth sample; +- one current color sample; +- one history sample; +- eight additional current-neighborhood samples for variance. + +It then copies output color and input depth into history resources. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:498-609` and `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-704`. + +### Structural interpretation + +Local is likely to have higher pass, state, and resource cost than installed TAAU/TRAA. + +That does **not** prove it is slower: + +- local reconstruct runs at render resolution; +- local compute passes may schedule differently from TSL fullscreen draws; +- TAAU/TRAA perform full texture copies; +- filterable samples and explicit loads have different costs; +- local and three currently produce different output domains and feature sets. + +Local `GpuTimer` measures only the compute passes encoded by `Upscaler`. It does not include the scene render, final present, TAAU/TRAA resolve, or their history/depth copies. See `../src/internal/GpuTimer.ts:45-98`. + +Therefore, a local `gpuTimings` total is **not comparable** with an untimed three resolver. + +### Compute ping-pong versus render/copy + +#### Local compute ping-pong + +**Pros** + +- Explicit pass boundaries and timestamp scopes. +- Direct storage writes into history/output resources. +- Flexible formats and debug-output routing. +- No full output-to-history color copy after accumulation; accumulation writes the next history directly. + +**Cons** + +- More bind groups, dispatches, barriers, and owned intermediate textures. +- Depends on raw WebGPU access through three internals. +- Fixed storage output format constrains composability. + +#### three render/copy path + +**Pros** + +- Fits TSL and renderer resource management. +- Produces linear half-float graph textures. +- Simpler visible pass graph. +- Uses renderer state save/restore. + +**Cons** + +- Fullscreen resolve plus full texture copies. +- Copy cost is easy to omit accidentally from profiling. +- Scalar pipeline hooks and global velocity state complicate nesting. + +### Fair-comparison requirement + +Output transforms must be normalized: + +- same linear/HDR input; +- same exposure; +- same tone map; +- same output transfer function; +- same sharpening policy; +- same target format where practical. + +Without that normalization, both visual quality and performance results mix resolver behavior with output processing. + + +## 8. Other screen-space nodes and temporal ownership + +Projection jitter and effect noise should not be conflated. + +### SSGI + +`SSGINode.useTemporalFiltering` changes ray direction and initial offsets from `frame.frameId`. It does not own camera jitter and does not store or reproject its own image history. See `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:181-193` and `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:352-382`. + +Its shader combines: + +- interleaved gradient noise; +- a hash-like `rand`; +- six temporal rotations; +- four spatial offsets. + +See `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:7-9` and `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:576-610`. + +The installed documentation says temporal filtering expects TRAA, but the local resolver can be the one final temporal resolver if the SSGI signal is composited into its input. + +### GTAO + +GTAO optionally rotates its sampling directions by frame ID. It does not move the camera or maintain image history. See `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:157-169` and `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:265-288`. + +Its spatial pattern comes from a generated magic-square texture, not conventional blue noise. See `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:171-177` and `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:487-521`. + +### SSR + +Installed SSR defaults `stochastic` to `false`. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:15-25` and `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:52-76`. + +The repository's examples 06 and 09 do not enable stochastic SSR. See: + +- `../examples/06-screenspace-gi/main.ts:165-173` +- `../examples/09-kitchen-sink/main.ts:140-147` + +When stochastic mode is enabled, SSR advances an independent noise index and uses analytic R² noise. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:737-760` and `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:863-892`. + +SSR has optional multibounce history through `setHistory()`, but the audited examples do not wire it. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:661-678`. + +### DenoiseNode + +`DenoiseNode` is spatial. It uses a 16-sample rotating kernel and a generated simplex-noise texture, but it owns no temporal image history and does not projection-jitter the camera. See `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:118-132`, `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:184-239`, and `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:284-319`. + +### RecurrentDenoiseNode and TemporalReprojectNode + +`RecurrentDenoiseNode` can temporally accumulate a reprojected input and uses analytic R² noise to rotate its spatial kernel. See `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js:321-386` and `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js:466-518`. + +`TemporalReprojectNode` owns history or accepts external history. It explicitly states that it does not apply camera subpixel jitter; it reprojects with motion and camera matrices. See `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:489-518` and `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:535-609`. + +Neither node moves the camera projection. Their temporal state can still overlap with the final resolver's history, and `TemporalReprojectNode` still participates in velocity-hook ownership. + +It still installs render-pipeline hooks to bind and later clear the shared velocity projection. That ownership collision is discussed next. + +### Noise taxonomy + +No conventional blue-noise texture is used by these audited paths. + +The distinct mechanisms are: + +- **IGN/hash:** SSGI interleaved gradient noise and `rand`. +- **Magic square:** GTAO's tiled direction texture. +- **Simplex:** `DenoiseNode`'s generated noise texture. +- **Analytic R²:** SSR stochastic sampling and recurrent-denoise rotation through `RNoise`. + +See `../node_modules/three/examples/jsm/tsl/utils/RNoise.js:3-49`. + +Sharing temporal identity can coordinate when these patterns advance. It does not imply replacing all of them with the upscaler's Halton samples. + + +## 9. Hook and velocity collision + +### Scalar render-pipeline hooks + +`RenderPipeline` stores one `onBeforeRenderPipeline` callback and one `onAfterRenderPipeline` callback. They are scalar properties, not callback lists. See `../node_modules/three/src/renderers/common/RenderPipeline.js:195-204`. + +The pipeline calls each scalar once around its graph render. See `../node_modules/three/src/renderers/common/RenderPipeline.js:121-150`. + +These nodes assign those scalar properties directly: + +- local `UpscalerNode`: `../src/UpscalerNode.ts:206-217`; +- TRAA: `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:443-464`; +- TAAU: `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:502-527`; +- `TemporalReprojectNode`: `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:728-748`. + +If more than one is nested in a graph, setup order can overwrite an earlier callback. + +### Global mutable velocity + +three exports one shared singleton named `velocity`, created through `nodeImmutable()`. The wrapper is shared, while its internal `projectionMatrix` remains mutable through `setProjectionMatrix()`. See `../node_modules/three/src/nodes/accessors/VelocityNode.js:24-98` and `../node_modules/three/src/nodes/accessors/VelocityNode.js:218-224`. + +When both lifecycles are coordinated, the required frame ordering is: + +1. capture the stable, unjittered camera projection; +2. bind that projection for velocity generation; +3. apply projection jitter to the scene camera; +4. render all velocity/color/depth inputs; +5. clear camera jitter; +6. release or restore velocity ownership only after all dependent passes are done. + +Another node must not replace or clear the stable velocity projection between steps 2 and 4. + +### Example 10 risk + +Example 10 can nest `TemporalReprojectNode` inside the local `UpscalerNode` graph. See `../examples/10-ssgi-denoise/main.ts:161-192` and `../examples/10-ssgi-denoise/main.ts:213-221`. + +Both nodes assign the same scalar hooks. `UpscalerNode` uses them for camera-jitter begin/end, while `TemporalReprojectNode` uses them to bind and clear the shared velocity projection. + +Setup order deterministically leaves only the last callback pair installed. Depending on build order, either the local camera-jitter lifecycle or `TemporalReprojectNode`'s velocity-projection lifecycle may not run as intended. `TemporalReprojectNode` also has a one-time `updateBefore()` fallback, so the first frame may differ from steady state. + +The exact runtime consequence is an **unverified, source-derived integration risk**, not a confirmed failure from this audit. The example's documented GPU-observed quality conflict concerns stacked temporal accumulation under FSR jitter; it does not prove this hook collision itself occurred. See `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:630-659`, `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:728-748`, and `../examples/10-ssgi-denoise/main.ts:32-52`. + + +## 10. Shared temporal sampling options + +### A. Application/render-pipeline-owned `TemporalSamplingContext` + +The application or render pipeline owns frame identity, projection jitter, reset generation, and ordered begin/end behavior. Resolvers and effects consume the context. + +**Pros** + +- Avoids direct SSGI/SSR dependency on one upscaler. +- Supports local FSR, TRAA, TAAU, or a future resolver as alternatives. +- Creates one authoritative projection-jitter owner. +- Coordinates reset and rendered cadence across effects. +- Provides a natural place to compose hooks. + +**Cons** + +- Requires integration work above individual nodes. +- Needs careful behavior when a pass renders less often than the display loop. +- Public API should wait until the bench proves the contract. + +**Recommendation:** preferred direction to prototype before considering a public API. + +### B. Upscaler-owned provider + +The local upscaler publishes jitter and frame state for SSGI/SSR/denoisers. + +**Pros** + +- Smaller immediate change. +- Reuses data the upscaler already computes. + +**Cons** + +- Couples unrelated effects to one resolver. +- Makes TRAA/TAAU substitution harder. +- Encourages the upscaler to own application scheduling. +- Risks turning implementation details into public API too early. + +**Recommendation:** not preferred. + +### C. Independent noise, one projection-jitter owner + +Keep SSGI/GTAO/SSR/denoiser noise schedules independent. Ensure only the final resolver moves the camera. + +**Pros** + +- Minimal change. +- Preserves each shader's optimized distribution. +- Avoids double projection jitter. + +**Cons** + +- Reset and frame cadence can drift. +- Debugging cross-pass temporal behavior remains harder. +- Effects may advance noise on frames where their output was not rendered. + +**Recommendation:** acceptable short-term state. + +### Conceptual context fields + +A future context should be able to provide: + +- `frameId` and `sampleIndex`; +- current and previous jitter; +- stable unjittered projection; +- current and previous jittered projections; +- render and display dimensions; +- history/reset generation token; +- whether jitter was actually applied for this render; +- `beginFrame()` and `endFrame()` lifecycle boundaries. + +The context should synchronize **identity, reset, and cadence**. It should not require every shader to derive its samples from one distribution. + + +## 11. Integration design + +This is a recommendation, not current API. + +### 11.1 External-jitter mode for `Upscaler` + +Add an external mode in which: + +- the coordinator passes exact current and previous offsets to dispatch; +- offsets are explicitly defined in render-pixel units and top-left convention; +- the upscaler does not internally advance `JitterSequence`; +- `beginFrame()` does not independently move the camera; +- a reset-generation token invalidates history exactly once. + +The internal schedule remains the default for existing imperative users. + +### 11.2 Ordered hook coordinator + +Replace scalar callback assignment with ordered participants: + +1. capture stable camera state; +2. prepare velocity providers; +3. apply projection jitter; +4. render graph dependencies; +5. clear projection jitter; +6. finalize temporal state. + +The coordinator can adapt to three's scalar hook by installing one callback that runs an ordered list. + +### 11.3 Dedicated velocity projection provider + +Do not rely only on the exported global `velocity` singleton. + +Provide an explicit velocity projection/provider per render pipeline or camera so nested temporal effects cannot clear one another's state accidentally. + +### 11.4 Optional identity for noisy effects + +SSGI, GTAO, SSR, and denoisers may receive: + +- sample/frame identity; +- reset generation; +- whether this effect actually rendered this frame. + +They should retain their own IGN, magic-square, simplex, R², or domain-specific distributions. + +### 11.5 TemporalReproject integration + +Allow `TemporalReprojectNode` to consume a stable projection provider and disable its own before/after hook ownership. + +It can still own effect history without becoming a projection-jitter owner. + +### 11.6 TRAA/TAAU diagnostics + +TRAA/TAAU may optionally consume externally supplied jitter in bench-only diagnostic variants. + +They remain **alternative final resolvers**. They must never run upstream of local FSR in a product comparison. + +### 11.7 What must synchronize + +Synchronize: + +- frame identity; +- reset generation; +- rendered cadence; +- current/previous projection state; +- exactly one camera-jitter application. + +Do not automatically synchronize every noise distribution. + + +## 12. Benchmark plan + +Use `bench` as the canonical timing harness. It already controls render/display resolution, shares one display transform across its current modes, and surfaces local timestamp results. See `./src/BenchPipeline.ts:75-161` and `./src/main.ts:86-120`. + +Use examples as visual fixtures: + +- `../examples/04-aliasing-torture`: thin geometry, moiré, slow camera motion. +- `../examples/05-transparency`: transparents, particles, reactive-mask behavior. +- `../examples/06-screenspace-gi`: reduced-resolution GTAO/SSR/SSGI feeding the raw upscaler. +- `../examples/09-kitchen-sink`: in-graph SSGI/SSR composition and local node jitter. +- `../examples/10-ssgi-denoise`: stacked effect-history experiments and hook-collision characterization. + +### 12.1 Two comparison classes + +#### Stock-product comparison + +Preserve: + +- each resolver's stock jitter schedule; +- stock reconstruction and history filtering; +- stock public defaults; +- one resolver per run. + +Normalize only the surrounding application conditions needed for a fair output. + +#### Normalized diagnostic comparison + +Allow controlled substitutions such as: + +- identical injected jitter sequence; +- identical current-frame input; +- identical output transform; +- sharpening disabled on both, or equivalent sharpening added to both; +- internal parameter sweeps. + +Label these results as diagnostics, not stock product behavior. + +### 12.2 Modes + +At 1×: + +- local Native AA; +- three TRAA. + +At reduced resolution: + +- local temporal at 1.5×; +- local temporal at 2×; +- local temporal at 3×; +- three TAAU at the same three ratios. + +Do not put TRAA in the reduced-resolution result set as if it were an upscaler. + +### 12.3 Deterministic setup + +For each capture: + +- fixed canvas physical dimensions and DPR; +- fixed camera path and animation time; +- fixed scene random seeds; +- fixed effect settings; +- fixed renderer features; +- MSAA off; +- one final temporal resolver; +- clean history at the start of each block; +- fixed warm-up frame count; +- fixed measured frame count; +- no UI interaction during the sample window. + +Record: + +- browser and version; +- OS; +- adapter name; +- WebGPU backend; +- three version; +- commit SHA; +- display and render dimensions; +- timestamp-query availability; +- power/thermal state where practical. + +Local can use its explicit reset API. TRAA/TAAU do not expose an equivalent public camera-cut reset, so recreate their node/graph for each capture block or document a deliberately forced resize/reseed procedure. + +### 12.4 Output normalization + +For visual comparison: + +- feed the same linear HDR scene; +- use the same tone map and sRGB conversion; +- compare the same output format where possible; +- disable local RCAS or add a matched sharpen after TAAU; +- keep exposure behavior equivalent; +- capture before browser scaling. + +Keep a separate product-default gallery showing local RCAS/direct output versus three's normal graph output. Do not mix that gallery with resolver-isolation conclusions. + +### 12.5 Timing + +Measure at least: + +- scene/G-buffer render; +- resolver work; +- history/depth copies; +- output transform; +- sharpening; +- final present where relevant; +- whole-frame GPU time. + +Local `GpuTimer` alone is insufficient because it covers only local compute passes. Instrument three's resolve and copies with compatible GPU timestamps, or use a GPU trace that includes both implementations. + +If per-copy timestamping is unavailable, use labeled command scopes or frame captures and report the limitation explicitly. + +### 12.6 Sampling method + +- Warm all pipelines and histories before measured sequences. +- Use repeated A/B/B/A or **ABBA** ordering to reduce thermal and drift bias. +- Report median and p95, not only average. +- Keep raw frame samples. +- Repeat enough blocks to show run-to-run spread. +- Treat CPU FPS as supporting information, not a substitute for GPU timing. + +### 12.7 Scene scenarios + +Include: + +- static camera and static scene for convergence; +- slow camera pan; +- fast camera translation; +- rotating camera; +- independently moving foreground object; +- newly revealed background/disocclusion; +- thin fences and wires; +- high-frequency floor or foliage; +- bright HDR highlights; +- transparent/additive particles; +- noisy SSGI; +- glossy stochastic SSR if enabled; +- camera cut and explicit reset; +- resize and history seed. + +### 12.8 Acceptance criteria + +A benchmark configuration is valid only when: + +- exactly one projection-jitter owner is active; +- velocity is generated from the stable projection; +- color, depth, velocity, and reactive inputs are aligned; +- history is clean at test start, using explicit local reset or three node/graph recreation; +- output transforms are documented and matched; +- no WebGPU validation errors occur; +- TAAU lock persistence has been independently characterized; +- hook assignment and clearing order are logged or inspected; +- timings include equivalent work or clearly identify exclusions. + +Quality review should record: + +- static convergence; +- edge shimmer; +- detail retention; +- motion blur; +- ghost trails; +- disocclusion recovery; +- thin-feature stability; +- transparency behavior; +- SSGI/SSR noise convergence; +- camera-cut recovery. + +No winner should be declared from source operation counts alone. + + +## 13. Recommended next actions + +### 1. Fix or document integration inconsistencies before benchmarking + +- Correct the `../src/types.ts:91-111` jitter-default text to match current `UpscalerNode` behavior, or intentionally change the behavior and document that decision. +- Verify installed TAAU lock persistence on GPU. +- Characterize scalar hook overwrite and velocity clearing in example 10. + +### 2. Add a bench-only TRAA/TAAU comparator + +- Instantiate one final resolver at a time. +- Use identical scene inputs and output transform. +- Keep TRAA at 1× and TAAU at reduced resolution. +- Preserve stock schedules for the product test. + +### 3. Harden timing and capture + +- Time three resolve and copy work, not only local compute. +- Add deterministic camera/animation scripts. +- Record environment metadata. +- Store raw median/p95 samples and screenshots. + +### 4. Prototype selected local algorithm variants + +Behind internal or bench-only flags: + +- matrix-transformed previous depth; +- motion-responsive current weighting; +- TRAA-style subpixel correction. + +Do not expose public tuning until the variants have visual and performance evidence. + +### 5. Prototype `TemporalSamplingContext` internally + +Build it in bench/internal scope with: + +- one projection-jitter owner; +- ordered hook composition; +- explicit stable velocity state; +- frame/reset generation; +- rendered cadence. + +Avoid committing to public API shape yet. + +### 6. Feed optional temporal identity to one noisy node + +Start with SSGI: + +- share frame/sample identity and reset generation; +- preserve SSGI's own ray distribution; +- verify that skipped renders do not advance its pattern incorrectly. + +### 7. Decide public API after evidence + +Only then decide whether to expose: + +- external jitter; +- linear/HDR output; +- a temporal context; +- velocity providers; +- benchmark-proven tuning controls. + +### Direct answer: is shared jitter the right move? + +**Yes** to exactly one projection-jitter owner. + +**No** to forcing SSGI, SSR, GTAO, or denoiser noise to derive from the FSR Halton sequence. + +Shared frame/reset identity is a reasonable prototype for coordinating rendered cadence and history invalidation, but its benefit still needs to be demonstrated. Each shader should keep the sampling distribution suited to its own problem. diff --git a/bench/results/.gitignore b/bench/results/.gitignore new file mode 100644 index 0000000..8524d3f --- /dev/null +++ b/bench/results/.gitignore @@ -0,0 +1,4 @@ +raw/ +*.png +*.csv +*.heatmap.png diff --git a/bench/results/README.md b/bench/results/README.md new file mode 100644 index 0000000..b006ab3 --- /dev/null +++ b/bench/results/README.md @@ -0,0 +1,66 @@ +# Benchmark Results + +E00 artifacts are produced by `npm run bench:run` and `npm run bench:capture`. +The immutable experiment contract remains in `experiments/e00-harness.json`. + +## Layout + +- `raw/E00//run.json` records arguments, the manifest SHA-256, the + baseline Git SHA, and a digest of every tracked or untracked non-ignored file + in the working tree. +- Performance runs emit one JSON and CSV file per ratio, ABBA repetition, and + block position. JSON retains every fresh frame-tagged sample, missing counts, + per-pass median/p95, and compute-sum median/p95. +- Capture runs emit canvas-only lossless PNGs. Filenames identify blinded + label, variant, reload, scenario, subrun, ratio, frame, and debug view. +- `capture-analysis.json` records all numerical reload pairs. Passing pairs stay + in the aggregate file; failed pairs additionally emit standalone metrics and + difference heatmaps. +- `review.html` presents the sampled A/B pairs without variant names, outlines + every declared ROI, stores progress locally, and exports + `rubric-reviewed.json`. Review state is isolated by manifest, working-tree, + and ordered-record digests. Review-only validation rejects stale contracts, + changed source trees, incomplete capture protocols, and failed numerical + analyses before accepting grades. Validate an export without recapturing: + + ```bash + node scripts/run-benchmark.mjs \ + --review-only \ + --output bench/results/raw/E00/ \ + --rubric /path/to/rubric-reviewed.json + ``` + +- Each run retains the required browser log channels. Any validation error, + uncaught exception, or reported device loss invalidates the run. + +Raw output is ignored by Git. Promote only controller-reviewed summaries or +small diagnostic artifacts under a separately authorized manifest. + +The authored source-style bundles do not have result manifests yet. Their hypotheses, +cumulative ordering, and required later evidence are documented in +`bench/docs/PARITY-CANDIDATES.md`; do not store interactive or smoke output as adoption +evidence. + +Q6-Q8 construct reduced-resolution three.js effect graphs for GTAO, SSR, SSGI, +the spatial denoiser, and the recurrent denoiser. Their readiness, source-owned +sampling state, node-frame maps, velocity history, and effect history are reset +before recorded frame zero as specified by the E00 manifest. + +## Focused RCAS comparison + +Run the practical E01 comparison with: + +```bash +npm run bench:compare:rcas +``` + +It captures legacy RCAS versus the FSR 3.1.5 lower limiter, then the lower +limiter versus the temporal denoise default. It also records short ratio-2 +timing blocks, writes a summary report, starts a local review server, and opens +the report in the default browser. Press Ctrl+C when review is complete. + +To reopen an existing report without rerunning the GPU work: + +```bash +npm run bench:compare:rcas -- --reuse bench/results/raw/E01/ +``` diff --git a/bench/results/experiments/README.md b/bench/results/experiments/README.md new file mode 100644 index 0000000..b229c71 --- /dev/null +++ b/bench/results/experiments/README.md @@ -0,0 +1,120 @@ +# Parity Experiment Manifests + +## Purpose + +Each JSON file is the immutable execution contract for one parity experiment. It fixes the hypothesis, comparison, write scope, evidence, validation gates, and rollback conditions before implementation starts. + +Once a manifest reaches `implementing`, agents must not edit it. If the contract is wrong or incomplete, the controller closes that run and creates a new manifest revision; an implementer cannot reinterpret or extend the existing contract. + +## Immutable Manifest Schema + +Every manifest is valid JSON and contains all of these fields: + +- `schema_version`: manifest schema identifier, currently `1`. +- `experiment`: object with immutable `id`, `name`, `phase`, `task`, and `status`. +- `purpose`: bounded reason for running the experiment. +- `hypothesis`: falsifiable expected outcome. +- `comparison`: object with `class`, `baseline_variant`, `candidate_variant`, and `single_variable`. +- `source`: pinned reference name, version, commit, and relevant reference behavior. +- `baseline`: full local SHA, short SHA, branch, and recorded verification evidence. +- `dependencies`: experiment IDs that must already be adopted and documented. +- `target_artifact`: exact visual, correctness, or performance behavior the candidate is intended to improve or preserve. +- `known_regression_risks`: explicit quality, correctness, performance, platform, and maintenance risks evaluated by the experiment. +- `per_pass_and_compute_timing_budget`: numerical median/p95 limits for every affected pass and the total compute sum, including the declared noise-floor rule. +- `required_platform_divergence_evidence`: required platform reason, affected scenarios, visual impact, median/p95 impact, and maintenance cost. +- `scope`: exact repository-relative `allowed_write_paths` and explicit `forbidden_changes`. +- `commands`: required static and real-device commands, in execution order where order matters. +- `scenarios`: exact scenario IDs, ratios, dimensions, timestep, MSAA state, and required debug views. +- `capture_protocol`: exact absolute and event-relative frame captures, numerical image-equivalence rules, artifact rubric, and pass/fail aggregation. +- `validation_gates`: static, runtime, visual, timing, and review requirements. +- `timing_protocol`: feature requirements, warm-up, sample count, block order/repetitions, isolation, statistics, metadata, and invalidation rules. +- `expected_behavior`: output domain, presentation transform, jitter owner, velocity convention, reset behavior, resource graph, and binding expectations. +- `required_evidence`: complete evidence record required before a decision. +- `rollback_conditions`: conditions that immediately invalidate or revert the candidate. +- `result_contract`: allowed decision values and the evidence required to set one. +- `report_contract`: required agent status and handoff fields. + +Arrays of file paths are literal allowlists, not examples or directory prefixes. A listed directory never authorizes unlisted descendants. Repository-relative paths are resolved from `/Users/dex/Developer/fsr3`. + +Deterministic effect scenarios must define their renderer/device/pipeline/resource readiness barrier before recorded frame zero. Unrecorded readiness work may compile and allocate, but its temporal history must be cleared or source-defined reseeding must occur on recorded frame zero. A source-owned sampling sequence must cite the exact installed source/version and state its frame-index, rotation, offset, and noise formulas; a pinned private bridge requires a runtime shape guard and cannot be presented as public API. + +Capture thresholds are immutable manifest inputs. Measured repeated-baseline behavior is evidence about the environment and may block a run, but it cannot widen max-absolute, RMSE, dimension, alpha, rubric, or aggregation limits. + +## Result Decisions + +The controller records exactly one decision after the evidence gates: + +- `adopt`: candidate meets its quality, correctness, performance, and review gates. +- `iterate`: evidence supports another explicitly remanifested candidate; the current run is not adopted. +- `retain-local`: the local baseline remains preferable on measured quality, performance, or maintainability. +- `platform-divergence`: source parity is unavailable or materially worse because of a documented web/WebGPU constraint, and the measured local alternative is retained. +- `blocked`: the experiment cannot reach a valid decision because required capability, dependency, environment, or evidence is unavailable. + +An implementation status such as `PASS` is not an adoption decision. Only the controller may set a result decision. + +## Required Evidence Fields + +Every result record must include: + +- manifest ID, contract revision, immutable manifest digest, and working-tree digest; +- baseline and candidate IDs plus source and local SHAs; +- exact changed files and final diff summary; +- commands, exit codes, test counts, and timestamps; +- browser, browser version, operating system, adapter, backend, WebGPU features, three version, dimensions, DPR, ratio, settings, and shader/pipeline keys; +- validation logs covering WGSL compilation, WebGPU errors, uncaught exceptions, and device loss; +- scenario, frame, debug view, blinded A/B label, ROI, and artifact-rubric records for every capture; +- raw fresh GPU samples with frame tags, missing-sample count, per-pass and compute-sum median/p95, ABBA block results, and measured noise floor; +- reset, jitter-owner, active-resolver, output-domain, resource-graph, and binding checks; +- reviewer findings, fix-round count, unresolved concerns, decision, and decision rationale; +- platform reason and unsupported or degraded capability; +- affected scenario IDs, ratios, frames, debug views, and platforms; +- visual impact with blinded captures, ROIs, rubric grades, max-absolute error, and RMSE; +- per-pass and compute-sum median/p95 impact with raw samples and the measured noise floor; +- maintenance cost covering extra code paths, feature detection, tests, validation matrix, and likely upkeep for `platform-divergence`. + +Evidence must describe failures and inconclusive measurements as recorded. Missing evidence cannot be inferred from screenshots, FPS, a successful build, or an agent assertion. + +## Controller Responsibilities + +The controller: + +1. Authors and freezes the manifest before assigning implementation. +2. Confirms prerequisite decisions and the clean baseline. +3. Gives each agent the manifest and its exact file allowlist. +4. Rejects any unlisted write and classifies it as `SCOPE_BLOCKED`. +5. Serializes all implementation agents whose allowlists overlap by one or more exact paths. +6. Verifies changed paths and static commands before GPU work. +7. Runs or assigns real-device validation and evidence collection. +8. Assigns a fresh read-only task review. +9. Consolidates critical and important findings into no more than two fix rounds. +10. Makes the result decision and updates `bench/docs/PARITY-PROGRESS.md`. +11. Starts dependent work only from an adopted, documented integration state. + +The controller may narrow a task without changing its manifest. Any expansion requires a new manifest revision before work resumes. + +Parallel work is permitted only when every participant is read-only, independent, and has no shared mutable artifact or integration state. Implementation agents with overlapping allowlists are never concurrent, even if they intend to edit different paths from those lists. + +## Agent Responsibilities + +Implementers: + +- write only exact paths in `scope.allowed_write_paths`; +- make only the change described by the hypothesis and `single_variable`; +- stop with `SCOPE_BLOCKED` before touching any additional path; +- do not edit manifests, the progress ledger, unrelated algorithms, public APIs, dependencies, CI, or documentation unless each exact path is allowed; +- run required commands that are available in their assigned environment; +- return only the required report fields with complete artifacts, gate results, evidence, and concerns. + +Reviewers and research agents remain read-only. Validation agents may create artifacts only when their exact artifact paths are listed in a separate manifest. No agent may expand scope, silently fix adjacent issues, commit, create a branch/worktree, or run baseline and candidate simultaneously during authoritative timing. + +## Handoff Status + +Every agent handoff uses exactly one of: + +`PASS | FAIL | BLOCKED | SCOPE_BLOCKED | USER_DECISION_REQUIRED` + +It also includes: + +`changed_files | commands | artifacts | gates | concerns` + +`PASS` means the assigned task and its available gates passed; it does not mean the candidate is adopted. diff --git a/bench/results/experiments/e00-harness.json b/bench/results/experiments/e00-harness.json new file mode 100644 index 0000000..a36b94e --- /dev/null +++ b/bench/results/experiments/e00-harness.json @@ -0,0 +1,1878 @@ +{ + "schema_version": 1, + "experiment": { + "id": "E00", + "name": "Harness Foundation", + "phase": 1, + "task": "Tasks 1.1 through 1.6", + "status": "implementing", + "fix_round": 2, + "fix_round_limit_reached": true, + "contract_revision": 4, + "contract_correction": "Controller redesign 4 binds every run and review to one manifest plus working-tree digest, isolates review persistence, closes user-owned CDP targets, and finalizes E00 timing acceptance from measured baseline variance." + }, + "purpose": "Build and accept the deterministic, single-variant benchmark foundation required before any FSR parity algorithm experiment begins.", + "hypothesis": "The local baseline can run through a deterministic one-variant harness with frame-exact captures, fresh timestamp-query samples, and reproducible reset behavior while preserving its shader algorithms and output. Baseline-versus-baseline ABBA runs will establish a usable machine and browser timing noise floor.", + "comparison": { + "class": "infrastructure-equivalence", + "baseline_variant": "local-baseline-5d6a65e", + "candidate_variant": "local-baseline-through-e00-harness", + "single_variable": "Execution, capture, validation, and timing infrastructure; resolver shader algorithms, pass math, resource formats, and public behavior remain unchanged." + }, + "source": { + "name": "FidelityFX SDK FSR Upscaler", + "version": "3.1.5", + "commit": "60f4ea81909200d8542eca14dccb2628b763a9a3", + "reference_behavior": "Source behavior is pinned for later parity experiments. E00 does not port or authorize any source shader algorithm." + }, + "baseline": { + "sha": "5d6a65e5681e5e95590f3e9a11ce75e43354ca13", + "short_sha": "5d6a65e", + "branch": "feat-match-fsr3", + "verification": { + "command": "npm test", + "result": "passed", + "tests_passed": 54, + "tests_failed": 0 + } + }, + "dependencies": [], + "target_artifact": { + "name": "Deterministic baseline-equivalence evidence", + "description": "Frame-exact baseline captures, complete validation logs, and fresh frame-tagged GPU timing distributions produced by one active resolver without changing baseline image behavior.", + "success_signal": "Five-reload capture sets pass the numerical equivalence protocol, every deterministic scenario and debug gate passes, and four ABBA repetitions establish an acceptable timing noise floor at every required ratio." + }, + "known_regression_risks": [ + "Harness orchestration changes frame order, projection jitter ownership, velocity generation, or history reset timing.", + "Wall-clock, OrbitControls, resize handlers, random values, asynchronous readback order, or upstream effects make supposedly identical runs non-deterministic.", + "Capture compares different dimensions, transfer domains, alpha handling, presentation transforms, frames, or debug views.", + "Timing collection returns stale, missing, duplicated, or untagged samples or changes the measured GPU graph.", + "Variant registration instantiates two resolvers, compiles inactive candidates into the measured graph, or leaves stale timing labels.", + "Automated reset fails to clear temporal, exposure-adaptation, scenario-event, PRNG, jitter, and pending-readback state.", + "Q5 through Q8 fixture integration changes production shader algorithms instead of supplying deterministic upstream inputs.", + "A browser or adapter capability difference is mislabeled as algorithm evidence without the required platform-divergence record." + ], + "per_pass_and_compute_timing_budget": { + "scope": "Every existing GPU pass label and the compute sum at ratios 1, 1.5, 2, and 3.", + "median_relative_delta_limit": "max(3%, 2 * N_median)", + "p95_relative_delta_limit": "max(5%, 2 * N_p95)", + "per_pass_rule": "For each pass label, compare the mean of the two A run statistics with the mean of the two B run statistics inside each ABBA repetition. A pass supports an individual performance claim only when its own N_median and N_p95 meet the declared noise limits; otherwise retain its samples in compute sum, mark the pass ineligible, and require a later pass-specific microbenchmark before making a claim.", + "compute_sum_rule": "Apply the same median and p95 limits independently to the sum of all resolver compute-pass samples in at least three of four repetitions.", + "new_gpu_work": "No additional production GPU pass or dispatch is permitted in E00. Instrumentation query resolve and readback work is excluded from compute sum but reported separately.", + "noise_floor_acceptance": { + "N_median_max": 0.015, + "N_p95_max": 0.025, + "unit": "fractional relative delta", + "failure": "E00 is blocked if any required ratio's compute-sum noise floor exceeds either maximum after one clean rerun. A per-pass failure does not block aggregate E00 acceptance, but it forbids an individual claim for that pass until a later microbenchmark establishes a usable floor." + } + }, + "required_platform_divergence_evidence": { + "platform_reason": "Identify the exact browser, WebGPU, backend, adapter, driver, feature, limit, validation rule, or timestamp-query behavior preventing parity and include reproducible logs.", + "affected_scenarios": "List every affected platform, scenario ID, ratio, frame, debug view, and ROI; state which unaffected controls still pass.", + "visual_impact": "Provide blinded baseline/candidate captures and numerical max-absolute, RMSE, alpha, dimension, and artifact-rubric results for each affected ROI.", + "median_p95_impact": "Provide raw fresh samples, missing counts, per-pass and compute-sum median/p95 deltas, ABBA repetition results, and the independently derived noise floor.", + "maintenance_cost": "Record additional code paths, capability checks, tests, artifact fixtures, cross-platform validation obligations, production-bundle cost, and expected future upkeep." + }, + "scope": { + "allowed_write_paths": [ + "bench/src/types/benchmark.d.ts", + "bench/src/benchmark/config.ts", + "bench/src/benchmark/variants.ts", + "bench/src/benchmark/BenchmarkResolver.ts", + "bench/src/BenchPipeline.ts", + "src/internal/ComputePass.ts", + "src/shaders/shaders.test.ts", + "bench/src/benchmark/clock.ts", + "bench/src/benchmark/scenarios.ts", + "bench/src/BenchScene.ts", + "bench/src/main.ts", + "src/internal/GpuTimer.ts", + "bench/src/benchmark/collector.ts", + "bench/src/benchmark/api.ts", + "bench/src/benchmark/environment.ts", + "scripts/benchmark-contract.mjs", + "scripts/benchmark-contract.test.mjs", + "scripts/run-benchmark.mjs", + "bench/results/README.md", + "bench/results/.gitignore", + "package.json" + ], + "forbidden_changes": [ + "Any write to a path not exactly listed in allowed_write_paths.", + "Any parity shader algorithm, filter, sampling kernel, accumulation, rectification, lock, exposure, reactive, depth, RCAS, EASU, color-domain, or presentation-order change.", + "Any public API or public default change.", + "Any production resource format, binding layout, bind-group order, pass graph, output-domain, or dispatch behavior change except passing unchanged baseline behavior through the benchmark resolver adapter.", + "Any dependency addition or package-manager lockfile change.", + "Any GPU-dependent CI test or CI workflow change.", + "Any replacement of raw WGSL or the existing three WebGPU bridge with TSL.", + "Any edit to the parity plan, this manifest, or bench/PARITY-PROGRESS.md by an implementer.", + "Any commit, branch, worktree, stash, or destructive Git operation." + ], + "implementation_serialization": { + "rule": "All implementation agents whose immutable allowed_write_paths overlap by one or more exact paths execute serially.", + "handoff": "The controller verifies the prior writer is complete or stopped and records its changed paths before assigning the next overlapping writer.", + "parallel_exception": "Parallel participants must be independent and read-only, with no shared mutable artifact, integration state, or write permission." + } + }, + "commands": { + "static": [ + "npm run lint", + "npm run typecheck", + "npm test", + "npm run build" + ], + "real_device": [ + "npm run dev -- --host 127.0.0.1", + "npm run bench:capture -- --experiment E00 --variant baseline --comparison baseline --ratios 1,1.5,2,3", + "npm run bench:run -- --experiment E00 --variant baseline --comparison baseline --ratios 1,1.5,2,3 --blocks 4 --warmup 240 --samples 600" + ], + "required_log_channels": [ + "Log.entryAdded", + "Runtime.consoleAPICalled", + "Runtime.exceptionThrown" + ] + }, + "scenarios": { + "output": { + "width": 1920, + "height": 1080, + "device_pixel_ratio": 1, + "timestep_seconds": 0.016666666666666666, + "msaa": false + }, + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "formula_conventions": { + "frame": "n is the zero-based integer frame index.", + "time": "t = n / 60 seconds; no wall-clock value is read.", + "look_at": "Camera orientation is the right-handed three.js lookAt quaternion for the stated position, target, and up vector [0,1,0], recomputed from absolute values each frame.", + "projection": "Perspective fields are vertical field of view in degrees, aspect width/height, near, and far. Projection jitter is applied later by the sole active upscaler and is not part of these unjittered camera formulas.", + "baseline_scene_formula": { + "knots": "For knot i in {0,1,2}: position=(-6+6*i,2.2,-4), rotationEulerXYZ=(0.35*t+i,0.5*t,0).", + "spheres": "For sphere i in {0,1,2,3}, a=0.9*t+i*pi/2 and position=(5.5*cos(a),1.1+0.4*sin(2*t+i),5.5*sin(a)); rotation is fixed identity.", + "floor_fence_bulb": "Floor, 60 pickets, and bulb retain the fixed transforms created by BenchScene: floor rotationX=-pi/2; picket i position=(-12+0.4*i,1.2,3.5); bulb position=(0,5.5,-2).", + "lighting": "Directional light color 0xfff2df, intensity 3.2, position (8,14,6); hemisphere sky 0x9fb4d4, ground 0x2a2620, intensity 0.9.", + "exposure": "settings.exposure=1.0 and autoExposure=true unless the scenario states otherwise." + }, + "baseline_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "room_scene_formula": { + "camera_target": [ + 0, + 3, + -5 + ], + "geometry": "Fixed example-06/09/10 room: floor 60x60 at rotationX=-pi/2; walls at (-8,5,-4), (8,5,-4), and (0,5,-12); box i in {0..4} at (-5+2.5*i,1+0.25*i,-6+3*(i mod 2)); sphere at (2,1.6,-2).", + "lighting": "Studio lighting from examples/shared/props plus ambient color 0x404860 at intensity 0.4; all transforms and intensities remain fixed.", + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "installed_three_effect_sequences": { + "version": "0.185.1", + "frame_mapping": "After the readiness barrier, shape-check renderer._nodes.nodeFrame, then set frameId=0, renderId=0, time=0, deltaTime=0, lastTime=performance.now(), updateMap=new WeakMap(), updateBeforeMap=new WeakMap(), and updateAfterMap=new WeakMap(). Replacing all three maps is required because NodeFrame stores prior frame/render IDs per node; reusing frame IDs without fresh maps can suppress updates. Permit exactly one renderer animation-loop tick per recorded scenario frame and no other NodeFrame.update() call; Animation.update calls NodeFrame.update() before the user callback, so the exact effect frame ID for recorded scenario frame n is f(n)=n+1. The harness must verify these mutable fields against node_modules/three/src/renderers/common/Renderer.js, node_modules/three/src/renderers/common/Animation.js, and node_modules/three/src/nodes/core/NodeFrame.js before using this pinned private bridge.", + "readiness_barrier": [ + "Await renderer.init(), verify backend.device is a live GPUDevice, and record adapter/features before graph construction.", + "Construct the exact selected subrun graph and deterministic DenoiseNode noise texture, then execute unrecorded readiness renders with recording disabled until every expected input/effect/upscaler texture has a backend GPUTexture, NodeManager._buildQueue.length is 0, NodeManager._buildInProgress is false, and all selected shader pipelines have completed compilation.", + "Await device.queue.onSubmittedWorkDone() and require zero Log.entryAdded WGSL/WebGPU errors, Runtime.exceptionThrown events, uncaught console errors, and device-loss events.", + "Retain the compiled graph and allocated resources. Reset upscaler history and all scenario counters. Set every selected SSRNode._noiseIndex.value to 0 after shape-checking that private uniform. Q8 additionally calls setSize(1,1) on TemporalReprojectNode and RecurrentDenoiseNode so recorded n=0 synchronously takes each source-defined needsRestart seed/clear path and cannot read readiness history.", + "Reset NodeFrame counters and replace all three NodeFrame update maps with the pinned bridge described by frame_mapping. No selected effect or upscaler render occurs between that reset and recorded n=0. The readiness frames are never assigned scenario frame numbers and none of their history survives into recorded output.", + "Recorded n=0 is the first post-reset effect/upscaler execution and is captured when required; it is never replaced by an unrecorded accumulation frame." + ], + "gtao": { + "source": "node_modules/three/examples/jsm/tsl/display/GTAONode.js at installed three 0.185.1", + "sequence": "Q6 uses useTemporalFiltering=false, therefore temporalDirection(n)=0 exactly. If enabled in a future manifest, temporalDirection(n)=[60,300,180,240,120,0][f(n) mod 6]/360.", + "noise": "The static 5x5 repeat-wrapped magic-square texture is generated with the Siamese algorithm in GTAONode.generateMagicSquare(5): start i=2,j=4; place integers 1..25 with the source wrap/collision rules; texel k with magic value m uses angle=2*pi*m/25 and RGBA8=(uint8((0.5+0.5*cos(angle))*255),uint8((0.5+0.5*sin(angle))*255),127,255). No PRNG or per-frame noise index is used." + }, + "ssgi": { + "source": "node_modules/three/examples/jsm/tsl/display/SSGINode.js at installed three 0.185.1; rand is node_modules/three/src/nodes/math/MathNode.js", + "frame_sequence": "With useTemporalFiltering=true, temporalDirection(n)=[60,300,180,240,120,0][f(n) mod 6]/360 and temporalOffset(n)=[0,0.5,0.25,0.75][f(n) mod 4].", + "pixel_noise": "For integer screen pixel p=(px,py), spatialOffset(p)=0.25*((py-px) bitAnd 3), interleavedGradientNoise(p)=fract(52.9829189*fract(0.06711056*px+0.00583715*py)), and noiseJitterIdx(n)=0.02*temporalDirection(n). Define rand(q)=fract(sin(mod(12.9898*q.x+78.233*q.y,pi))*43758.5453). initialRayStep(n,p,uv)=fract(spatialOffset(p)+temporalOffset(n))+rand(2*(uv+(noiseJitterIdx(n),noiseJitterIdx(n)))-1).", + "slice_rotation": "For slice i in [0,sliceCount), rotationAngle(n,p,i)=(i+interleavedGradientNoise(p)+temporalDirection(n))*pi/sliceCount. Q6/Q7 use the node defaults unless the scenario states counts; Q8 fixes sliceCount=2 and stepCount=8." + }, + "ssr": { + "source": "node_modules/three/examples/jsm/tsl/display/SSRNode.js and node_modules/three/examples/jsm/tsl/utils/RNoise.js at installed three 0.185.1", + "sequence": "Q6-Q8 construct SSR with stochastic=false. Readiness advances SSRNode._noiseIndex independently of NodeFrame, so the barrier explicitly shape-checks and resets that private uniform to 0. Recorded updateBefore then computes noiseIndex(n)=(n+1) mod 2147483647. The non-stochastic shader does not sample bindAnalyticNoise, so no frame-varying SSR noise affects output. Mirror ray direction, march, and box-blur taps are deterministic functions of current G-buffer values." + }, + "denoise_node": { + "source": "node_modules/three/examples/jsm/tsl/display/DenoiseNode.js and node_modules/three/examples/jsm/math/SimplexNoise.js at installed three 0.185.1", + "default_limitation": "DenoiseNode.generateDefaultNoise() constructs SimplexNoise() with Math.random, so the installed default texture is not reload-deterministic and may not be used for E00 evidence.", + "deterministic_override": "Before compilation, assign the existing DenoiseNode.noiseNode property to a 64x64 RepeatWrapping RGBA8 texture generated by SimplexNoise({random}) with xorshift32 seed 0x0d3e015e. xorshift32 applies x^=x<<13, x^=x>>>17, x^=x<<5 with uint32 masking and returns x/4294967296. SimplexNoise sets p[j]=floor(random()*256) for j=0..255 and perm[j]=p[j bitAnd 255] for j=0..511. For i,j in 0..63, RGBA channels are uint8((noise(i,j),noise(i+64,j),noise(i,j+64),noise(i+64,j+64))*0.5*255+0.5*255) exactly as the source's per-channel (noise*0.5+0.5)*255 conversion.", + "rotation": "DenoiseNode.index remains its source default 0 for Q6-Q8. The installed expression indexes noiseTexel with index.mod(4)*2*pi; at index 0 this selects red without scaling its value. At pixel uv, sample the deterministic repeat-wrapped texture at noiseUv=(uv.x,1-uv.y)*(resolution/64), let r be sampled red, and use rotation vector (sin(r),cos(r)); the 16 source sample vectors use angle_i=2*pi*2*i/16 and radius_i=i/15." + }, + "recurrent_denoise": { + "source": "node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js and node_modules/three/examples/jsm/tsl/utils/RNoise.js at installed three 0.185.1", + "frame_sequence": "RecurrentDenoiseNode.updateBefore assigns noiseIndex(n)=f(n)=n+1. Let k=n+1+83, tileSize=32, screenPixel=floor(uv*resolution), offset=floor(fract((0.7548776662*k,0.5698402910*k))*32), and coords=(screenPixel+offset) mod 32.", + "analytic_noise": "Let P=1.32471795724474602596 and z=coords.x/P+coords.y/(P^2)+83. The four noise channels are fract(z*P*(1/P)), fract(z*(2*P)*(1/P^2)), fract(z*(3*P)*0.4198754210), and fract(z*(4*P)*(1/P^3)). Rotation angle=2*pi*noise.r and matrix=((cos,-sin),(sin,cos)).", + "kernel": "For tap i in 0..7, theta=(i+0.5)*2.399827721492203 and radius=sqrt((i+0.5)/8); baseOffset=(cos(theta),sin(theta))*radius before the source's deterministic adaptive skew and edge-stopping terms." + }, + "temporal_reproject": { + "source": "node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js at installed three 0.185.1", + "sequence": "TemporalReprojectNode has no random sampling index. Its recorded n=0 needsRestart path seeds internal history from current beauty, then subsequent frames are deterministic functions of current inputs, previous history/depth/normal, velocity, and camera matrices." + } + }, + "roi_coordinates": "ROIs are normalized [x,y,width,height] in top-left canvas coordinates and are resolved to integer pixels by floor(x*width), floor(y*height), ceil(widthFraction*width), ceil(heightFraction*height)." + }, + "required": [ + { + "id": "Q0", + "name": "input-debug-validation", + "purpose": "Validate color, depth, motion, disocclusion, accumulation age, locks, exposure, shading change, and reactivity inputs and debug outputs.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 143, + "duration_seconds": 2.4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "jitter-free velocity", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "119" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "floor_grid": [ + 0.05, + 0.55, + 0.9, + 0.45 + ], + "fence_and_spheres": [ + 0.05, + 0.28, + 0.9, + 0.42 + ] + } + } + }, + { + "id": "Q1", + "name": "static-convergence", + "purpose": "Establish deterministic accumulation and settled baseline output.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "zero object and camera velocity except jitter compensation", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "119", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "thin_features": [ + 0.08, + 0.3, + 0.84, + 0.55 + ] + } + } + }, + { + "id": "Q2", + "name": "slow-aliasing-dolly", + "purpose": "Replay an absolute slow dolly across aliasing-torture geometry.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "u=n/239; position=(9-3*u,6,12-4*u), target=(0,1.6,0), up=(0,1,0).", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera velocity", + "auto exposure", + "locks" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "floor_grid": [ + 0, + 0.48, + 1, + 0.52 + ], + "fence": [ + 0.05, + 0.35, + 0.9, + 0.32 + ] + } + } + }, + { + "id": "Q3", + "name": "object-motion-disocclusion", + "purpose": "Replay independent object motion and silhouette disocclusion.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly; sphere and knot transforms are assigned from absolute t each frame.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "independent object velocity", + "auto exposure", + "locks" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "moving_spheres": [ + 0.12, + 0.2, + 0.76, + 0.58 + ], + "fence_silhouette": [ + 0.05, + 0.35, + 0.9, + 0.3 + ] + } + } + }, + { + "id": "Q4", + "name": "camera-motion-hold", + "purpose": "Replay slow yaw, fast translation, rotation, and a final hold.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 479, + "duration_seconds": 8 + }, + "camera_formula": "Let T=(0,1.6,0), R=15, h=4.4, theta0=atan2(12,9). For 0<=n<=119, theta=theta0+0.25*n/119 and position=T+(R*cos(theta),h,R*sin(theta)). Let C119 be that n=119 position. For 120<=n<=239, u=(n-120)/119 and position=C119+u*(-8,0,-6). Let C239 be that n=239 position. For 240<=n<=359, theta=atan2(C239.z-T.z,C239.x-T.x)+0.9*(n-240)/119, radius=sqrt((C239.x-T.x)^2+(C239.z-T.z)^2), and position=T+(radius*cos(theta),C239.y-T.y,radius*sin(theta)). For 360<=n<=479 hold the n=359 position. Target=T and up=(0,1,0) for all n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Motion segment boundaries are n=120, n=240, and n=360; no cut, resize, or reset after the initial reset before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera velocity", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "2*P-1", + "118", + "119", + "120", + "121", + "238", + "239", + "240", + "241", + "358", + "359", + "360", + "361", + "383", + "479" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "thin_geometry": [ + 0.05, + 0.25, + 0.9, + 0.55 + ] + } + } + }, + { + "id": "Q5", + "name": "seeded-transparency-reactivity", + "purpose": "Replay transparent surfaces and seeded particles with aligned reactive inputs.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Freeze baseline objects at t=0. For particle i in {0..127}, consume four initialization values ux,uy,uz,up from the seeded PRNG and assign phase=2*pi*up, base=(-5+10*ux,0.7+4*uy,-5+10*uz); at frame n assign position=(base.x+0.35*sin(0.7*t+phase),base.y+0.6*((0.35*t+up) mod 1),base.z+0.35*cos(0.7*t+phase)). Particle transform is recomputed absolutely and does not advance the PRNG.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; reset re-seeds and regenerates all particle constants before replay. No cut or resize.", + "randomness": { + "seed_uint32": 1592594996, + "seed_hex": "0x5eed1234", + "prng": "xorshift32: x ^= x<<13; x ^= x>>>17; x ^= x<<5 with uint32 masking after each operation; output=x/4294967296", + "policy": "Exactly four outputs per particle are consumed in ascending particle index during reset. Rendering consumes no random values." + }, + "active_inputs_effects": [ + "bench opaque color", + "transparent particle final color", + "manual particle coverage reactive mask", + "float depth", + "jitter-free velocity", + "auto exposure" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "accumulation-age", + "locks", + "reactivity" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "particle_volume": [ + 0.18, + 0.1, + 0.64, + 0.72 + ], + "opaque_edges": [ + 0.05, + 0.35, + 0.9, + 0.45 + ] + } + } + }, + { + "id": "Q6", + "name": "isolated-screenspace-effects", + "purpose": "Exercise isolated GTAO, SSR, and SSGI inputs.", + "initial_camera": { + "position": [ + 0, + 4, + 10 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(0,4,10), target=(0,3,-5), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Run three independent reset-before-n=0 subruns in order GTAO, SSR, SSGI. Each subrun reloads the page and creates one resolver.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier independently to each GTAO, SSR, and SSGI subrun. The exact barrier must complete before n=0; readiness renders are unrecorded, then upscaler history and NodeFrame are reset. GTAO, SSR without multibounce, SSGI, and DenoiseNode own no temporal history, so no readiness accumulation remains.", + "effect_sampling_sequence": "GTAO uses installed_three_effect_sequences.gtao with temporalDirection(n)=0. SSR uses installed_three_effect_sequences.ssr with stochastic=false, so its incremented noise index is dead shader state. SSGI uses installed_three_effect_sequences.ssgi with f(n)=n+1. SSR and SSGI spatial denoisers use installed_three_effect_sequences.denoise_node with the seeded noiseNode override and fixed index=0.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for the deterministic DenoiseNode SimplexNoise texture; GTAO magic-square and SSGI shader noise use their exact source formulas.", + "policy": "Generate the DenoiseNode texture once before compilation, then make no CPU PRNG calls. Record f(n)=n+1, SSGI temporalDirection/temporalOffset, GTAO temporalDirection=0, SSR stochastic=false, and DenoiseNode index=0 for every capture." + }, + "active_inputs_effects": [ + "Subrun gtao: beauty * GTAO(depth, normal)", + "Subrun ssr: beauty + spatially denoised SSR(beauty, depth, normal, packed metalness and roughness)", + "Subrun ssgi: beauty * AO + diffuse * spatially denoised SSGI GI", + "float depth", + "jitter-free velocity", + "normal MRT", + "effect-specific packed material or diffuse MRT", + "RCAS denoise=true" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "floor_reflection": [ + 0.05, + 0.5, + 0.9, + 0.5 + ], + "wall_contact_and_bounce": [ + 0.08, + 0.08, + 0.84, + 0.6 + ] + } + } + }, + { + "id": "Q7", + "name": "in-graph-screenspace-composition", + "purpose": "Exercise in-graph SSGI and SSR composition with one jitter owner.", + "initial_camera": { + "position": [ + 0, + 4, + 10.5 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(7*sin(0.15*t),4,9+1.5*cos(0.15*t)), target=(0,3,-5), up=(0,1,0).", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut or resize. The complete graph is rebuilt only before n=0.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier to the complete in-graph SSGI+SSR subrun. The renderer/device, selected pipelines, graph queue, effect textures, G-buffer resources, and upscaler resources must be ready before n=0. Readiness renders are unrecorded; SSGI, non-stochastic SSR, and DenoiseNode are stateless, then upscaler history and NodeFrame are reset so recorded n=0 is the first accumulated frame.", + "effect_sampling_sequence": "SSGI uses installed_three_effect_sequences.ssgi with f(n)=n+1. SSR uses installed_three_effect_sequences.ssr with stochastic=false and no sampled analytic noise. Both spatial DenoiseNode instances use installed_three_effect_sequences.denoise_node with seed 0x0d3e015e and fixed index=0. The upscaler is the only projection-jitter owner.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for deterministic DenoiseNode SimplexNoise textures; SSGI shader noise follows its pinned analytical formulas.", + "policy": "Generate identical deterministic denoise textures before compilation and make no later CPU PRNG calls. Record f(n)=n+1, SSGI direction/offset, SSR stochastic=false, and each DenoiseNode index=0." + }, + "active_inputs_effects": [ + "in-graph beauty", + "SSGI AO and spatially denoised GI", + "spatially denoised SSR", + "float depth", + "jitter-free velocity", + "packed normal plus roughness MRT", + "packed diffuse plus metalness MRT", + "RCAS denoise=true", + "upscaler jitter=true and sole owner" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "glossy_floor": [ + 0.05, + 0.48, + 0.9, + 0.52 + ], + "colored_walls": [ + 0.05, + 0.05, + 0.9, + 0.62 + ] + } + } + }, + { + "id": "Q8", + "name": "recurrent-denoiser-characterization", + "purpose": "Record the existing recurrent-denoiser conflict without changing it.", + "initial_camera": { + "position": [ + 0, + 4, + 10.5 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(7*sin(0.15*t),4,9+1.5*cos(0.15*t)), target=(0,3,-5), up=(0,1,0).", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Run three independent reset-before-n=0 subruns in order builtin, spatial, recurrent. Each subrun reloads, rebuilds the graph, and resets all effect and upscaler history.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier independently to builtin, spatial, and recurrent subruns; it must complete before n=0. After readiness and before NodeFrame reset, builtin only resets upscaler history; spatial resets upscaler history and forces RecurrentDenoiseNode.setSize(1,1); recurrent resets upscaler history and forces both TemporalReprojectNode.setSize(1,1) and RecurrentDenoiseNode.setSize(1,1). Thus recorded n=0 takes the installed source needsRestart seed/clear paths and no unrecorded temporal or denoise history survives.", + "effect_sampling_sequence": "All subruns use installed_three_effect_sequences.ssgi at sliceCount=2, stepCount=8, and f(n)=n+1. SSR is non-stochastic per installed_three_effect_sequences.ssr and its DenoiseNode uses the fixed seeded texture/index contract. Spatial and recurrent denoisers use installed_three_effect_sequences.recurrent_denoise with noiseIndex(n)=n+1; recurrent additionally uses installed_three_effect_sequences.temporal_reproject, which has no random index and seeds from current beauty on n=0.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for the deterministic SSR DenoiseNode SimplexNoise texture; SSGI and RecurrentDenoiseNode use their pinned shader formulas.", + "policy": "After deterministic texture creation, make no CPU PRNG calls. Record f(n)=n+1, SSGI direction/offset, RecurrentDenoiseNode noiseIndex and RNoise offset, SSR stochastic=false, and DenoiseNode index=0 for each frame." + }, + "active_inputs_effects": [ + "SSGI slices=2 and steps=8", + "Subrun builtin: raw SSGI GI with FSR owning temporal accumulation", + "Subrun spatial: recurrentDenoise mode=diffuse and accumulate=false", + "Subrun recurrent: temporalReproject plus recurrentDenoise mode=diffuse and accumulate=true", + "SSR=true", + "RCAS denoise=true", + "upscaler jitter=true" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "flat_walls": [ + 0.08, + 0.08, + 0.84, + 0.52 + ], + "occlusion_edges": [ + 0.18, + 0.24, + 0.64, + 0.54 + ] + } + } + }, + { + "id": "Q9", + "name": "exposure-transition", + "purpose": "Replay a light step at frame 60, a linear ramp over frames 120 through 179, and a reverse transition at frame 180.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Directional color and position remain baseline. Directional intensity I(n)=3.2 for 0<=n<60; I(n)=8.0 for 60<=n<120; I(n)=8.0-6.0*(n-120)/59 for 120<=n<180; I(n)=3.2 for 180<=n<=239. Hemisphere intensity remains 0.9. settings.exposure=1.0 and autoExposure=true for every n; no external exposure texture is bound.", + "events": "Light step 3.2 to 8.0 at n=60; inclusive linear ramp 8.0 to 2.0 over n=120..179; reverse step 2.0 to 3.2 at n=180. Reset only before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "zero scene velocity except jitter compensation", + "auto exposure", + "shading-change detector", + "locks" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "2*P-1", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "149", + "178", + "179", + "180", + "181", + "182", + "184", + "188", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "lit_knots": [ + 0.18, + 0.16, + 0.64, + 0.38 + ], + "hdr_bulb": [ + 0.43, + 0.08, + 0.14, + 0.2 + ] + } + } + }, + { + "id": "Q10", + "name": "reset-cut-resize", + "purpose": "Replay a camera cut and history reset at frame 60 and deterministic resize resets at frames 120 and 180.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "For 0<=n<60 position=(9,6,12); for 60<=n<=239 position=(-7,4,9). Target=(0,1.6,0) and up=(0,1,0) for every n. Projection is 50-degree perspective with near=0.1 and far=200; aspect equals the active output width/height after each resize.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly with absolute t=n/60; object animation does not restart at the mid-run history resets.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n. Exposure adaptation history is cleared at each declared reset.", + "events": "Before n=0 configure 1920x1080 and reset. At the start of n=60 set camera position=(-7,4,9), recompute lookAt, then reset all upscaler histories and jitter to phase zero while the scenario frame remains 60. At the start of n=120 resize physical output to 1280x720 at DPR 1, rebuild size-dependent resources, and reset history/jitter. At the start of n=180 restore 1920x1080 at DPR 1, rebuild size-dependent resources, and reset history/jitter.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera and object velocity", + "auto exposure", + "locks", + "shading-change detector", + "size-dependent resource rebuild" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "122", + "124", + "128", + "136", + "143", + "179", + "180", + "181", + "182", + "184", + "188", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "moving_silhouettes": [ + 0.08, + 0.18, + 0.84, + 0.58 + ] + } + } + }, + { + "id": "Q11", + "name": "host-pre-exposure", + "purpose": "Replay an app-baked host pre-exposure step at frame 60, a hold, and a linear ramp home over frames 120 through 179, with the same value fed as the preExposureTexture dispatch input; proves DeltaPreExposure history correction (no shading-change firing, no accumulation reset, output brightness tracks the drive).", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Directional intensity fixed at 3.2 and hemisphere at 0.9 for every n. Host pre-exposure H(n)=1.0 for 0<=n<60; H(n)=2.5 for 60<=n<120; H(n)=2.5-1.5*(n-120)/59 for 120<=n<180; H(n)=1.0 for 180<=n<=239. The scene MRT color is multiplied by H(n) and the same value is bound as the preExposureTexture dispatch input. settings.exposure=1.0 and autoExposure=true for every n; no external exposure texture is bound.", + "events": "Host pre-exposure step 1.0 to 2.5 at n=60; inclusive linear ramp 2.5 to 1.0 over n=120..179; steady at 1.0 from n=180. Reset only before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color scaled by host pre-exposure", + "float depth", + "zero scene velocity except jitter compensation", + "auto exposure", + "host preExposureTexture", + "shading-change detector", + "locks" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "135", + "149", + "164", + "179", + "180", + "181", + "184", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "lit_knots": [ + 0.18, + 0.16, + 0.64, + 0.38 + ], + "hdr_bulb": [ + 0.43, + 0.08, + 0.14, + 0.2 + ] + } + } + } + ], + "required_debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "capture_protocol": { + "jitter_period_symbol": "P", + "jitter_period_definition": "The configured jitter phase count reported by the active baseline resolver for the current render and display dimensions.", + "jitter_period_by_ratio": { + "1": 8, + "1.5": 18, + "2": 32, + "3": 72 + }, + "harness_acceptance_matrix": { + "purpose": "E00 proves deterministic input/debug capture, static convergence, and cut/reset/resize handling without pre-running every later experiment's visual matrix. Q2-Q9 remain required implemented scenarios and receive reduced GPU smoke coverage; their authoritative captures belong to the domain experiments that consume them.", + "review_sampling": "Run all 45 numerical reload comparisons for every selected tuple. Human review samples the blinded A-B reload-1 pair at the frames and views below. Each review record receives one grade after the reviewer inspects the full frame and every declared ROI listed in that record.", + "reviewer_count": 1, + "human_review_scenarios": { + "Q0": { + "frames": [ + "P", + "2*P-1" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "Q1": { + "frames": [ + "119" + ], + "debug_views": [ + "final", + "accumulation-age" + ] + }, + "Q10": { + "frames": [ + "60", + "120", + "180" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ] + } + }, + "scenarios": { + "Q0": { + "frames": [ + "0", + "P", + "2*P-1" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "Q1": { + "frames": [ + "0", + "23", + "119" + ], + "debug_views": [ + "final", + "accumulation-age" + ] + }, + "Q10": { + "frames": [ + "59", + "60", + "61", + "119", + "120", + "121", + "179", + "180", + "181" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ] + } + } + }, + "absolute_frame_expressions": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "119" + ], + "event_relative_frames": { + "Q9_frame_60_light_step": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q9_frame_120_ramp_start": [ + -1, + 0, + 1, + 29, + 58, + 59 + ], + "Q9_frame_180_reverse_transition": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_60_camera_cut_and_reset": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_120_resize": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_180_restore_resize": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ] + }, + "rules": [ + "Resolve every frame expression to an integer and record P and the resolved frame list in the artifact metadata.", + "Reset and reload baseline and comparison runs to the same initial state before replay.", + "Capture only the canvas at the scenario's active physical dimensions; hide GUI, stats, and browser chrome.", + "Use the identical baseline presentation path and browser capture settings for both blinded labels.", + "Record scenario, ratio, frame, debug view, blinded label, and ROI with every image." + ], + "numerical_equivalence": { + "independent_reload_capture_count_per_variant": 5, + "comparison_pairs": "Compare all 10 unordered pairs among the five baseline reloads to derive repeatability, all 10 unordered pairs among the five candidate reloads, and the 25 baseline-to-candidate pairs for each identical scenario, subrun, ratio, resolved frame, debug view, and ROI tuple.", + "pixel_domain": "Decode lossless PNG to straight RGBA8. Compare R, G, and B as normalized display-referred sRGB code values byte/255 before any linearization, color management, resizing, filtering, or alpha premultiplication.", + "max_absolute_rgb_tolerance": 0.00392156862745098, + "rmse_rgb_tolerance": 0.000980392156862745, + "dimension_rule": "Both images must exactly equal the scenario's active physical canvas width and height. Q10 expects 1920x1080 before frame 120, 1280x720 for frames 120 through 179, and 1920x1080 from frame 180.", + "alpha_rule": "Every decoded pixel in both images must have alpha byte 255. Alpha is an exact gate and is excluded from RGB max-absolute and RMSE only after this gate passes.", + "metric_formula": "For N pixels and three RGB channels, maxAbsolute=max(abs(a_c-b_c))/255 and RMSE=sqrt(sum((a_c-b_c)/255)^2/(3*N)). Calculate full-frame and each declared ROI independently.", + "failure_artifacts": "For every failed pair, retain both PNGs, an absolute-difference heatmap with a fixed 0..1/255 legend, metric JSON, scenario metadata, and the failed ROI." + }, + "artifact_rubric": { + "0": "No visible difference at 1x or 4x nearest-neighbor inspection; numerical gates pass.", + "1": "Difference is detectable only at 4x inspection or in a difference heatmap; normal 1x viewing is unaffected.", + "2": "Difference is visible at 1x when the reviewer knows the ROI but is not distracting during motion.", + "3": "Difference is distracting at 1x or causes clear shimmer, ghosting, ringing, blur, exposure lag, mask shift, or reset residue.", + "4": "Invalid or severe output: black/NaN pixels, dimension/domain mismatch, full-screen flashing, device/validation failure, or unusable temporal instability." + }, + "pass_fail_aggregation": { + "pair": "PASS only when dimensions and alpha pass and full-frame plus every ROI max-absolute and RMSE are within tolerance.", + "capture_tuple": "PASS only when all 45 required repeatability and cross-variant pairs pass; every tuple selected by harness_acceptance_matrix.human_review_scenarios must receive a reviewer grade of 0 or 1, and the median grade across the experiment must be 0.", + "scenario": "PASS only when every required subrun, ratio, resolved frame, debug view, and ROI capture tuple passes.", + "experiment": "The E00 visual equivalence gate passes only when every tuple in capture_protocol.harness_acceptance_matrix passes. Q2-Q9 must pass reduced GPU smoke coverage before E00 closes and receive authoritative visual coverage in their consuming domain experiments. One failed pair fails its tuple and scenario; reruns may replace the complete five-capture set once, never only the failed image." + } + }, + "validation_gates": { + "static": [ + "git diff --check reports no errors.", + "The changed-path set is a subset of scope.allowed_write_paths.", + "Lint, typecheck, all unit tests, and build pass.", + "Structural tests prove the registry cannot instantiate more than one active resolver.", + "Baseline shader strings remain byte-for-byte identical or are demonstrated behaviorally equivalent." + ], + "runtime": [ + "Unknown or unsupported variants fail before rendering.", + "Exactly one resolver, one pass graph, and one projection-jitter owner are active per page load.", + "Candidate and baseline pipeline metadata identify shader key, settings, resource graph, supported ratios, WGSL overrides, and assembled chunks.", + "There are zero WGSL compilation errors, WebGPU validation errors, uncaught runtime exceptions, and device-loss events.", + "Every authoritative timing run has timestamp-query support and fresh frame-tagged samples.", + "No stale timing label survives a graph change and pending readbacks drain before finalization." + ], + "visual": [ + "Every required capture pair passes capture_protocol.numerical_equivalence: exact dimensions and alpha=255, max absolute RGB <= 1/255, and RGB RMSE <= 0.25/255 in display-referred sRGB code-value space. Measured repeatability is recorded but cannot widen either fixed threshold.", + "Motion vectors remain smooth and jitter-free, disocclusion does not flash full screen, accumulation age converges, locks and exposure reset correctly, shading-change is stable on a still scene, and reactivity remains aligned.", + "No black output, NaN output, incorrect dimensions, double tone mapping, output-domain shift, stale history, or shifted mask is present." + ], + "timing": [ + "Each run produces exactly 600 fresh measured samples after 240 warm-up frames.", + "All four ABBA repetitions complete at ratios 1, 1.5, 2, and 3.", + "At least three of four repetitions pass the equivalence threshold.", + "Baseline-versus-baseline median compute-sum delta is no greater than the larger of 3 percent or twice the measured baseline noise floor.", + "Baseline-versus-baseline p95 compute-sum delta is no greater than the larger of 5 percent or twice the measured baseline noise floor.", + "At every ratio, compute-sum N_median must be <=1.5 percent and N_p95 <=2.5 percent. Baseline equivalence then uses median max(3 percent,2*N_median) and p95 max(5 percent,2*N_p95), each passing at least three of four repetitions. One full cold-browser rerun is allowed; a second compute-sum noise-floor failure blocks E00. Per-pass noise eligibility is recorded separately and an ineligible pass cannot support an individual performance claim." + ], + "review": [ + "A fresh read-only reviewer verifies manifest compliance, scope, one-variant isolation, reset behavior, sample freshness, and evidence completeness.", + "Critical and important findings are consolidated into no more than two fix rounds.", + "No parity experiment starts until E00 is adopted and documented." + ] + }, + "timing_protocol": { + "required_feature": "timestamp-query", + "performance_claim_without_feature": "invalid", + "warmup_frames_per_run": 240, + "fresh_samples_per_run": 600, + "block_sequence": [ + "A", + "B", + "B", + "A" + ], + "block_repetitions": 4, + "variant_mapping": { + "A": "local-baseline-5d6a65e", + "B": "local-baseline-through-e00-harness" + }, + "noise_floor_derivation": { + "inputs": "For each ratio, pass label, compute sum, and statistic kind k in {median,p95}, each ABBA repetition r yields A1_r, B1_r, B2_r, A2_r from 600 fresh samples per run.", + "same_variant_relative_deltas": "dA_r=abs(A1_r-A2_r)/((A1_r+A2_r)/2) and dB_r=abs(B1_r-B2_r)/((B1_r+B2_r)/2).", + "quantile": "Sort the four dA values and independently the four dB values. q95 uses linear interpolation at zero-based index 0.95*(count-1). N_k=max(q95(dA),q95(dB)).", + "acceptable_noise_floor": "For compute sum at every ratio, N_median<=0.015 and N_p95<=0.025. Apply the same limits to per-pass claim eligibility when baseline median is at least 0.020 milliseconds; smaller passes are timer-resolution-limited. Any pass that exceeds either limit remains in compute sum but cannot support an individual performance claim until a pass-specific microbenchmark establishes a usable floor.", + "retry_policy": "If either compute-sum noise limit fails, discard the complete ratio run, cold-restart the browser once, and repeat all four ABBA repetitions. A second failure blocks E00 for that environment. Per-pass eligibility failures are recorded without triggering the aggregate E00 retry.", + "comparison_delta": "Within repetition r, meanA=(A1_r+A2_r)/2, meanB=(B1_r+B2_r)/2, and D_r=abs(meanA-meanB)/((meanA+meanB)/2). Median passes when D_r<=max(0.03,2*N_median); p95 passes when D_r<=max(0.05,2*N_p95). Each statistic must pass in at least three of four repetitions.", + "recording": "Record all A1/B1/B2/A2 statistics, dA, dB, sorted delta arrays, interpolated q95 values, N_median, N_p95, comparison limits, D values, pass counts, and retry state." + }, + "isolation": "Navigate or reload between runs. Instantiate and execute one resolver only; split-screen and concurrent A/B execution are forbidden for timing.", + "statistics": [ + "raw frame-tagged samples", + "missing-sample count", + "per-pass median", + "per-pass p95", + "compute-sum median", + "compute-sum p95", + "per-repetition A/B delta", + "baseline noise floor" + ], + "required_metadata": [ + "experiment ID", + "manifest digest", + "local SHA", + "working-tree digest including tracked and untracked non-ignored files", + "browser and version", + "operating system", + "GPU adapter", + "backend", + "WebGPU features", + "three version", + "physical dimensions", + "device pixel ratio", + "render ratio", + "fixed timestep", + "variant ID", + "shader key", + "pipeline key", + "settings", + "resource graph" + ], + "invalid_run_conditions": [ + "Missing timestamp-query support.", + "Any stale, duplicated, untagged, or missing measured sample.", + "Any validation, runtime, device-loss, output-domain, reset, resolver-isolation, or jitter-owner failure.", + "Any interactive input, wall-clock animation, resize handler, OrbitControls damping, or non-deterministic random input active in automated mode.", + "Baseline and comparison use different inputs, dimensions, ratios, presentation, capture settings, or browser process configuration." + ] + }, + "expected_behavior": { + "output_domain": "Display-referred sRGB produced by the existing internal ACES plus sRGB transform and stored in the existing output texture.", + "presentation_transform": "Identity sampling of the already transformed output; renderer remains NoToneMapping with LinearSRGBColorSpace and must not apply another tone or transfer transform.", + "jitter_owner": "The active upscaler resolver is the sole projection-jitter owner in temporal mode; automated inputs are rendered after its begin-frame jitter is applied.", + "velocity_convention": "Jitter-free NDC velocity converted by motionScale (0.5, -0.5), with reprojection prevUV = uv - motion.", + "reset_behavior": "Reset returns the deterministic frame clock and jitter sequence to frame zero, clears every temporal history, accumulation-age, lock, exposure-adaptation, pending timing, and scenario-event state, then reproduces frame-zero output without stale data.", + "resource_graph": "The existing local render, reconstruct, accumulate, luminance/exposure, RCAS or blit, debug, output, and history resources remain unchanged. The harness adds orchestration and metadata only and instantiates one graph.", + "binding_expectations": [ + "Every existing compute pass keeps the shared FsrConstants uniform buffer at group 0 binding 0.", + "Existing bind-group entry order, storage formats, sample types, depth-only views, and output mip-level view remain unchanged.", + "WGSL pipeline constants may specialize declared override values, but E00 introduces no alternate shader algorithm or additional production binding.", + "Disabled or unselected variants leave no alternate algorithm body, unused binding, pass, texture, or active resolver in the measured graph." + ] + }, + "required_evidence": [ + "Manifest ID and cryptographic digest.", + "Baseline and candidate IDs, source commit, local SHA, branch, and exact changed paths.", + "Command lines, exit codes, test totals, and execution timestamps.", + "Environment, adapter, WebGPU feature, dimensions, ratio, settings, shader key, pipeline key, and resource-graph metadata.", + "WGSL compilation, WebGPU validation, console, exception, and device-loss logs.", + "Resolved capture frames including P, scenario IDs, debug views, blinded labels, ROIs, and artifact-rubric observations.", + "Five-reload baseline and candidate capture sets, all fixed-threshold max-absolute/RMSE results, dimensions, alpha checks, rubric grades, and measured repeatability; repeatability cannot alter the manifest thresholds.", + "Fresh frame-tagged raw timestamp samples, missing count, per-pass and compute-sum median/p95, all ABBA repetition results, and measured noise floor.", + "Assertions for one active resolver, one jitter owner, output domain, reset state, resource graph, and binding stability.", + "Fresh reviewer findings, fix-round count, unresolved concerns, and controller decision rationale.", + "For platform-divergence: exact platform reason, affected platforms/scenarios/ratios/frames/debug views/ROIs, blinded visual and numerical impact, raw per-pass and compute-sum median/p95 impact, measured noise floor, and maintenance cost." + ], + "rollback_conditions": [ + "Any write outside scope.allowed_write_paths or any parity shader algorithm change.", + "Any WGSL compilation error, WebGPU validation error, uncaught runtime exception, or device loss.", + "Black or NaN output, incorrect dimensions, non-255 alpha, double tone mapping, output-domain mismatch, max absolute RGB above 1/255, or RGB RMSE above 0.25/255.", + "More than one active resolver or projection-jitter owner, or velocity generated from a jittered projection.", + "Full-screen disocclusion flashing, accumulation age that never converges, stale locks or history after reset, shifted reactive masks, or reset that retains prior timing or temporal state.", + "Missing, stale, duplicated, or untagged timing samples; undrained readbacks; or timing without timestamp-query.", + "Failure to complete exact sample counts and four ABBA repetitions at every required ratio.", + "More than two consolidated fix rounds without a new controller-authored manifest." + ], + "result_contract": { + "allowed_decisions": [ + "adopt", + "iterate", + "retain-local", + "platform-divergence", + "blocked" + ], + "decision_owner": "controller", + "decision_requirements": { + "adopt": "All static, runtime, visual, timing, evidence, and review gates pass.", + "iterate": "Evidence supports a bounded redesigned harness candidate that requires a new immutable manifest revision.", + "retain-local": "The harness candidate is validly measured but cannot preserve baseline correctness, determinism, quality, or acceptable overhead.", + "platform-divergence": "A documented browser or WebGPU platform constraint prevents the source-aligned harness behavior and measured evidence supports a bounded local alternative.", + "blocked": "A required dependency, capability, environment, or evidence gate remains unavailable after the allowed retry and fix policy." + } + }, + "report_contract": { + "allowed_statuses": [ + "PASS", + "FAIL", + "BLOCKED", + "SCOPE_BLOCKED", + "USER_DECISION_REQUIRED" + ], + "required_fields": [ + "status", + "changed_files", + "commands", + "artifacts", + "gates", + "concerns" + ], + "scope_rule": "If any additional write path or forbidden change appears necessary, stop before modifying it and return SCOPE_BLOCKED.", + "pass_rule": "PASS means only that the assigned implementation scope and all available assigned gates passed; it does not authorize adoption." + } +} \ No newline at end of file diff --git a/bench/src/BenchPipeline.ts b/bench/src/BenchPipeline.ts index 40599fc..8a25f0e 100644 --- a/bench/src/BenchPipeline.ts +++ b/bench/src/BenchPipeline.ts @@ -1,16 +1,127 @@ import * as THREE from 'three/webgpu'; -import { mrt, output, texture, velocity } from 'three/tsl'; - import { - Upscaler, - QualityMode, - getQualityModeRatio, - type DebugView, -} from '@pmndrs/upscaler'; + diffuseColor, + metalness, + mrt, + normalView, + output, + pass, + roughness, + texture, + uniform, + vec4, + velocity, +} from 'three/tsl'; +import { ao } from 'three/addons/tsl/display/GTAONode.js'; +import { denoise } from 'three/addons/tsl/display/DenoiseNode.js'; +import { recurrentDenoise } from 'three/addons/tsl/display/RecurrentDenoiseNode.js'; +import { ssr } from 'three/addons/tsl/display/SSRNode.js'; +import { ssgi } from 'three/addons/tsl/display/SSGINode.js'; +import { temporalReproject } from 'three/addons/tsl/display/TemporalReprojectNode.js'; +import { SimplexNoise } from 'three/addons/math/SimplexNoise.js'; + +import { DebugView, QualityMode, getQualityModeRatio } from '@pmndrs/upscaler'; /** Bench render modes — what fills the screen each frame. */ export type BenchMode = 'native' | 'bilinear' | 'fsr1-spatial' | 'upscale-temporal'; +type EffectScenario = { + id: 'Q6' | 'Q7' | 'Q8'; + subrun: string | null; + scene: THREE.Scene; + camera: THREE.PerspectiveCamera; +}; + +type SizedEffectNode = { + setSize(width: number, height: number): void; +}; + +type EffectUpdateNode = SizedEffectNode & { + updateBefore(frame: unknown): void; +}; + +type SsrPrivateNode = { + _noiseIndex: { value: number }; +}; + +type TextureExpectation = { + name: string; + texture: THREE.Texture | (() => THREE.Texture); + width: number; + height: number; + format?: GPUTextureFormat; +}; + +type NodeFrameBridge = { + frameId: number; + renderId: number; + time: number; + deltaTime: number; + lastTime: number; + updateMap: WeakMap; + updateBeforeMap: WeakMap; + updateAfterMap: WeakMap; + update(): void; +}; + +function deterministicDenoiseNoise(): THREE.DataTexture { + let seed = 0x0d3e015e; + const simplex = new SimplexNoise({ + random(): number { + seed = (seed ^ ((seed << 13) >>> 0)) >>> 0; + seed = (seed ^ (seed >>> 17)) >>> 0; + seed = (seed ^ ((seed << 5) >>> 0)) >>> 0; + return seed / 4294967296; + }, + }); + const size = 64; + const data = new Uint8Array(size * size * 4); + for (let i = 0; i < size; i++) { + for (let j = 0; j < size; j++) { + const offset = (i * size + j) * 4; + data[offset] = (simplex.noise(i, j) * 0.5 + 0.5) * 255; + data[offset + 1] = (simplex.noise(i + size, j) * 0.5 + 0.5) * 255; + data[offset + 2] = (simplex.noise(i, j + size) * 0.5 + 0.5) * 255; + data[offset + 3] = (simplex.noise(i + size, j + size) * 0.5 + 0.5) * 255; + } + } + const noise = new THREE.DataTexture(data, size, size); + noise.wrapS = THREE.RepeatWrapping; + noise.wrapT = THREE.RepeatWrapping; + noise.needsUpdate = true; + return noise; +} + +function seededDenoise( + node: unknown, + depth: unknown, + normal: unknown, + camera: THREE.PerspectiveCamera, +): ReturnType { + const originalRandom = Math.random; + let constructorSeed = 0x0d3e015e; + Math.random = (): number => { + constructorSeed = (constructorSeed ^ ((constructorSeed << 13) >>> 0)) >>> 0; + constructorSeed = (constructorSeed ^ (constructorSeed >>> 17)) >>> 0; + constructorSeed = (constructorSeed ^ ((constructorSeed << 5) >>> 0)) >>> 0; + return constructorSeed / 4294967296; + }; + let effect: ReturnType; + try { + effect = denoise(node as never, depth as never, normal as never, camera); + } finally { + Math.random = originalRandom; + } + const mutable = effect as unknown as { + noiseNode: { value?: THREE.Texture }; + index: { value: number }; + }; + mutable.noiseNode.value?.dispose(); + mutable.noiseNode = texture(deterministicDenoiseNoise()) as never; + mutable.index.value = 0; + return effect; +} + /** * Owns everything between "a scene + camera" and "pixels on the canvas": * the low-resolution scene render target (color + velocity MRT + depth), @@ -21,33 +132,60 @@ export type BenchMode = 'native' | 'bilinear' | 'fsr1-spatial' | 'upscale-tempor * comparisons are apples-to-apples. */ export class BenchPipeline { - readonly upscaler: Upscaler; + readonly resolver: BenchmarkResolver; //* Presentation private readonly _renderer: THREE.WebGPURenderer; private readonly _quad: THREE.QuadMesh; private readonly _quadMaterial: THREE.NodeMaterial; + private readonly _depthOnlyMaterial: THREE.MeshBasicMaterial; + private readonly _effectMaterial: THREE.NodeMaterial; + private readonly _effectQuad: THREE.QuadMesh; //* Scene Target private _renderTarget: THREE.RenderTarget | null = null; - private readonly _mrtNode = mrt({ output, velocity }); + private _reactiveTarget: THREE.RenderTarget | null = null; + private _mrtNode = mrt({ output, velocity }); // Single-output MRT for non-temporal modes — its output count matches the // count:1 render target so color attachment 0 is actually written. - private readonly _mrtOutputOnly = mrt({ output }); + private _mrtOutputOnly = mrt({ output }); + //* Host pre-exposure drive (scenario Q11) — off by default so every other + //* scenario keeps its exact historical scene shader and dispatch inputs. + private readonly _hostPreExposure = uniform(1); + private _hostPreExposureValue: number | null = null; + private readonly _hostPreExposureTextures = new Map(); + private _effectScenario: EffectScenario | null = null; + private _effectPass: ReturnType | null = null; + private _effectSsrNodes: SsrPrivateNode[] = []; + private _effectSizedNodes: SizedEffectNode[] = []; + private _effectTemporalNodes: unknown[] = []; + private _effectTextures: TextureExpectation[] = []; + private _velocitySeedFrame = -2; private _mode: BenchMode = 'upscale-temporal'; private _quality: QualityMode = QualityMode.Quality; private _displayWidth = 0; private _displayHeight = 0; - constructor(renderer: THREE.WebGPURenderer) { + /** + * Creates a pipeline around exactly one resolver factory. + * @param renderer - Initialized WebGPU renderer + * @param resolverFactory - Registry-owned factory for the active variant + * @param metadata - Selected variant metadata + */ + constructor( + renderer: THREE.WebGPURenderer, + resolverFactory: BenchmarkResolverFactory, + metadata: BenchmarkVariantMetadata, + ) { this._renderer = renderer; - this.upscaler = new Upscaler({ renderer }); - this.upscaler.init(); + this.resolver = resolverFactory(renderer, metadata); // Motion vectors must be jitter-free — hand the velocity node the // upscaler's unjittered projection (contents refresh every frame). - velocity.setProjectionMatrix(this.upscaler.unjitteredProjectionMatrix); + velocity.setProjectionMatrix( + this.resolver.unjitteredProjectionMatrix as THREE.Matrix4, + ); this._quadMaterial = new THREE.NodeMaterial(); // Plain fullscreen present — no depth interaction, no fog. @@ -55,6 +193,15 @@ export class BenchPipeline { this._quadMaterial.depthWrite = false; this._quadMaterial.fog = false; this._quad = new THREE.QuadMesh(this._quadMaterial); + this._effectMaterial = new THREE.NodeMaterial(); + this._effectMaterial.depthTest = false; + this._effectMaterial.depthWrite = false; + this._effectMaterial.fog = false; + this._effectQuad = new THREE.QuadMesh(this._effectMaterial); + this._depthOnlyMaterial = new THREE.MeshBasicMaterial({ + colorWrite: false, + depthWrite: true, + }); } get mode(): BenchMode { @@ -65,6 +212,30 @@ export class BenchPipeline { return this._renderTarget; } + get metadata(): BenchmarkVariantMetadata { + return this.resolver.metadata; + } + + get usesEffectGraph(): boolean { + return this._effectScenario !== null; + } + + /** + * Selects the pinned three.js effect graph built during configuration. + * @param id - Q6, Q7, or Q8 + * @param subrun - Manifest-selected effect subrun + * @param scene - Fixed room fixture + * @param camera - Scenario camera + */ + configureEffectScenario( + id: 'Q6' | 'Q7' | 'Q8', + subrun: string | null, + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + ): void { + this._effectScenario = { id, subrun, scene, camera }; + } + /** * (Re)builds the pipeline for a display size, mode, and quality preset. * @param displayWidth - Canvas width in physical pixels @@ -77,6 +248,41 @@ export class BenchPipeline { displayHeight: number, mode: BenchMode, quality: QualityMode, + ): void { + const ratio = mode === 'native' ? 1 : getQualityModeRatio(quality); + this._configure(displayWidth, displayHeight, mode, quality, ratio, false); + } + + /** + * Configures an exact manifest ratio for automated execution. + * @param displayWidth - Physical output width + * @param displayHeight - Physical output height + * @param ratio - Manifest display/render ratio + * @param reactive - Allocate the Q5 reactive coverage target + */ + configureBenchmark( + displayWidth: number, + displayHeight: number, + ratio: number, + reactive: boolean, + ): void { + this._configure( + displayWidth, + displayHeight, + 'upscale-temporal', + QualityMode.Quality, + ratio, + reactive, + ); + } + + private _configure( + displayWidth: number, + displayHeight: number, + mode: BenchMode, + quality: QualityMode, + ratio: number, + reactive: boolean, ): void { this._mode = mode; this._quality = quality; @@ -84,11 +290,10 @@ export class BenchPipeline { this._displayHeight = displayHeight; //* Upscaler - const ratio = mode === 'native' ? 1 : getQualityModeRatio(quality); - this.upscaler.configure({ + this.resolver.configure({ displayWidth, displayHeight, - customUpscaleRatio: ratio, + ratio, path: mode === 'upscale-temporal' ? 'temporal' @@ -102,39 +307,395 @@ export class BenchPipeline { // match the MRT output count used in render() (a count:2 target // rendered without a velocity output leaves color attachment 0 black). this._renderTarget?.dispose(); - const rw = this.upscaler.renderWidth; - const rh = this.upscaler.renderHeight; + this._reactiveTarget?.dispose(); + this._reactiveTarget = null; + const rw = this.resolver.renderWidth; + const rh = this.resolver.renderHeight; const temporal = mode === 'upscale-temporal'; - const depthTexture = new THREE.DepthTexture(rw, rh); - depthTexture.type = THREE.FloatType; - this._renderTarget = new THREE.RenderTarget(rw, rh, { - count: temporal ? 2 : 1, - type: THREE.HalfFloatType, - depthTexture, - }); + if (this._effectScenario) { + this._renderTarget = new THREE.RenderTarget(rw, rh, { + count: 1, + type: THREE.HalfFloatType, + depthBuffer: false, + }); + } else { + const depthTexture = new THREE.DepthTexture(rw, rh); + depthTexture.type = THREE.FloatType; + this._renderTarget = new THREE.RenderTarget(rw, rh, { + count: temporal ? 2 : 1, + type: THREE.HalfFloatType, + depthTexture, + }); + } // MRT routes node outputs to attachments BY TEXTURE NAME (see // three's getTextureIndex) — these must match the mrt({...}) keys. this._renderTarget.textures[0].name = 'output'; - if (temporal) this._renderTarget.textures[1].name = 'velocity'; + if (temporal && !this._effectScenario) this._renderTarget.textures[1].name = 'velocity'; + + if (reactive) { + this._reactiveTarget = new THREE.RenderTarget(rw, rh, { + count: 1, + type: THREE.HalfFloatType, + }); + this._reactiveTarget.textures[0].name = 'output'; + } // Present the (re)created output texture on the quad. - this._quadMaterial.colorNode = texture(this.upscaler.outputTexture); + this._quadMaterial.colorNode = texture(this.resolver.outputTexture as THREE.Texture); this._quadMaterial.needsUpdate = true; + if (this._effectScenario) this._buildEffectGraph(ratio); + this.resolver.resetTiming(); + } + + private _buildEffectGraph(ratio: number): void { + const effect = this._effectScenario!; + const scenePass = pass(effect.scene, effect.camera); + this._effectPass = scenePass; + this._effectSsrNodes = []; + this._effectSizedNodes = []; + this._effectTemporalNodes = []; + this._effectTextures = []; + + const combined = effect.id === 'Q7' || effect.id === 'Q8'; + const isolated = effect.subrun; + if (combined) { + scenePass.setMRT( + mrt({ + output, + velocity, + normal: vec4(normalView, roughness), + diffuse: vec4(diffuseColor.rgb, metalness), + }), + ); + } else if (isolated === 'ssgi') { + scenePass.setMRT(mrt({ output, normal: normalView, velocity, diffuse: diffuseColor })); + } else if (isolated === 'ssr') { + scenePass.setMRT( + mrt({ + output, + normal: normalView, + velocity, + material: vec4(metalness, roughness, 0, 0), + }), + ); + } else { + scenePass.setMRT(mrt({ output, normal: normalView, velocity })); + } + scenePass.setResolutionScale(1 / ratio); + if (!scenePass.renderTarget.depthTexture) + throw new Error('Effect PassNode did not allocate its required depth texture.'); + scenePass.renderTarget.depthTexture.type = THREE.FloatType; + + const beauty = scenePass.getTextureNode('output'); + const depth = scenePass.getTextureNode('depth'); + const normal = scenePass.getTextureNode('normal'); + const vel = scenePass.getTextureNode('velocity'); + let rgb = beauty.rgb; + + if (isolated === 'gtao') { + const gtao = ao(depth, normal, effect.camera); + gtao.useTemporalFiltering = false; + const aoTexture = (gtao as unknown as { getTextureNode(): ReturnType }) + .getTextureNode(); + rgb = beauty.rgb.mul(aoTexture.r); + const target = gtao as unknown as { _aoRenderTarget: THREE.RenderTarget }; + if (!target._aoRenderTarget) + throw new Error('Pinned GTAONode._aoRenderTarget shape changed.'); + this._pinEffectResolution(gtao as unknown as EffectUpdateNode); + this._trackEffectTexture('gtao.output', target._aoRenderTarget.texture); + } + + if (combined || isolated === 'ssgi') { + const diffuse = scenePass.getTextureNode('diffuse'); + const giPass = ssgi(beauty, depth, normal, effect.camera); + if (effect.id === 'Q8') { + giPass.sliceCount.value = 2; + giPass.stepCount.value = 8; + } + const aoTexture = giPass.getAONode() as unknown as ReturnType; + const giRaw = giPass.getGINode(); + let gi = giRaw as unknown as ReturnType; + + if (effect.id === 'Q8' && effect.subrun === 'spatial') { + const spatial = recurrentDenoise(giRaw as never, effect.camera, { + depth: depth as never, + normal: normal as never, + raw: giRaw as never, + mode: 'diffuse', + accumulate: false, + }); + gi = spatial as unknown as ReturnType; + this._effectSizedNodes.push(spatial as unknown as SizedEffectNode); + this._pinEffectResolution(spatial as unknown as EffectUpdateNode); + const spatialTarget = ( + spatial as unknown as { getRenderTarget(): THREE.RenderTarget } + ).getRenderTarget(); + this._trackEffectTexture('recurrent.spatial-output', spatialTarget.texture); + } else if (effect.id === 'Q8' && effect.subrun === 'recurrent') { + const reproject = temporalReproject( + giRaw as never, + depth as never, + normal as never, + vel as never, + effect.camera, + { mode: 'diffuse' }, + ); + const recurrent = recurrentDenoise(reproject as never, effect.camera, { + depth: depth as never, + normal: normal as never, + raw: giRaw as never, + mode: 'diffuse', + accumulate: true, + }); + reproject.setHistoryTexture(recurrent as never); + gi = recurrent as unknown as ReturnType; + this._effectSizedNodes.push( + reproject as unknown as SizedEffectNode, + recurrent as unknown as SizedEffectNode, + ); + this._effectTemporalNodes.push(reproject, recurrent); + this._pinEffectResolution(reproject as unknown as EffectUpdateNode); + this._pinEffectResolution(recurrent as unknown as EffectUpdateNode); + const temporalPrivate = reproject as unknown as { + _historyRenderTarget: THREE.RenderTarget; + _resolveRenderTarget: THREE.RenderTarget; + _previousNormalTexture: THREE.Texture; + }; + const recurrentTarget = ( + recurrent as unknown as { getRenderTarget(): THREE.RenderTarget } + ).getRenderTarget(); + if ( + !temporalPrivate._historyRenderTarget || + !temporalPrivate._resolveRenderTarget || + !temporalPrivate._previousNormalTexture + ) + throw new Error('Pinned TemporalReprojectNode render-target shape changed.'); + if (!temporalPrivate._historyRenderTarget.depthTexture) + throw new Error('Temporal history depth texture is unavailable.'); + temporalPrivate._historyRenderTarget.depthTexture.type = THREE.FloatType; + this._trackEffectTexture( + 'temporal.history-color', + temporalPrivate._historyRenderTarget.texture, + ); + if (temporalPrivate._historyRenderTarget.depthTexture) + this._trackEffectTexture( + 'temporal.history-depth', + temporalPrivate._historyRenderTarget.depthTexture, + undefined, + undefined, + 'depth32float', + ); + this._trackEffectTexture( + 'temporal.resolve', + temporalPrivate._resolveRenderTarget.texture, + ); + this._trackEffectTexture( + 'temporal.previous-normal', + () => temporalPrivate._previousNormalTexture, + ); + this._trackEffectTexture('recurrent.output', recurrentTarget.texture); + } else if (effect.id !== 'Q8') { + const denoised = seededDenoise(giRaw, depth, normal, effect.camera); + gi = denoised as unknown as ReturnType< + typeof vec4 + >; + const noise = (denoised as unknown as { noiseNode: { value: THREE.Texture } }) + .noiseNode.value; + this._trackEffectTexture( + 'denoise.ssgi-noise', + noise, + 64, + 64, + 'rgba8unorm', + ); + } + rgb = beauty.rgb.mul(aoTexture.r).add(diffuse.rgb.mul(gi.rgb)); + const privatePass = giPass as unknown as { + _ssgiRenderTarget: THREE.RenderTarget; + }; + if (!privatePass._ssgiRenderTarget) + throw new Error('Pinned SSGINode render-target shape changed.'); + this._pinEffectResolution(giPass as unknown as EffectUpdateNode); + privatePass._ssgiRenderTarget.textures.forEach((effectTexture, index) => + this._trackEffectTexture(`ssgi.attachment-${index}`, effectTexture), + ); + } + + if (combined || isolated === 'ssr') { + const material = + isolated === 'ssr' + ? scenePass.getTextureNode('material') + : scenePass.getTextureNode('diffuse'); + const rough = isolated === 'ssr' ? material.g : normal.a; + const metal = isolated === 'ssr' ? material.r : material.a; + const reflection = ssr(beauty, depth, normal as never, { + stochastic: false, + metalnessNode: metal, + roughnessNode: rough, + camera: effect.camera, + }); + const privateSsr = reflection as unknown as SsrPrivateNode & { + _ssrRenderTarget: THREE.RenderTarget; + }; + if (!privateSsr._noiseIndex || !privateSsr._ssrRenderTarget) + throw new Error('Pinned SSRNode private shape changed.'); + this._effectSsrNodes.push(privateSsr); + this._pinEffectResolution(reflection as unknown as EffectUpdateNode); + this._trackEffectTexture('ssr.base', privateSsr._ssrRenderTarget.texture); + const blurTarget = ( + reflection as unknown as { _blurRenderTarget?: THREE.RenderTarget } + )._blurRenderTarget; + if (!blurTarget) throw new Error('Pinned SSRNode._blurRenderTarget shape changed.'); + this._trackEffectTexture('ssr.blur-mips', blurTarget.texture); + const reflectionTexture = ( + reflection as unknown as { getTextureNode(): unknown } + ).getTextureNode(); + const filteredNode = seededDenoise( + reflectionTexture, + depth, + normal, + effect.camera, + ); + const filtered = filteredNode as unknown as ReturnType; + const noise = (filteredNode as unknown as { noiseNode: { value: THREE.Texture } }) + .noiseNode.value; + this._trackEffectTexture('denoise.ssr-noise', noise, 64, 64, 'rgba8unorm'); + rgb = rgb.add(filtered.rgb); + } + + this._effectMaterial.colorNode = vec4(rgb, beauty.a); + this._effectMaterial.needsUpdate = true; + const depthTexture = scenePass.renderTarget.depthTexture; + const velocityTexture = scenePass.getTexture('velocity'); + if (!depthTexture || !velocityTexture) + throw new Error('Effect graph did not expose depth and velocity textures.'); + scenePass.renderTarget.textures.forEach((effectTexture, index) => + this._trackEffectTexture(`scene-mrt.attachment-${index}`, effectTexture), + ); + this._trackEffectTexture('scene-mrt.depth', depthTexture, undefined, undefined, 'depth32float'); + this._trackEffectTexture('scene-mrt.velocity', velocityTexture); + this._trackEffectTexture('effect.intermediate', this._renderTarget!.texture); + this._trackEffectTexture( + 'resolver.output', + this.resolver.outputTexture as THREE.Texture, + this.resolver.displayWidth, + this.resolver.displayHeight, + ); + } + + private _pinEffectResolution(node: EffectUpdateNode): void { + if (typeof node.setSize !== 'function' || typeof node.updateBefore !== 'function') + throw new Error('Pinned effect sizing shape changed.'); + const width = this.resolver.renderWidth; + const height = this.resolver.renderHeight; + const originalUpdate = node.updateBefore; + node.setSize(width, height); + node.updateBefore = function updateAtResolverResolution(frame: unknown): void { + const renderer = (frame as { renderer?: THREE.WebGPURenderer }).renderer; + if (!renderer || typeof renderer.getDrawingBufferSize !== 'function') + throw new Error('Pinned effect NodeFrame renderer shape changed.'); + const mutableRenderer = renderer as unknown as { + getDrawingBufferSize(target: THREE.Vector2): THREE.Vector2; + }; + const originalGetDrawingBufferSize = mutableRenderer.getDrawingBufferSize; + mutableRenderer.getDrawingBufferSize = (target: THREE.Vector2): THREE.Vector2 => + target.set(width, height); + try { + originalUpdate.call(node, frame); + } finally { + mutableRenderer.getDrawingBufferSize = originalGetDrawingBufferSize; + } + }; + } + + private _trackEffectTexture( + name: string, + textureValue: THREE.Texture | (() => THREE.Texture), + width = this.resolver.renderWidth, + height = this.resolver.renderHeight, + format?: GPUTextureFormat, + ): void { + this._effectTextures.push({ name, texture: textureValue, width, height, format }); + } + + private _rawTexture(textureValue: THREE.Texture): GPUTexture | null { + const backend = this._renderer.backend as unknown as { + get(texture: THREE.Texture): { texture?: GPUTexture } | undefined; + }; + return backend.get(textureValue)?.texture ?? null; + } + + private _assertTexture(expectation: TextureExpectation): void { + const textureValue = + typeof expectation.texture === 'function' + ? expectation.texture() + : expectation.texture; + const raw = this._rawTexture(textureValue); + if (!raw) throw new Error(`GPU texture ${expectation.name} is not backed.`); + if (raw.width !== expectation.width || raw.height !== expectation.height) + throw new Error( + `GPU texture ${expectation.name} is ${raw.width}x${raw.height}; ` + + `expected ${expectation.width}x${expectation.height}.`, + ); + if (expectation.format && raw.format !== expectation.format) + throw new Error( + `GPU texture ${expectation.name} uses ${raw.format}; expected ${expectation.format}.`, + ); + } + + private _assertDispatchTextures( + color: THREE.Texture, + depth: THREE.Texture | undefined, + velocityTexture: THREE.Texture | undefined, + reactive: THREE.Texture | undefined, + ): void { + const width = this.resolver.renderWidth; + const height = this.resolver.renderHeight; + this._assertTexture({ name: 'dispatch.color', texture: color, width, height }); + if (depth) + this._assertTexture({ name: 'dispatch.depth', texture: depth, width, height }); + if (velocityTexture) + this._assertTexture({ + name: 'dispatch.velocity', + texture: velocityTexture, + width, + height, + }); + if (reactive) + this._assertTexture({ name: 'dispatch.reactive', texture: reactive, width, height }); + this._assertTexture({ + name: 'resolver.output', + texture: this.resolver.outputTexture as THREE.Texture, + width: this.resolver.displayWidth, + height: this.resolver.displayHeight, + }); } /** - * Renders one frame: scene → render target → FSR passes → canvas quad. + * Renders deterministic scene inputs without dispatching or presenting. * @param scene - Scene to render * @param camera - Scene camera - * @param deltaTime - Seconds since last frame + * @param reactiveScene - Optional Q5 particle-coverage scene */ - render(scene: THREE.Scene, camera: THREE.PerspectiveCamera, deltaTime: number): void { + renderInput( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + reactiveScene?: THREE.Scene, + ): void { const rt = this._renderTarget; if (!rt) return; const temporal = this._mode === 'upscale-temporal'; //* Scene Pass (jittered when temporal) - this.upscaler.beginFrame(camera); + this.resolver.beginFrame(camera); + if (this._effectPass) { + this._renderer.setMRT(null); + this._renderer.setRenderTarget(rt); + this._effectQuad.render(this._renderer); + this._renderer.setRenderTarget(null); + this.resolver.endFrame(camera); + return; + } // The MRT output count MUST match the render target's attachment count: // rendering into a count:2 target without the velocity output leaves // color attachment 0 unwritten (black). Non-temporal modes therefore @@ -144,23 +705,343 @@ export class BenchPipeline { this._renderer.render(scene, camera); this._renderer.setRenderTarget(null); this._renderer.setMRT(null); - this.upscaler.endFrame(camera); - //* FSR Passes - this.upscaler.dispatch( + //* Optional Q5 Reactive Coverage + if (reactiveScene && this._reactiveTarget) { + const autoClear = this._renderer.autoClear; + this._renderer.autoClear = false; + this._renderer.setMRT(this._mrtOutputOnly); + this._renderer.setRenderTarget(this._reactiveTarget); + this._renderer.clear(true, true, false); + + // Populate an opaque-only depth attachment first. Particles live on + // layer 1, so the manual coverage pass cannot flag occluded volume. + const overrideMaterial = scene.overrideMaterial; + const background = scene.background; + const cameraLayerMask = camera.layers.mask; + scene.overrideMaterial = this._depthOnlyMaterial; + scene.background = null; + camera.layers.disable(1); + this._renderer.render(scene, camera); + scene.overrideMaterial = overrideMaterial; + scene.background = background; + camera.layers.mask = cameraLayerMask; + this._renderer.render(reactiveScene, camera); + this._renderer.setRenderTarget(null); + this._renderer.setMRT(null); + this._renderer.autoClear = autoClear; + } + this.resolver.endFrame(camera); + } + + /** + * Dispatches the active resolver against the most recent input render. + * @param camera - Camera used for input rendering + * @param deltaTime - Fixed or interactive timestep + * @param frameTag - Deterministic frame identity for fresh timing + */ + /** + * Opts this pipeline into host pre-exposure driving (scenario Q11): the + * scene MRT color is scaled by the driven value — emulating an app that + * bakes its own exposure into the render — and the same value is fed to + * the resolver as `preExposureTexture` every dispatch. + */ + enableHostPreExposureDrive(): void { + this._mrtNode = mrt({ output: output.mul(this._hostPreExposure), velocity }); + this._mrtOutputOnly = mrt({ output: output.mul(this._hostPreExposure) }); + this._hostPreExposureValue = 1; + } + + /** + * Sets the host pre-exposure for the next rendered frame. Requires + * {@link enableHostPreExposureDrive}. + * @param value - Positive exposure factor baked into the scene color + */ + setHostPreExposure(value: number): void { + if (this._hostPreExposureValue === null) + throw new Error('Host pre-exposure drive is not enabled for this pipeline.'); + this._hostPreExposure.value = value; + this._hostPreExposureValue = value; + } + + // The 1×1 texture is created per distinct value: three only re-uploads + // texture data for textures its own render graph consumes, so mutating one + // texture's texels would never reach the GPU. initTexture() forces upload. + private _hostPreExposureTexture(value: number): THREE.DataTexture { + let tex = this._hostPreExposureTextures.get(value); + if (!tex) { + tex = new THREE.DataTexture( + new Float32Array([value, 0, 0, 1]), + 1, + 1, + THREE.RGBAFormat, + THREE.FloatType, + ); + tex.needsUpdate = true; + this._renderer.initTexture(tex); + this._hostPreExposureTextures.set(value, tex); + } + return tex; + } + + dispatchResolver( + camera: THREE.PerspectiveCamera, + deltaTime: number, + frameTag: number, + ): void { + const rt = this._renderTarget; + if (!rt) return; + const temporal = this._mode === 'upscale-temporal'; + const effectDepth = this._effectPass?.renderTarget.depthTexture ?? undefined; + const effectVelocity = this._effectPass?.getTexture('velocity'); + const color = rt.textures[0]; + const depth = effectDepth ?? rt.depthTexture ?? undefined; + const velocityTexture = this._effectPass + ? effectVelocity + : temporal + ? rt.textures[1] + : undefined; + const reactive = this._reactiveTarget?.textures[0]; + this._assertDispatchTextures(color, depth, velocityTexture, reactive); + this.resolver.dispatch( { - color: rt.textures[0], - depth: rt.depthTexture ?? undefined, - velocity: temporal ? rt.textures[1] : undefined, + color, + depth, + velocity: velocityTexture, + reactive, + preExposureTexture: + this._hostPreExposureValue !== null + ? this._hostPreExposureTexture(this._hostPreExposureValue) + : undefined, deltaTime, + frameTag, }, camera, ); + } - //* Present — output is already display-referred sRGB + /** Presents the resolver output without another transfer transform. */ + present(): void { this._quad.render(this._renderer); } + /** Advances the pinned NodeFrame exactly once for one automated frame. */ + advanceAutomatedFrame(frame: number): void { + const nodeFrame = this._nodeFrame(); + if (nodeFrame.frameId !== frame) + throw new Error( + `Pinned NodeFrame expected frameId ${frame} before update; got ${nodeFrame.frameId}.`, + ); + nodeFrame.update(); + nodeFrame.time = (frame + 1) / 60; + nodeFrame.deltaTime = 1 / 60; + nodeFrame.lastTime = performance.now(); + } + + /** + * Compiles and allocates the selected effect graph before recorded frame zero. + * @param camera - Effect scenario camera + */ + async prepareEffectReadiness(camera: THREE.PerspectiveCamera): Promise { + if (!this._effectScenario) return; + const backend = this._renderer.backend as unknown as { + device?: GPUDevice; + get(texture: THREE.Texture): { texture?: GPUTexture } | undefined; + }; + if (!backend.device || typeof backend.device.queue?.onSubmittedWorkDone !== 'function') + throw new Error('Pinned WebGPU backend device shape changed.'); + const nodes = this._nodeManager(); + this._resetEffectState(camera); + this._resetNodeFrame(0); + + let ready = false; + for (let readinessFrame = 0; readinessFrame < 180; readinessFrame++) { + this.advanceAutomatedFrame(readinessFrame); + this.renderInput(this._effectScenario.scene, camera); + this.dispatchResolver(camera, 1 / 60, -1 - readinessFrame); + this.present(); + await backend.device.queue.onSubmittedWorkDone(); + try { + this._effectTextures.forEach((expectation) => this._assertTexture(expectation)); + ready = nodes._buildQueue.length === 0 && nodes._buildInProgress === false; + } catch { + ready = false; + } + if (ready) break; + await new Promise((resolve) => setTimeout(resolve, 0)); + } + if (!ready) throw new Error('Effect graph readiness barrier timed out.'); + + await this.drainTiming(); + this.reset(this._effectScenario.scene, camera); + } + + private _nodeManager(): { + nodeFrame: NodeFrameBridge; + _buildQueue: unknown[]; + _buildInProgress: boolean; + } { + const nodes = (this._renderer as unknown as { _nodes?: unknown })._nodes as { + nodeFrame?: NodeFrameBridge; + _buildQueue?: unknown[]; + _buildInProgress?: boolean; + }; + if ( + !nodes?.nodeFrame || + !Array.isArray(nodes._buildQueue) || + typeof nodes._buildInProgress !== 'boolean' + ) + throw new Error('Pinned renderer._nodes readiness shape changed.'); + return nodes as { + nodeFrame: NodeFrameBridge; + _buildQueue: unknown[]; + _buildInProgress: boolean; + }; + } + + private _nodeFrame(): NodeFrameBridge { + const nodeFrame = this._nodeManager().nodeFrame; + if ( + !(nodeFrame.updateMap instanceof WeakMap) || + !(nodeFrame.updateBeforeMap instanceof WeakMap) || + !(nodeFrame.updateAfterMap instanceof WeakMap) || + typeof nodeFrame.update !== 'function' + ) + throw new Error('Pinned NodeFrame mutable shape changed.'); + return nodeFrame; + } + + private _resetEffectState(camera: THREE.PerspectiveCamera): void { + if (!this._effectScenario) return; + for (const ssrNode of this._effectSsrNodes) { + if (!ssrNode._noiseIndex || typeof ssrNode._noiseIndex.value !== 'number') + throw new Error('Pinned SSRNode._noiseIndex shape changed.'); + ssrNode._noiseIndex.value = 0; + } + if (this._effectScenario.id === 'Q8') + for (const sizedNode of this._effectSizedNodes) { + if (typeof sizedNode.setSize !== 'function') + throw new Error('Pinned recurrent effect setSize shape changed.'); + sizedNode.setSize(1, 1); + } + + for (const temporalNode of this._effectTemporalNodes) { + const candidate = temporalNode as { + _cameraUniforms?: { updateFromCamera(cameraValue: THREE.Camera): void }; + _noiseIndex?: { value: number }; + }; + if (candidate._cameraUniforms) { + if (typeof candidate._cameraUniforms.updateFromCamera !== 'function') + throw new Error('Pinned temporal camera-uniform shape changed.'); + candidate._cameraUniforms.updateFromCamera(camera); + candidate._cameraUniforms.updateFromCamera(camera); + } + if (candidate._noiseIndex) candidate._noiseIndex.value = 0; + } + } + + private _resetNodeFrame(frame: number): void { + const nodeFrame = this._nodeFrame(); + nodeFrame.frameId = frame; + nodeFrame.renderId = 0; + nodeFrame.time = frame / 60; + nodeFrame.deltaTime = 0; + nodeFrame.lastTime = performance.now(); + nodeFrame.updateMap = new WeakMap(); + nodeFrame.updateBeforeMap = new WeakMap(); + nodeFrame.updateAfterMap = new WeakMap(); + } + + private _seedVelocityHistory( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + ): void { + const velocityNode = velocity as unknown as { + update(frame: { + frameId: number; + camera: THREE.Camera; + object: THREE.Object3D; + }): void; + updateAfter(frame: { object: THREE.Object3D }): void; + }; + if ( + typeof velocityNode.update !== 'function' || + typeof velocityNode.updateAfter !== 'function' + ) + throw new Error('Pinned VelocityNode reset bridge shape changed.'); + + scene.updateMatrixWorld(true); + camera.updateMatrixWorld(true); + const objects: THREE.Object3D[] = []; + scene.traverseVisible((object) => { + if ( + (object as THREE.Mesh).isMesh || + (object as THREE.Line).isLine || + (object as THREE.Points).isPoints + ) + objects.push(object); + }); + for (let passIndex = 0; passIndex < 2; passIndex++) { + const frameId = this._velocitySeedFrame--; + for (const object of objects) { + velocityNode.update({ frameId, camera, object }); + velocityNode.updateAfter({ object }); + } + } + } + + /** + * Interactive convenience preserving the original one-call lifecycle. + * @param scene - Scene to render + * @param camera - Scene camera + * @param deltaTime - Seconds since the previous frame + * @param frameTag - Optional interactive frame tag + */ + render( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + deltaTime: number, + frameTag = 0, + ): void { + this.renderInput(scene, camera); + this.dispatchResolver(camera, deltaTime, frameTag); + this.present(); + } + + /** Waits until a fresh timing slot can accept another frame. */ + prepareTiming(): Promise { + return this.resolver.waitForTimingCapacity(); + } + + /** Drains all pending timestamp-query readbacks. */ + drainTiming(): Promise { + return this.resolver.drainTiming(); + } + + /** Returns fresh timing samples collected since the last take/reset. */ + takeTimingSamples(): BenchmarkGpuFrameSample[] { + return this.resolver.takeTimingSamples(); + } + + /** + * Clears all temporal state and reseeds three's velocity history. + * @param scene - Scene at the reset event's absolute transforms + * @param camera - Camera at the reset event's absolute transform + * @param frame - Declared scenario frame retained by event resets + */ + reset(scene?: THREE.Scene, camera?: THREE.PerspectiveCamera, frame = 0): void { + this.resolver.reset(); + this._resetNodeFrame(frame); + if (camera) this._resetEffectState(camera); + const activeScene = this._effectScenario?.scene ?? scene; + if (activeScene && camera) { + ( + this.resolver.unjitteredProjectionMatrix as THREE.Matrix4 + ).copy(camera.projectionMatrix); + this._seedVelocityHistory(activeScene, camera); + } + } + /** Applies runtime settings from the UI (no reconfigure needed). */ applySettings(settings: { sharpness: number; @@ -172,11 +1053,21 @@ export class BenchPipeline { detectShadingChanges: boolean; debugView: DebugView; }): void { - Object.assign(this.upscaler.settings, settings); + Object.assign(this.resolver.settings, settings); + // Debug buffers are already normalized visualization colors; only the + // final linear/HDR result should pass through presentation tone mapping. + this._quadMaterial.toneMapped = settings.debugView === DebugView.None; } dispose(): void { + this._effectPass?.dispose(); this._renderTarget?.dispose(); - this.upscaler.dispose(); + this._reactiveTarget?.dispose(); + this._quadMaterial.dispose(); + this._effectMaterial.dispose(); + this._depthOnlyMaterial.dispose(); + this._hostPreExposureTextures.forEach((tex) => tex.dispose()); + this._hostPreExposureTextures.clear(); + this.resolver.dispose(); } } diff --git a/bench/src/BenchScene.ts b/bench/src/BenchScene.ts index ce05f38..317c247 100644 --- a/bench/src/BenchScene.ts +++ b/bench/src/BenchScene.ts @@ -9,8 +9,15 @@ import * as THREE from 'three/webgpu'; */ export interface BenchScene { scene: THREE.Scene; + roomScene: THREE.Scene; + cornellScene: THREE.Scene; + reactiveScene: THREE.Scene; /** Advances animations. @param time - Elapsed seconds @param animate - Freeze toggle */ update(time: number, animate: boolean): void; + /** Applies a deterministic absolute scenario frame. */ + applyFrame(frame: BenchmarkFrameState): void; + /** Recreates all seeded Q5 particle constants. */ + resetDeterministicState(): void; } /** Builds the checkerboard+grid floor texture on a canvas (no asset deps). */ @@ -57,6 +64,8 @@ function createGridTexture(): THREE.CanvasTexture { */ export function createBenchScene(): BenchScene { const scene = new THREE.Scene(); + const roomScene = new THREE.Scene(); + const reactiveScene = new THREE.Scene(); scene.background = new THREE.Color(0x10141a); scene.fog = new THREE.Fog(0x10141a, 40, 90); @@ -66,6 +75,133 @@ export function createBenchScene(): BenchScene { scene.add(sun); scene.add(new THREE.HemisphereLight(0x9fb4d4, 0x2a2620, 0.9)); + //* Screen-Space Effect Room ============================================== + roomScene.background = new THREE.Color(0x0a0c10); + const roomSun = new THREE.DirectionalLight(0xfff2df, 3.2); + roomSun.position.set(8, 14, 6); + roomScene.add(roomSun); + roomScene.add(new THREE.HemisphereLight(0x9fb4d4, 0x2a2620, 0.9)); + roomScene.add(new THREE.AmbientLight(0x404860, 0.4)); + + const roomFloor = new THREE.Mesh( + new THREE.PlaneGeometry(60, 60), + new THREE.MeshStandardMaterial({ + color: 0x20242c, + metalness: 0.9, + roughness: 0.12, + }), + ); + roomFloor.rotation.x = -Math.PI / 2; + roomScene.add(roomFloor); + + const wallGeometry = new THREE.BoxGeometry(20, 10, 0.4); + const leftWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0xc0392b, roughness: 0.9 }), + ); + leftWall.position.set(-8, 5, -4); + leftWall.rotation.y = Math.PI / 2; + roomScene.add(leftWall); + const rightWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0x2ecc71, roughness: 0.9 }), + ); + rightWall.position.set(8, 5, -4); + rightWall.rotation.y = -Math.PI / 2; + roomScene.add(rightWall); + const backWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0x8a8f98, roughness: 0.9 }), + ); + backWall.position.set(0, 5, -12); + roomScene.add(backWall); + + for (let i = 0; i < 5; i++) { + const box = new THREE.Mesh( + new THREE.BoxGeometry(1.6, 2 + i * 0.5, 1.6), + new THREE.MeshStandardMaterial({ color: 0xd8d2c4, roughness: 0.6 }), + ); + box.position.set(-5 + i * 2.5, 1 + i * 0.25, -6 + (i % 2) * 3); + roomScene.add(box); + } + const roomBall = new THREE.Mesh( + new THREE.SphereGeometry(1.4, 48, 32), + new THREE.MeshStandardMaterial({ + color: 0xdfe6f0, + metalness: 0.5, + roughness: 0.15, + }), + ); + roomBall.position.set(2, 1.6, -2); + roomScene.add(roomBall); + + //* Cornell Convergence Room (Q12) ======================================= + // Mirrors the first consumer's still-camera repro (GUIDES-HANDOFF-RESPONSE + // report 3): an enclosed box lit by a shadow-casting point light. three's + // WebGPU point shadows use an IGN-dithered Vogel filter whose dither is + // SCREEN-anchored, so under camera jitter every penumbra texel re-rolls + // each frame — deliberately unstable input luminance. A converged temporal + // pipeline must hold a still image against exactly this. + const cornellScene = new THREE.Scene(); + cornellScene.background = new THREE.Color(0x05060a); + const cornellLight = new THREE.PointLight(0xfff4e5, 60, 0, 2); + cornellLight.position.set(0, 5.4, 0.4); + cornellLight.castShadow = true; + cornellLight.shadow.mapSize.set(1024, 1024); + cornellLight.shadow.bias = -0.004; + cornellScene.add(cornellLight); + cornellScene.add(new THREE.AmbientLight(0x8090b0, 0.25)); + + const cornellWall = (color: number, width: number, height: number) => { + const wall = new THREE.Mesh( + new THREE.PlaneGeometry(width, height), + new THREE.MeshStandardMaterial({ color, roughness: 0.95 }), + ); + wall.receiveShadow = true; + cornellScene.add(wall); + return wall; + }; + const cornellFloor = cornellWall(0xd8d4cc, 6, 6); + cornellFloor.rotation.x = -Math.PI / 2; + const cornellCeiling = cornellWall(0xd8d4cc, 6, 6); + cornellCeiling.rotation.x = Math.PI / 2; + cornellCeiling.position.y = 6; + const cornellBack = cornellWall(0xd8d4cc, 6, 6); + cornellBack.position.set(0, 3, -3); + const cornellLeft = cornellWall(0xb02020, 6, 6); + cornellLeft.rotation.y = Math.PI / 2; + cornellLeft.position.set(-3, 3, 0); + const cornellRight = cornellWall(0x1fa03a, 6, 6); + cornellRight.rotation.y = -Math.PI / 2; + cornellRight.position.set(3, 3, 0); + + const cornellBoxMaterial = new THREE.MeshStandardMaterial({ color: 0xd0ccc2, roughness: 0.9 }); + const cornellTall = new THREE.Mesh(new THREE.BoxGeometry(1.9, 3.6, 1.9), cornellBoxMaterial); + cornellTall.position.set(-1.05, 1.8, -0.7); + cornellTall.rotation.y = 0.3; + const cornellShort = new THREE.Mesh(new THREE.BoxGeometry(1.7, 1.7, 1.7), cornellBoxMaterial); + cornellShort.position.set(1.15, 0.85, 0.9); + cornellShort.rotation.y = -0.35; + for (const box of [cornellTall, cornellShort]) { + box.castShadow = true; + box.receiveShadow = true; + cornellScene.add(box); + } + + // Emissive ceiling panel — a bright thin region under the light, the kind + // of high-contrast edge the convergence meter is most sensitive to. + const cornellPanel = new THREE.Mesh( + new THREE.PlaneGeometry(2, 1.6), + new THREE.MeshStandardMaterial({ + color: 0x000000, + emissive: 0xfff4e5, + emissiveIntensity: 4, + }), + ); + cornellPanel.rotation.x = Math.PI / 2; + cornellPanel.position.set(0, 5.98, 0.4); + cornellScene.add(cornellPanel); + //* Floor const floor = new THREE.Mesh( new THREE.PlaneGeometry(120, 120), @@ -123,8 +259,63 @@ export function createBenchScene(): BenchScene { bulb.position.set(0, 5.5, -2); scene.add(bulb); - function update(time: number, animate: boolean): void { - if (!animate) return; + //* Seeded Transparency Fixture ============================================ + const particleCount = 128; + const particleGeometry = new THREE.SphereGeometry(0.06, 8, 6); + const particles = new THREE.InstancedMesh( + particleGeometry, + new THREE.MeshBasicMaterial({ + color: 0x7fdfff, + transparent: true, + opacity: 0.7, + blending: THREE.AdditiveBlending, + depthWrite: false, + }), + particleCount, + ); + const reactiveParticles = new THREE.InstancedMesh( + particleGeometry, + new THREE.MeshBasicMaterial({ + color: 0xffffff, + depthTest: true, + depthWrite: false, + }), + particleCount, + ); + particles.layers.set(1); + reactiveParticles.layers.set(1); + particles.visible = false; + reactiveParticles.visible = false; + scene.add(particles); + reactiveScene.add(reactiveParticles); + + const particleBase = new Float32Array(particleCount * 3); + const particlePhase = new Float32Array(particleCount); + const particleUp = new Float32Array(particleCount); + + function resetDeterministicState(): void { + let seed = 0x5eed1234; + const random = (): number => { + seed = (seed ^ ((seed << 13) >>> 0)) >>> 0; + seed = (seed ^ (seed >>> 17)) >>> 0; + seed = (seed ^ ((seed << 5) >>> 0)) >>> 0; + return seed / 4294967296; + }; + + for (let i = 0; i < particleCount; i++) { + const ux = random(); + const uy = random(); + const uz = random(); + const up = random(); + particleBase[i * 3] = -5 + 10 * ux; + particleBase[i * 3 + 1] = 0.7 + 4 * uy; + particleBase[i * 3 + 2] = -5 + 10 * uz; + particlePhase[i] = 2 * Math.PI * up; + particleUp[i] = up; + } + } + + function updateObjects(time: number): void { knots.forEach((knot, i) => { knot.rotation.x = time * 0.35 + i; knot.rotation.y = time * 0.5; @@ -139,5 +330,46 @@ export function createBenchScene(): BenchScene { }); } - return { scene, update }; + function updateParticles(time: number): void { + const matrix = new THREE.Matrix4(); + for (let i = 0; i < particleCount; i++) { + const phase = particlePhase[i]; + const up = particleUp[i]; + const x = particleBase[i * 3] + 0.35 * Math.sin(0.7 * time + phase); + const y = + particleBase[i * 3 + 1] + 0.6 * ((0.35 * time + up) % 1); + const z = + particleBase[i * 3 + 2] + 0.35 * Math.cos(0.7 * time + phase); + matrix.makeTranslation(x, y, z); + particles.setMatrixAt(i, matrix); + reactiveParticles.setMatrixAt(i, matrix); + } + particles.instanceMatrix.needsUpdate = true; + reactiveParticles.instanceMatrix.needsUpdate = true; + } + + resetDeterministicState(); + updateObjects(0); + updateParticles(0); + + function update(time: number, animate: boolean): void { + if (!animate) return; + updateObjects(time); + } + + function applyFrame(frame: BenchmarkFrameState): void { + updateObjects(frame.animateScene ? frame.sceneTime : 0); + updateParticles(frame.time); + particles.visible = frame.particlesVisible; + reactiveParticles.visible = frame.particlesVisible; + sun.intensity = frame.directionalIntensity; + // The Q11 host pre-exposure multiplier lives in the MRT output node, + // which the background never passes through — scale it here so the + // whole frame is uniformly pre-exposed like a real app's render. + (scene.background as THREE.Color) + .setHex(0x10141a) + .multiplyScalar(frame.hostPreExposure ?? 1); + } + + return { scene, roomScene, cornellScene, reactiveScene, update, applyFrame, resetDeterministicState }; } diff --git a/bench/src/benchmark/BenchmarkResolver.ts b/bench/src/benchmark/BenchmarkResolver.ts new file mode 100644 index 0000000..90c53bf --- /dev/null +++ b/bench/src/benchmark/BenchmarkResolver.ts @@ -0,0 +1,240 @@ +import type * as THREE from 'three/webgpu'; + +import { Upscaler } from '@pmndrs/upscaler'; +import { + RCAS_HOISTED_EXPOSURE_SHADER, + RCAS_LEGACY_SHADER, + RCAS_PER_TAP_SHADER, + RCAS_TONEMAP_SPACE_SHADER, +} from '../../../src/shaders/rcas'; + +interface BenchmarkTimerBridge { + readonly enabled: boolean; + setNextFrameTag(frameTag: number): void; + setAuthoritative(authoritative: boolean): void; + waitForAvailableSlot(): Promise; + drain(): Promise; + reset(): void; + takeSamples(): Array<{ + frameTag: number; + sequence: number; + passes: Array<{ label: string; milliseconds: number }>; + }>; +} + +/** + * Adapts the unchanged production upscaler to the benchmark lifecycle. + */ +export class BaselineBenchmarkResolver implements BenchmarkResolver { + readonly metadata: BenchmarkVariantMetadata; + + private readonly _upscaler: Upscaler; + + constructor( + renderer: THREE.WebGPURenderer, + metadata: BenchmarkVariantMetadata, + rcasShader?: string, + candidateBundle?: string, + ) { + this.metadata = metadata; + const options = { + renderer, + _rcasShader: rcasShader, + _candidateBundle: candidateBundle, + }; + this._upscaler = new Upscaler(options); + this._upscaler.init(); + if (typeof metadata.settings.rcasDenoise === 'boolean') + this._upscaler.settings.rcasDenoise = metadata.settings.rcasDenoise; + } + + get outputTexture(): THREE.Texture { + return this._upscaler.outputTexture; + } + + get renderWidth(): number { + return this._upscaler.renderWidth; + } + + get renderHeight(): number { + return this._upscaler.renderHeight; + } + + get displayWidth(): number { + return this._upscaler.displayWidth; + } + + get displayHeight(): number { + return this._upscaler.displayHeight; + } + + get upscaleRatio(): number { + return this._upscaler.upscaleRatio; + } + + get jitterPhaseCount(): number { + return this._upscaler.jitterPhaseCount; + } + + get timestampQuerySupported(): boolean { + return this._timer.enabled; + } + + get unjitteredProjectionMatrix(): THREE.Matrix4 { + return this._upscaler.unjitteredProjectionMatrix; + } + + get settings(): Record { + return this._upscaler.settings as unknown as Record; + } + + get timings(): ReadonlyMap { + return this._upscaler.gpuTimings; + } + + private get _timer(): BenchmarkTimerBridge { + // The benchmark bridge deliberately stays private to the bench. Normal + // library users retain the existing no-op/latest-map timing behavior. + return (this._upscaler as unknown as { _timer: BenchmarkTimerBridge })._timer; + } + + configure(config: BenchmarkResolverConfigure): void { + this._upscaler.configure({ + displayWidth: config.displayWidth, + displayHeight: config.displayHeight, + customUpscaleRatio: config.ratio, + path: config.path, + }); + } + + beginFrame(camera: unknown): void { + this._upscaler.beginFrame(camera as THREE.PerspectiveCamera); + } + + endFrame(camera: unknown): void { + this._upscaler.endFrame(camera as THREE.PerspectiveCamera); + } + + dispatch(inputs: BenchmarkResolverDispatch, camera: unknown): void { + this._timer.setNextFrameTag(inputs.frameTag); + this._upscaler.dispatch( + { + color: inputs.color as THREE.Texture, + depth: inputs.depth as THREE.Texture | undefined, + velocity: inputs.velocity as THREE.Texture | undefined, + reactive: inputs.reactive as THREE.Texture | undefined, + transparencyAndComposition: inputs.transparencyAndComposition as + | THREE.Texture + | undefined, + preExposureTexture: inputs.preExposureTexture as THREE.Texture | undefined, + deltaTime: inputs.deltaTime, + }, + camera as THREE.PerspectiveCamera, + ); + } + + reset(): void { + this._upscaler.resetHistory(); + this._timer.reset(); + } + + resetTiming(): void { + this._timer.reset(); + } + + setAuthoritativeTiming(authoritative: boolean): void { + this._timer.setAuthoritative(authoritative); + } + + waitForTimingCapacity(): Promise { + return this._timer.waitForAvailableSlot(); + } + + drainTiming(): Promise { + return this._timer.drain(); + } + + takeTimingSamples(): BenchmarkGpuFrameSample[] { + return this._timer.takeSamples(); + } + + dispose(): void { + this._upscaler.dispose(); + } +} + +/** + * Creates the unchanged local baseline resolver. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata for the selected identity + * @returns One baseline resolver instance + */ +export function createBaselineResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + RCAS_LEGACY_SHADER, + ); +} + +/** + * Creates the isolated FSR 3.1.5 RCAS numeric candidate. Pinned to the + * per-tap shader (the production form when these identities were measured) so + * their timings stay frozen; current production is `rcas-tonemap-space-v1`. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata for the candidate identity + * @returns One candidate resolver instance + */ +export function createRcasNumericParityResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + RCAS_PER_TAP_SHADER, + ); +} + +/** + * Creates an RCAS load-strategy experiment candidate (item 1 of + * bench/docs/NEXT-STEPS.md): production pipeline, only the RCAS shader varies. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata carrying the experiment identity + * @returns One candidate resolver instance + */ +export function createRcasExperimentResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + metadata.id === 'rcas-hoisted-exposure-v1' + ? RCAS_HOISTED_EXPOSURE_SHADER + : RCAS_TONEMAP_SPACE_SHADER, + ); +} + +/** + * Creates one of the cumulative source-style benchmark candidates. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata carrying the candidate bundle ID + * @returns One candidate resolver instance + */ +export function createSourceBundleResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + // The bundles run RCAS without FLAG_INPUT_REINHARD, where the per-tap + // and conditioned forms are identical — keep the measured identity. + RCAS_PER_TAP_SHADER, + metadata.id, + ); +} diff --git a/bench/src/benchmark/api.ts b/bench/src/benchmark/api.ts new file mode 100644 index 0000000..ec4efce --- /dev/null +++ b/bench/src/benchmark/api.ts @@ -0,0 +1,242 @@ +import type * as THREE from 'three/webgpu'; + +import { DebugView } from '@pmndrs/upscaler'; + +import type { BenchPipeline } from '../BenchPipeline'; +import type { BenchScene } from '../BenchScene'; +import { BenchmarkClock } from './clock'; +import { BenchmarkCollector } from './collector'; + +interface BenchmarkApiContext { + renderer: THREE.WebGPURenderer; + camera: THREE.PerspectiveCamera; + bench: BenchScene; + pipeline: BenchPipeline | null; + config: BenchmarkRunConfig; + metadata: BenchmarkVariantMetadata; + scenario: BenchmarkScenarioDefinition; + environment: BenchmarkEnvironment; + manifestDigest: string; + validation: BenchmarkValidationRecord[]; + resize(dimensions: BenchmarkDimensions): void; +} + +const DEBUG_VIEWS: Record = { + final: DebugView.None, + 'motion-vectors': DebugView.MotionVectors, + disocclusion: DebugView.Disocclusion, + 'accumulation-age': DebugView.AccumulationAge, + locks: DebugView.Locks, + exposure: DebugView.Exposure, + 'shading-change': DebugView.ShadingChange, + reactivity: DebugView.Reactivity, +}; + +/** + * Exact deterministic browser API consumed by the CDP runner. + */ +class BrowserBenchmarkApi implements UpscalerBenchmarkApi { + readonly ready = true; + readonly config: BenchmarkRunConfig; + readonly metadata: BenchmarkVariantMetadata; + + private readonly _context: BenchmarkApiContext; + private readonly _clock = new BenchmarkClock(); + private readonly _captures: BenchmarkCaptureResult[] = []; + private _result: BenchmarkResult; + + constructor(context: BenchmarkApiContext) { + this._context = context; + this.config = context.config; + this.metadata = context.metadata; + this._result = { + status: context.scenario.unsupported ? 'unsupported' : 'ready', + experiment: 'E00', + manifestDigest: context.manifestDigest, + config: context.config, + variant: context.metadata, + scenario: context.scenario.id, + unsupported: context.scenario.unsupported, + environment: context.environment, + timing: null, + captures: this._captures, + validation: context.validation, + }; + } + + get result(): BenchmarkResult { + return this._result; + } + + async step(frame?: number): Promise { + this._requireSupported(); + const target = frame ?? this._clock.frame; + if (!Number.isInteger(target) || target < 0) + throw new Error(`Benchmark frame must be a non-negative integer: ${target}`); + if (target < this._clock.frame) await this.reset(); + while (this._clock.frame <= target) await this._renderOne(this._clock.frame); + return target; + } + + async reset(): Promise { + const { pipeline, bench, config, scenario, camera } = this._context; + if (pipeline) await pipeline.drainTiming(); + const canvas = this._context.renderer.domElement; + const resized = + canvas.width !== config.dimensions.width || canvas.height !== config.dimensions.height; + if (resized) this._context.resize(config.dimensions); + if (resized && pipeline?.usesEffectGraph) + await pipeline.prepareEffectReadiness(this._context.camera); + bench.resetDeterministicState(); + const frameZero = scenario.frame(0); + this._applyFrameState(frameZero); + pipeline?.reset(this._inputScene(frameZero), camera, 0); + this._clock.reset(); + } + + async capture(request: BenchmarkCaptureRequest): Promise { + this._requireSupported(); + const pipeline = this._context.pipeline!; + if (!this._context.scenario.debugViews.includes(request.debugView)) + throw new Error( + `Debug view ${request.debugView} is not declared for ${this._context.scenario.id}.`, + ); + pipeline.applySettings({ + sharpness: 0.8, + rcasDenoise: + pipeline.resolver.metadata.settings.rcasDenoise === true || + ['Q6', 'Q7', 'Q8'].includes(this._context.scenario.id), + maxAccumulation: 24, + exposure: 1, + autoExposure: true, + lockThinFeatures: true, + detectShadingChanges: true, + debugView: DEBUG_VIEWS[request.debugView], + }); + await this.reset(); + await this.step(request.frame); + const device = ( + this._context.renderer.backend as unknown as { device?: GPUDevice } + ).device; + if (!device) throw new Error('WebGPU device unavailable before capture.'); + await device.queue.onSubmittedWorkDone(); + + const capture = { + scenario: this._context.scenario.id, + subrun: this.config.subrun, + ratio: this.config.ratio, + frame: request.frame, + debugView: request.debugView, + width: this._context.renderer.domElement.width, + height: this._context.renderer.domElement.height, + jitterPeriod: pipeline.resolver.jitterPhaseCount, + }; + this._captures.push(capture); + return capture; + } + + async run(options: BenchmarkRunOptions = {}): Promise { + if (this._context.scenario.unsupported) return this._result; + const pipeline = this._context.pipeline!; + const warmupFrames = options.warmupFrames ?? this.config.warmupFrames; + const sampleFrames = options.sampleFrames ?? this.config.sampleFrames; + if (!Number.isInteger(warmupFrames) || warmupFrames < 0) + throw new Error(`Invalid warmup frame count: ${warmupFrames}`); + if (!Number.isInteger(sampleFrames) || sampleFrames < 1) + throw new Error(`Invalid sample frame count: ${sampleFrames}`); + + pipeline.resolver.setAuthoritativeTiming(this.config.authoritativeTiming); + if (this.config.authoritativeTiming && !pipeline.resolver.timestampQuerySupported) + throw new Error('Performance mode requires timestamp-query support.'); + + await this.reset(); + for (let index = 0; index < warmupFrames; index++) await this.step(); + await pipeline.drainTiming(); + pipeline.takeTimingSamples(); + pipeline.resolver.resetTiming(); + + const firstMeasuredFrame = this._clock.frame; + const expectedFrames = Array.from( + { length: sampleFrames }, + (_, index) => firstMeasuredFrame + index, + ); + const collector = new BenchmarkCollector( + expectedFrames, + this.metadata.pipeline.timingPassLabels, + ); + for (let index = 0; index < sampleFrames; index++) await this.step(); + await pipeline.drainTiming(); + collector.add(pipeline.takeTimingSamples()); + const timing = collector.summarize(); + this._result = { ...this._result, status: 'complete', timing }; + + if ( + this.config.authoritativeTiming && + (timing.invalidityCount !== 0 || + timing.receivedFrames !== sampleFrames) + ) { + this._result = { ...this._result, status: 'failed' }; + throw new Error( + `Invalid fresh timing set: expected ${sampleFrames}, received ${timing.receivedFrames}, ` + + `missing ${timing.missingFrameCount}, invalidities ${timing.invalidityCount}.`, + ); + } + return this._result; + } + + private _requireSupported(): void { + const unsupported = this._context.scenario.unsupported; + if (unsupported) + throw new Error(`${unsupported.code}: ${unsupported.capability}: ${unsupported.reason}`); + if (!this._context.pipeline) throw new Error('Benchmark resolver is unavailable.'); + } + + private async _renderOne(frame: number): Promise { + const { pipeline, scenario, camera, bench, config } = this._context; + if (!pipeline) return; + const scenarioFrame = scenario.frame(Math.min(frame, scenario.endFrame)); + if (scenarioFrame.resize) this._context.resize(scenarioFrame.resize); + this._applyFrameState(scenarioFrame); + if (scenarioFrame.hostPreExposure !== undefined) + pipeline.setHostPreExposure(scenarioFrame.hostPreExposure); + if (scenarioFrame.resetHistory) pipeline.reset(bench.scene, camera, frame); + + await pipeline.prepareTiming(); + pipeline.advanceAutomatedFrame(frame); + pipeline.renderInput( + this._inputScene(scenarioFrame), + camera, + scenarioFrame.particlesVisible ? bench.reactiveScene : undefined, + ); + pipeline.dispatchResolver(camera, config.timestepSeconds, frame); + pipeline.present(); + this._clock.seek(frame + 1); + } + + /** Scene rendered as upscaler input for a frame (Q12 swaps in cornell). */ + private _inputScene(frame: BenchmarkFrameState): THREE.Scene { + return frame.scene === 'cornell' ? this._context.bench.cornellScene : this._context.bench.scene; + } + + private _applyFrameState(frame: BenchmarkFrameState): void { + const { camera, bench } = this._context; + camera.position.fromArray(frame.cameraPosition); + camera.up.set(0, 1, 0); + camera.lookAt(...frame.cameraTarget); + camera.updateProjectionMatrix(); + camera.updateMatrixWorld(true); + bench.applyFrame(frame); + bench.scene.updateMatrixWorld(true); + bench.roomScene.updateMatrixWorld(true); + bench.reactiveScene.updateMatrixWorld(true); + } +} + +/** + * Creates the globally exposed deterministic benchmark API. + * @param context - Initialized browser benchmark dependencies + * @returns Exact CDP-facing API + */ +export function createBenchmarkApi(context: BenchmarkApiContext): UpscalerBenchmarkApi { + return new BrowserBenchmarkApi(context); +} diff --git a/bench/src/benchmark/clock.ts b/bench/src/benchmark/clock.ts new file mode 100644 index 0000000..4f0d1b8 --- /dev/null +++ b/bench/src/benchmark/clock.ts @@ -0,0 +1,34 @@ +/** + * Integer-frame clock used by every automated E00 scenario. + */ +export class BenchmarkClock { + private _frame = 0; + + get frame(): number { + return this._frame; + } + + get time(): number { + return this._frame / 60; + } + + /** Returns the current frame and advances exactly once. */ + step(): number { + return this._frame++; + } + + /** Restores recorded frame zero without consulting wall time. */ + reset(): void { + this._frame = 0; + } + + /** + * Positions the clock at an exact integer frame. + * @param frame - Zero-based frame index + */ + seek(frame: number): void { + if (!Number.isInteger(frame) || frame < 0) + throw new Error(`Benchmark frame must be a non-negative integer: ${frame}`); + this._frame = frame; + } +} diff --git a/bench/src/benchmark/collector.ts b/bench/src/benchmark/collector.ts new file mode 100644 index 0000000..c42e9ff --- /dev/null +++ b/bench/src/benchmark/collector.ts @@ -0,0 +1,236 @@ +function percentile(values: readonly number[], quantile: number): number | null { + if (values.length === 0) return null; + const sorted = [...values].sort((a, b) => a - b); + const position = quantile * (sorted.length - 1); + const lower = Math.floor(position); + const upper = Math.ceil(position); + if (lower === upper) return sorted[lower]; + return sorted[lower] + (sorted[upper] - sorted[lower]) * (position - lower); +} + +function summarizePass( + label: string, + samples: number[], + expectedFrames: number, +): BenchmarkPassSummary { + return { + label, + samples, + median: percentile(samples, 0.5), + p95: percentile(samples, 0.95), + missingCount: expectedFrames - samples.length, + }; +} + +/** + * Collects complete fresh frame samples and derives manifest statistics. + */ +export class BenchmarkCollector { + private readonly _expectedFrameTags: readonly number[]; + private readonly _expectedPassLabels: readonly string[] | null; + private readonly _samples: BenchmarkGpuFrameSample[] = []; + + constructor(expectedFrameTags: readonly number[], expectedPassLabels?: readonly string[]) { + this._expectedFrameTags = [...expectedFrameTags]; + this._expectedPassLabels = expectedPassLabels + ? [...expectedPassLabels].sort() + : null; + } + + /** + * Adds fresh samples returned by the resolver timer. + * @param samples - Complete frame-tagged samples + */ + add(samples: readonly BenchmarkGpuFrameSample[]): void { + this._samples.push( + ...samples.map((sample) => ({ + ...sample, + passes: sample.passes.map((pass) => ({ ...pass })), + })), + ); + } + + /** Builds raw, missing, per-pass, and compute-sum statistics. */ + summarize(): BenchmarkTimingSummary { + const expected = new Set(this._expectedFrameTags); + const candidates = new Map(); + const invalidSamples: BenchmarkInvalidTimingSample[] = []; + const sequences = new Set(); + let duplicateFrameCount = 0; + let duplicateSequenceCount = 0; + let unexpectedFrameCount = 0; + let duplicatePassLabelCount = 0; + let invalidValueCount = 0; + + for (const sample of this._samples) { + if (sequences.has(sample.sequence)) { + duplicateSequenceCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-sequence', + detail: `Sequence ${sample.sequence} was received more than once.`, + }); + continue; + } + sequences.add(sample.sequence); + if (!expected.has(sample.frameTag)) { + unexpectedFrameCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'unexpected-frame', + detail: `Frame ${sample.frameTag} was not requested.`, + }); + continue; + } + if (candidates.has(sample.frameTag)) { + duplicateFrameCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-frame', + detail: `Frame ${sample.frameTag} was received more than once.`, + }); + continue; + } + + const labels = new Set(); + let structurallyValid = true; + for (const pass of sample.passes) { + if (labels.has(pass.label)) { + duplicatePassLabelCount++; + structurallyValid = false; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-pass-label', + detail: `Pass label ${pass.label} occurs more than once.`, + }); + } + labels.add(pass.label); + if (!Number.isFinite(pass.milliseconds) || pass.milliseconds < 0) { + invalidValueCount++; + structurallyValid = false; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'invalid-pass-value', + detail: `${pass.label} has invalid duration ${pass.milliseconds}.`, + }); + } + } + if (structurallyValid) candidates.set(sample.frameTag, sample); + } + + // The modal sorted label signature is the complete measured graph. This + // avoids allowing one malformed first/last frame to define completeness. + const signatureCounts = new Map(); + for (const sample of candidates.values()) { + const signature = sample.passes.map((pass) => pass.label).sort().join('\u0000'); + signatureCounts.set(signature, (signatureCounts.get(signature) ?? 0) + 1); + } + const expectedSignature = this._expectedPassLabels + ? this._expectedPassLabels.join('\u0000') + : ([...signatureCounts].sort( + (a, b) => b[1] - a[1] || a[0].localeCompare(b[0]), + )[0]?.[0] ?? ''); + const expectedPassLabels = this._expectedPassLabels + ? [...this._expectedPassLabels] + : expectedSignature + ? expectedSignature.split('\u0000') + : []; + const byFrame = new Map(); + let inconsistentPassSetCount = 0; + for (const [frameTag, sample] of candidates) { + const signature = sample.passes.map((pass) => pass.label).sort().join('\u0000'); + if (signature !== expectedSignature || expectedPassLabels.length === 0) { + inconsistentPassSetCount++; + invalidSamples.push({ + frameTag, + sequence: sample.sequence, + reason: 'inconsistent-pass-set', + detail: `Expected [${expectedPassLabels.join(', ')}], received [${sample.passes + .map((pass) => pass.label) + .sort() + .join(', ')}].`, + }); + continue; + } + byFrame.set(frameTag, sample); + } + + const passes = expectedPassLabels.map((label) => { + const values: number[] = []; + for (const frameTag of this._expectedFrameTags) { + const value = byFrame + .get(frameTag) + ?.passes.find((pass) => pass.label === label)?.milliseconds; + if (value !== undefined) values.push(value); + } + return summarizePass(label, values, this._expectedFrameTags.length); + }); + + const computeSums: number[] = []; + for (const frameTag of this._expectedFrameTags) { + const sample = byFrame.get(frameTag); + if (sample) computeSums.push( + sample.passes.reduce((sum, pass) => sum + pass.milliseconds, 0), + ); + } + + return { + expectedFrames: this._expectedFrameTags.length, + receivedFrames: byFrame.size, + missingFrameCount: this._expectedFrameTags.length - byFrame.size, + duplicateFrameCount, + duplicateSequenceCount, + unexpectedFrameCount, + duplicatePassLabelCount, + invalidValueCount, + inconsistentPassSetCount, + invalidityCount: + duplicateFrameCount + + duplicateSequenceCount + + unexpectedFrameCount + + duplicatePassLabelCount + + invalidValueCount + + inconsistentPassSetCount + + (this._expectedFrameTags.length - byFrame.size), + expectedPassLabels, + invalidSamples, + passes, + computeSum: summarizePass( + 'compute-sum', + computeSums, + this._expectedFrameTags.length, + ), + raw: [...byFrame.values()].sort((a, b) => a.frameTag - b.frameTag), + }; + } +} + +/** + * Serializes a timing result without dropping raw evidence. + * @param summary - Collector summary + * @returns Pretty JSON + */ +export function timingSummaryToJson(summary: BenchmarkTimingSummary): string { + return JSON.stringify(summary, null, 2); +} + +/** + * Serializes one row per fresh frame/pass plus compute sum. + * @param summary - Collector summary + * @returns RFC-4180-compatible CSV text + */ +export function timingSummaryToCsv(summary: BenchmarkTimingSummary): string { + const rows = ['frame,sequence,label,milliseconds']; + for (const sample of summary.raw) { + for (const pass of sample.passes) + rows.push(`${sample.frameTag},${sample.sequence},${pass.label},${pass.milliseconds}`); + const sum = sample.passes.reduce((total, pass) => total + pass.milliseconds, 0); + rows.push(`${sample.frameTag},${sample.sequence},compute-sum,${sum}`); + } + return `${rows.join('\n')}\n`; +} diff --git a/bench/src/benchmark/config.ts b/bench/src/benchmark/config.ts new file mode 100644 index 0000000..9b8ad50 --- /dev/null +++ b/bench/src/benchmark/config.ts @@ -0,0 +1,91 @@ +const RATIOS = [1, 1.5, 2, 3] as const; +const VARIANTS = [ + 'baseline', + 'local-baseline-5d6a65e', + 'local-baseline-through-e00-harness', + 'rcas-fsr315-limiter', + 'rcas-fsr315-numeric', + 'rcas-hoisted-exposure-v1', + 'rcas-tonemap-space-v1', + 'source-filter-bundle-v1', + 'source-structural-bundle-v1', + 'source-spd-resolver-bundle-v1', +] as const; +const SCENARIOS = ['Q0', 'Q1', 'Q2', 'Q3', 'Q4', 'Q5', 'Q6', 'Q7', 'Q8', 'Q9', 'Q10', 'Q11', 'Q12'] as const; + +function numberParam(params: URLSearchParams, name: string, fallback: number): number { + const raw = params.get(name); + if (raw === null) return fallback; + const value = Number(raw); + if (!Number.isFinite(value)) throw new Error(`Invalid benchmark ${name}: ${raw}`); + return value; +} + +function enumParam( + params: URLSearchParams, + name: string, + values: readonly T[], + fallback: T, +): T { + const raw = params.get(name); + if (raw === null) return fallback; + if (!values.includes(raw as T)) throw new Error(`Invalid benchmark ${name}: ${raw}`); + return raw as T; +} + +/** + * Parses the deterministic E00 browser configuration. + * @param search - URL query string, including or excluding the leading `?` + * @returns A validated benchmark configuration + */ +export function parseBenchmarkConfig(search = window.location.search): BenchmarkRunConfig { + const params = new URLSearchParams(search); + const mode = enumParam( + params, + 'benchMode', + ['interactive', 'performance', 'capture'] as const, + 'interactive', + ); + const ratio = numberParam(params, 'ratio', 2); + if (!RATIOS.includes(ratio as (typeof RATIOS)[number])) + throw new Error(`Unsupported benchmark ratio: ${ratio}`); + + const width = Math.floor(numberParam(params, 'width', mode === 'interactive' ? window.innerWidth : 1920)); + const height = Math.floor( + numberParam(params, 'height', mode === 'interactive' ? window.innerHeight : 1080), + ); + if (width < 1 || height < 1) throw new Error('Benchmark dimensions must be positive integers.'); + + const variant = enumParam(params, 'variant', VARIANTS, 'baseline'); + const comparison = enumParam(params, 'comparison', VARIANTS, 'baseline'); + const scenario = enumParam(params, 'scenario', SCENARIOS, 'Q1'); + const experiment = params.get('experiment') ?? 'E00'; + if (experiment !== 'E00') throw new Error(`Unsupported benchmark experiment: ${experiment}`); + + const warmupFrames = Math.floor(numberParam(params, 'warmup', 240)); + const sampleFrames = Math.floor(numberParam(params, 'samples', 600)); + if (warmupFrames < 0 || sampleFrames < 1) + throw new Error('Benchmark warmup must be non-negative and samples must be positive.'); + + return { + experiment: 'E00', + mode, + variant, + comparison, + scenario, + subrun: params.get('subrun'), + ratio, + dimensions: { + width, + height, + devicePixelRatio: mode === 'interactive' ? Math.min(window.devicePixelRatio, 2) : 1, + }, + timestepSeconds: 1 / 60, + warmupFrames, + sampleFrames, + authoritativeTiming: mode === 'performance', + }; +} + +/** Ratios accepted by the immutable E00 manifest. */ +export const BENCHMARK_RATIOS: readonly number[] = RATIOS; diff --git a/bench/src/benchmark/environment.ts b/bench/src/benchmark/environment.ts new file mode 100644 index 0000000..07e3979 --- /dev/null +++ b/bench/src/benchmark/environment.ts @@ -0,0 +1,78 @@ +import * as THREE from 'three/webgpu'; + +interface RendererBackendDetails { + isWebGPUBackend?: boolean; + device?: GPUDevice; + adapter?: GPUAdapter; +} + +/** Computes the immutable E00 manifest SHA-256 digest in the browser. */ +export async function getManifestDigest(): Promise { + const response = await fetch('/results/experiments/e00-harness.json', { cache: 'no-store' }); + if (!response.ok) throw new Error(`Unable to read E00 manifest: ${response.status}`); + const bytes = await response.arrayBuffer(); + const digest = await crypto.subtle.digest('SHA-256', bytes); + return [...new Uint8Array(digest)].map((byte) => byte.toString(16).padStart(2, '0')).join(''); +} + +/** + * Captures browser, adapter, WebGPU, and fixed-run metadata. + * @param renderer - Initialized renderer + * @param config - Validated E00 configuration + * @returns Environment metadata required by benchmark artifacts + */ +export function collectBenchmarkEnvironment( + renderer: THREE.WebGPURenderer, + config: BenchmarkRunConfig, +): Promise { + const backend = renderer.backend as RendererBackendDetails; + const device = backend.device; + return navigator.gpu + .requestAdapter() + .then((adapter) => { + const adapterInfo = backend.adapter?.info ?? adapter?.info; + const adapterName = adapterInfo + ? [ + adapterInfo.vendor, + adapterInfo.architecture, + adapterInfo.device, + adapterInfo.description, + ] + .filter(Boolean) + .join(' ') + : 'adapter-info-unavailable'; + return { + browser: navigator.userAgent, + operatingSystem: navigator.platform, + adapter: adapterName, + backend: backend.isWebGPUBackend === true ? 'WebGPU' : 'unknown', + webgpuFeatures: device ? [...device.features].sort() : [], + threeVersion: THREE.REVISION, + dimensions: { ...config.dimensions }, + ratio: config.ratio, + fixedTimestep: config.timestepSeconds, + }; + }); +} + +/** + * Reports asynchronous device loss through the validation log channel. + * @param renderer - Initialized WebGPU renderer + * @param records - Mutable benchmark validation record list + */ +export function monitorDeviceLoss( + renderer: THREE.WebGPURenderer, + records: BenchmarkValidationRecord[], +): void { + const backend = renderer.backend as RendererBackendDetails; + void backend.device?.lost.then((info) => { + const record = { + channel: 'GPUDevice.lost', + level: 'error', + text: `${info.reason}: ${info.message}`, + timestamp: performance.now(), + }; + records.push(record); + console.error('WebGPU device lost.', record); + }); +} diff --git a/bench/src/benchmark/scenarios.ts b/bench/src/benchmark/scenarios.ts new file mode 100644 index 0000000..0000c60 --- /dev/null +++ b/bench/src/benchmark/scenarios.ts @@ -0,0 +1,380 @@ +const BASE_POSITION = [9, 6, 12] as const; +const BASE_TARGET = [0, 1.6, 0] as const; +const ROOM_TARGET = [0, 3, -5] as const; + +function state( + frame: number, + cameraPosition: readonly [number, number, number] = BASE_POSITION, + cameraTarget: readonly [number, number, number] = BASE_TARGET, +): BenchmarkFrameState { + return { + frame, + time: frame / 60, + cameraPosition, + cameraTarget, + sceneTime: frame / 60, + animateScene: false, + directionalIntensity: 3.2, + resetHistory: false, + resize: null, + particlesVisible: false, + }; +} + +function baselineAnimated(frame: number): BenchmarkFrameState { + return { ...state(frame), animateScene: true }; +} + +function q2(frame: number): BenchmarkFrameState { + const u = frame / 239; + return state(frame, [9 - 3 * u, 6, 12 - 4 * u]); +} + +function q4(frame: number): BenchmarkFrameState { + const target = BASE_TARGET; + const radius = 15; + const height = 4.4; + const theta0 = Math.atan2(12, 9); + const theta119 = theta0 + 0.25; + const c119 = [ + target[0] + radius * Math.cos(theta119), + target[1] + height, + target[2] + radius * Math.sin(theta119), + ] as const; + + if (frame <= 119) { + const theta = theta0 + (0.25 * frame) / 119; + return state(frame, [ + target[0] + radius * Math.cos(theta), + target[1] + height, + target[2] + radius * Math.sin(theta), + ]); + } + + const c239 = [c119[0] - 8, c119[1], c119[2] - 6] as const; + if (frame <= 239) { + const u = (frame - 120) / 119; + return state(frame, [c119[0] - 8 * u, c119[1], c119[2] - 6 * u]); + } + + const startTheta = Math.atan2(c239[2] - target[2], c239[0] - target[0]); + const orbitRadius = Math.hypot(c239[0] - target[0], c239[2] - target[2]); + const orbitFrame = Math.min(frame, 359); + const theta = startTheta + (0.9 * (orbitFrame - 240)) / 119; + return state(frame, [ + target[0] + orbitRadius * Math.cos(theta), + c239[1], + target[2] + orbitRadius * Math.sin(theta), + ]); +} + +function roomMotion(frame: number): BenchmarkFrameState { + const time = frame / 60; + return state(frame, [7 * Math.sin(0.15 * time), 4, 9 + 1.5 * Math.cos(0.15 * time)], ROOM_TARGET); +} + +function q9(frame: number): BenchmarkFrameState { + let directionalIntensity = 3.2; + if (frame >= 60 && frame < 120) directionalIntensity = 8; + else if (frame >= 120 && frame < 180) + directionalIntensity = 8 - (6 * (frame - 120)) / 59; + return { ...state(frame), directionalIntensity }; +} + +function q11(frame: number): BenchmarkFrameState { + // Host pre-exposure transition on an otherwise static scene: identity, + // 2.5× step at 60, hold, ramp back to 1 over 120–179. With DeltaPreExposure + // correction the shading-change view stays black through all of it and + // accumulation age never resets; output brightness simply tracks the drive. + let hostPreExposure = 1; + if (frame >= 60 && frame < 120) hostPreExposure = 2.5; + else if (frame >= 120 && frame < 180) hostPreExposure = 2.5 - (1.5 * (frame - 120)) / 59; + return { ...state(frame), hostPreExposure }; +} + +function q10(frame: number): BenchmarkFrameState { + const afterCut = frame >= 60; + return { + ...baselineAnimated(frame), + cameraPosition: afterCut ? [-7, 4, 9] : BASE_POSITION, + resetHistory: frame === 60 || frame === 120 || frame === 180, + resize: + frame === 120 + ? { width: 1280, height: 720, devicePixelRatio: 1 } + : frame === 180 + ? { width: 1920, height: 1080, devicePixelRatio: 1 } + : null, + particlesVisible: false, + }; +} + +function q12(frame: number): BenchmarkFrameState { + // The first consumer's cornell repro pose (report 3): still camera looking + // straight into the box. The only per-frame variation is the upscaler's + // own jitter (plus the screen-anchored shadow dither it provokes). + return { ...state(frame, [0, 2.6, 8.8], [0, 2.6, 0]), scene: 'cornell' }; +} + +const SCENARIOS: Record = { + Q0: { + id: 'Q0', + name: 'input-debug-validation', + endFrame: 143, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '119'], + debugViews: [ + 'final', + 'motion-vectors', + 'disocclusion', + 'accumulation-age', + 'locks', + 'exposure', + 'shading-change', + 'reactivity', + ], + rois: { + full: [0, 0, 1, 1], + floor_grid: [0.05, 0.55, 0.9, 0.45], + fence_and_spheres: [0.05, 0.28, 0.9, 0.42], + }, + subruns: [], + unsupported: null, + frame: baselineAnimated, + }, + Q1: { + id: 'Q1', + name: 'static-convergence', + endFrame: 239, + captures: ['0', '1', '2', '4', '8', '16', '23', 'P-1', 'P', '2*P-1', '119', '239'], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { full: [0, 0, 1, 1], thin_features: [0.08, 0.3, 0.84, 0.55] }, + subruns: [], + unsupported: null, + frame: state, + }, + Q2: { + id: 'Q2', + name: 'slow-aliasing-dolly', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age', 'locks'], + rois: { floor_grid: [0, 0.48, 1, 0.52], fence: [0.05, 0.35, 0.9, 0.32] }, + subruns: [], + unsupported: null, + frame: q2, + }, + Q3: { + id: 'Q3', + name: 'object-motion-disocclusion', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + moving_spheres: [0.12, 0.2, 0.76, 0.58], + fence_silhouette: [0.05, 0.35, 0.9, 0.3], + }, + subruns: [], + unsupported: null, + frame: baselineAnimated, + }, + Q4: { + id: 'Q4', + name: 'camera-motion-hold', + endFrame: 479, + captures: [ + '0', '23', 'P-1', 'P', '2*P-1', '118', '119', '120', '121', '238', '239', + '240', '241', '358', '359', '360', '361', '383', '479', + ], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age', 'shading-change'], + rois: { full: [0, 0, 1, 1], thin_geometry: [0.05, 0.25, 0.9, 0.55] }, + subruns: [], + unsupported: null, + frame: q4, + }, + Q5: { + id: 'Q5', + name: 'seeded-transparency-reactivity', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'accumulation-age', 'locks', 'reactivity'], + rois: { + full: [0, 0, 1, 1], + particle_volume: [0.18, 0.1, 0.64, 0.72], + opaque_edges: [0.05, 0.35, 0.9, 0.45], + }, + subruns: [], + unsupported: null, + frame: (frame) => ({ ...state(frame), particlesVisible: true }), + }, + Q6: { + id: 'Q6', + name: 'isolated-screenspace-effects', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + floor_reflection: [0.05, 0.5, 0.9, 0.5], + wall_contact_and_bounce: [0.08, 0.08, 0.84, 0.6], + }, + subruns: ['gtao', 'ssr', 'ssgi'], + unsupported: null, + frame: (frame) => state(frame, [0, 4, 10], ROOM_TARGET), + }, + Q7: { + id: 'Q7', + name: 'in-graph-screenspace-composition', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + glossy_floor: [0.05, 0.48, 0.9, 0.52], + colored_walls: [0.05, 0.05, 0.9, 0.62], + }, + subruns: [], + unsupported: null, + frame: roomMotion, + }, + Q8: { + id: 'Q8', + name: 'recurrent-denoiser-characterization', + endFrame: 239, + captures: ['0', '1', '2', '4', '8', '16', '23', 'P-1', 'P', '2*P-1', '59', '119', '239'], + debugViews: ['final', 'motion-vectors', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + flat_walls: [0.08, 0.08, 0.84, 0.52], + occlusion_edges: [0.18, 0.24, 0.64, 0.54], + }, + subruns: ['builtin', 'spatial', 'recurrent'], + unsupported: null, + frame: roomMotion, + }, + Q9: { + id: 'Q9', + name: 'exposure-transition', + endFrame: 239, + captures: [ + '0', '23', 'P-1', 'P', '2*P-1', '59', '60', '61', '62', '64', '68', '76', + '83', '119', '120', '121', '149', '178', '179', '180', '181', '182', '184', + '188', '196', '203', '239', + ], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { + full: [0, 0, 1, 1], + lit_knots: [0.18, 0.16, 0.64, 0.38], + hdr_bulb: [0.43, 0.08, 0.14, 0.2], + }, + subruns: [], + unsupported: null, + frame: q9, + }, + Q10: { + id: 'Q10', + name: 'reset-cut-resize', + endFrame: 239, + captures: [ + '0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '60', '61', '62', '64', + '68', '76', '83', '119', '120', '121', '122', '124', '128', '136', '143', + '179', '180', '181', '182', '184', '188', '196', '203', '239', + ], + debugViews: [ + 'final', + 'motion-vectors', + 'disocclusion', + 'accumulation-age', + 'locks', + 'exposure', + 'shading-change', + ], + rois: { full: [0, 0, 1, 1], moving_silhouettes: [0.08, 0.18, 0.84, 0.58] }, + subruns: [], + unsupported: null, + frame: q10, + }, + Q11: { + id: 'Q11', + name: 'host-pre-exposure', + endFrame: 239, + captures: [ + '0', '23', 'P-1', 'P', '59', '60', '61', '62', '64', '68', '76', '83', + '119', '120', '121', '135', '149', '164', '179', '180', '181', '184', + '196', '203', '239', + ], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { + full: [0, 0, 1, 1], + lit_knots: [0.18, 0.16, 0.64, 0.38], + hdr_bulb: [0.43, 0.08, 0.14, 0.2], + }, + subruns: [], + unsupported: null, + frame: q11, + }, + Q12: { + id: 'Q12', + name: 'cornell-still-convergence', + // Long enough to prove sustained convergence, not just initial settle. + endFrame: 479, + captures: ['0', '1', '23', 'P-1', 'P', '2*P-1', '119', '239', '479'], + debugViews: [ + 'final', + 'motion-vectors', + 'disocclusion', + 'accumulation-age', + 'locks', + 'shading-change', + ], + rois: { + full: [0, 0, 1, 1], + box_silhouettes: [0.3, 0.35, 0.45, 0.5], + shadow_penumbra: [0.15, 0.6, 0.7, 0.35], + }, + subruns: [], + unsupported: null, + frame: q12, + }, +}; + +/** + * Returns a manifest-defined scenario and validates its subrun. + * @param id - Scenario ID + * @param subrun - Optional effect subrun + * @returns Immutable scenario contract + */ +export function getBenchmarkScenario( + id: BenchmarkScenarioId, + subrun: string | null = null, +): BenchmarkScenarioDefinition { + const scenario = SCENARIOS[id]; + if (scenario.subruns.length > 0 && (!subrun || !scenario.subruns.includes(subrun))) + throw new Error(`Scenario ${id} requires subrun: ${scenario.subruns.join(', ')}`); + if (scenario.subruns.length === 0 && subrun) + throw new Error(`Scenario ${id} does not define subruns.`); + return scenario; +} + +/** + * Resolves manifest frame expressions against a jitter period. + * @param expressions - Integer or `P` expressions from the scenario contract + * @param jitterPeriod - Active resolver jitter phase count + * @returns Ordered, unique, integer capture frames + */ +export function resolveCaptureFrames( + expressions: readonly string[], + jitterPeriod: number, +): number[] { + const values = expressions.map((expression) => { + if (/^\d+$/.test(expression)) return Number(expression); + if (expression === 'P') return jitterPeriod; + if (expression === 'P-1') return jitterPeriod - 1; + if (expression === '2*P-1') return 2 * jitterPeriod - 1; + throw new Error(`Unsupported capture frame expression: ${expression}`); + }); + return [...new Set(values)]; +} + +/** All immutable E00 scenario contracts. */ +export const BENCHMARK_SCENARIOS: Readonly> = + SCENARIOS; diff --git a/bench/src/benchmark/variants.ts b/bench/src/benchmark/variants.ts new file mode 100644 index 0000000..6ee2e95 --- /dev/null +++ b/bench/src/benchmark/variants.ts @@ -0,0 +1,273 @@ +import { + createBaselineResolver, + createRcasExperimentResolver, + createRcasNumericParityResolver, + createSourceBundleResolver, +} from './BenchmarkResolver'; + +const SUPPORTED_RATIOS = [1, 1.5, 2, 3] as const; +const RESOURCE_GRAPH = [ + 'scene-color-depth-velocity', + 'exposure', + 'reconstruct', + 'shading-change-pyramid', + 'accumulate-history-locks', + 'rcas-or-blit', + 'debug-or-output', +] as const; +const ASSEMBLED_CHUNKS = [ + 'constants', + 'color', + 'depth', + 'tonemap', + 'pass-body', +] as const; + +let activeResolverCount = 0; + +function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { + const sourceBundle = + id === 'source-filter-bundle-v1' || + id === 'source-structural-bundle-v1' || + id === 'source-spd-resolver-bundle-v1'; + const structural = + id === 'source-structural-bundle-v1' || id === 'source-spd-resolver-bundle-v1'; + const spdResolver = id === 'source-spd-resolver-bundle-v1'; + const rcasExperiment = + id === 'rcas-hoisted-exposure-v1' || id === 'rcas-tonemap-space-v1'; + const rcasLimiterParity = id === 'rcas-fsr315-limiter'; + const rcasNumericParity = rcasLimiterParity || id === 'rcas-fsr315-numeric' || rcasExperiment; + const rcasDenoise = id === 'rcas-fsr315-numeric'; + const sourceResourceGraph = spdResolver + ? [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth-farthest-luma', + 'luma-spd-frame-info', + 'depth-clip-motion-divergence', + 'signed-difference-shading-spd', + 'prepare-reactivity-accumulation-new-locks', + 'four-frame-luma-instability', + 'source-resolver-history-lock-alpha', + 'rcas-or-blit', + 'debug-or-output', + ] + : structural + ? [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth-farthest-luma', + 'exposure-history-frame-info', + 'depth-clip-motion-divergence', + 'prepare-reactivity-accumulation-new-locks', + 'source-filter-local-state', + 'rcas-or-blit', + 'debug-or-output', + ] + : [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth', + 'exposure-history-frame-info', + 'depth-clip', + 'source-current-history-filters', + 'rcas-or-blit', + 'debug-or-output', + ]; + const sourceTimingLabels = spdResolver + ? [ + 'prepareInputs', + 'lumaSpd', + 'depthClip', + 'shadingSpd', + 'shadingResolve', + 'prepareReactivity', + 'lumaInstability', + 'accumulate', + 'rcas', + ] + : structural + ? [ + 'prepareInputs', + 'exposure', + 'depthClip', + 'prepareReactivity', + 'accumulate', + 'rcas', + ] + : ['prepareInputs', 'exposure', 'depthClip', 'accumulate', 'rcas']; + return { + id, + name: spdResolver + ? 'Source SPD temporal resolver bundle v1' + : structural + ? 'Source structural inputs/reactivity bundle v1' + : id === 'source-filter-bundle-v1' + ? 'Source reconstruction/filter bundle v1' + : id === 'rcas-hoisted-exposure-v1' + ? 'RCAS with hoisted exposure load' + : id === 'rcas-tonemap-space-v1' + ? 'RCAS sharpening in tonemap space' + : rcasDenoise + ? 'FSR 3.1.5 RCAS limiter + denoise' + : rcasLimiterParity + ? 'FSR 3.1.5 RCAS lower limiter' + : id === 'local-baseline-through-e00-harness' + ? 'Local baseline through E00 harness' + : 'Local baseline 5d6a65e', + supportedRatios: SUPPORTED_RATIOS, + settings: { + path: 'temporal', + sharpness: 0.8, + rcasDenoise, + maxAccumulation: 24, + exposure: 1, + autoExposure: true, + lockThinFeatures: true, + detectShadingChanges: true, + }, + resourceGraph: sourceBundle ? sourceResourceGraph : RESOURCE_GRAPH, + pipeline: { + shaderKey: sourceBundle || rcasExperiment + ? id + : rcasNumericParity + ? rcasDenoise + ? 'rcas-fsr315-numeric' + : 'rcas-fsr315-limiter' + : 'local-baseline-5d6a65e', + pipelineKey: sourceBundle ? id : 'temporal-baseline', + assembledChunks: sourceBundle + ? [ + 'constants', + 'source-inputs', + 'source-filter', + ...(structural ? ['source-reactivity'] : []), + ...(spdResolver ? ['source-spd', 'source-resolver-state'] : []), + 'rcas-source-math', + ] + : ASSEMBLED_CHUNKS, + wgslOverrides: sourceBundle + ? { + motionInputAtDisplayResolution: false, + motionCancelJitter: false, + prepareStructuralSignals: structural, + depthClipMotionDivergence: structural, + reactiveUseComponentMax: true, + reactiveApplyThreshold: true, + reactiveBinary: false, + reactiveThreshold: 0.04, + reactiveScale: 2, + reactiveBinaryValue: 1, + } + : {}, + timingPassLabels: sourceBundle + ? sourceTimingLabels + : ['exposure', 'reconstruct', 'shadingChange', 'accumulate', 'rcas'], + }, + }; +} + +const DEFAULT_DEFINITIONS: BenchmarkVariantDefinition[] = [ + 'baseline', + 'local-baseline-5d6a65e', + 'local-baseline-through-e00-harness', +].map((id) => ({ + metadata: metadata(id as BenchmarkVariantId), + create: createBaselineResolver, +})); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-fsr315-limiter'), + create: createRcasNumericParityResolver, +}); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-fsr315-numeric'), + create: createRcasNumericParityResolver, +}); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-hoisted-exposure-v1'), + create: createRcasExperimentResolver, +}); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-tonemap-space-v1'), + create: createRcasExperimentResolver, +}); +for (const id of [ + 'source-filter-bundle-v1', + 'source-structural-bundle-v1', + 'source-spd-resolver-bundle-v1', +] as const) { + DEFAULT_DEFINITIONS.push({ + metadata: metadata(id), + create: createSourceBundleResolver, + }); +} + +/** + * Registry enforcing one active resolver across the page. + */ +export class SingleVariantRegistry { + private readonly _definitions = new Map(); + private _active: BenchmarkResolver | null = null; + + constructor(definitions: readonly BenchmarkVariantDefinition[] = DEFAULT_DEFINITIONS) { + for (const definition of definitions) { + if (this._definitions.has(definition.metadata.id)) + throw new Error(`Duplicate benchmark variant: ${definition.metadata.id}`); + this._definitions.set(definition.metadata.id, definition); + } + } + + /** + * Validates a variant without constructing its GPU graph. + * @param id - Requested variant identity + * @param ratio - Requested display/render ratio + * @returns Immutable variant metadata + */ + resolve(id: string, ratio: number): BenchmarkVariantMetadata { + const definition = this._definitions.get(id); + if (!definition) throw new Error(`Unknown benchmark variant: ${id}`); + if (!definition.metadata.supportedRatios.includes(ratio)) + throw new Error(`Variant ${id} does not support ratio ${ratio}.`); + return definition.metadata; + } + + /** + * Creates the page's sole active resolver. + * @param id - Requested variant identity + * @param ratio - Requested display/render ratio + * @param renderer - Initialized renderer passed to the resolver factory + * @returns The active resolver + */ + create(id: string, ratio: number, renderer: unknown): BenchmarkResolver { + const variantMetadata = this.resolve(id, ratio); + if (this._active || activeResolverCount !== 0) + throw new Error('A benchmark resolver is already active.'); + + const definition = this._definitions.get(id)!; + const resolver = definition.create(renderer, variantMetadata); + this._active = resolver; + activeResolverCount++; + return resolver; + } + + /** Disposes and releases the sole resolver lease. */ + disposeActive(): void { + if (!this._active) return; + this._active.dispose(); + this._active = null; + activeResolverCount--; + } + + /** + * Releases a resolver already disposed by its owning pipeline. + * @param resolver - Resolver whose GPU resources were explicitly disposed + */ + releaseDisposed(resolver: BenchmarkResolver): void { + if (this._active !== resolver) + throw new Error('Cannot release a resolver that is not the active registry instance.'); + this._active = null; + activeResolverCount--; + } +} + +/** Returns the number of active resolvers without requiring a GPU. */ +export function getActiveResolverCount(): number { + return activeResolverCount; +} diff --git a/bench/src/main.ts b/bench/src/main.ts index a507fa5..612cc6f 100644 --- a/bench/src/main.ts +++ b/bench/src/main.ts @@ -1,11 +1,20 @@ -import * as THREE from 'three/webgpu'; import { OrbitControls } from 'three/addons/controls/OrbitControls.js'; +import * as THREE from 'three/webgpu'; import { DebugView, QualityMode } from '@pmndrs/upscaler'; import { BenchPipeline } from './BenchPipeline'; import { createBenchScene } from './BenchScene'; import { createBenchUI, type BenchState } from './BenchUI'; +import { createBenchmarkApi } from './benchmark/api'; +import { parseBenchmarkConfig } from './benchmark/config'; +import { + collectBenchmarkEnvironment, + getManifestDigest, + monitorDeviceLoss, +} from './benchmark/environment'; +import { getBenchmarkScenario } from './benchmark/scenarios'; +import { SingleVariantRegistry } from './benchmark/variants'; //* WebGPU Guard const fatal = document.getElementById('fatal')!; @@ -16,7 +25,14 @@ if (!navigator.gpu) { throw new Error('WebGPU unavailable'); } -//* State +//* Deterministic Configuration ============================================== +const config = parseBenchmarkConfig(); +const automated = config.mode !== 'interactive'; +const scenario = getBenchmarkScenario(config.scenario, config.subrun); +const registry = new SingleVariantRegistry(); +// Validation occurs before renderer construction, so bad identities never render. +const variantMetadata = registry.resolve(config.variant, config.ratio); + const state: BenchState = { mode: 'upscale-temporal', quality: QualityMode.Performance, @@ -32,65 +48,131 @@ const state: BenchState = { autoOrbit: true, }; -// Cap DPR — the point of an upscaler bench is control over pixel counts. -const dpr = Math.min(window.devicePixelRatio, 2); +const dpr = config.dimensions.devicePixelRatio; const displaySize = () => ({ - width: Math.floor(window.innerWidth * dpr), - height: Math.floor(window.innerHeight * dpr), + width: automated ? config.dimensions.width : Math.floor(window.innerWidth * dpr), + height: automated ? config.dimensions.height : Math.floor(window.innerHeight * dpr), }); //* Renderer const renderer = new THREE.WebGPURenderer({ antialias: false }); +// Q12's cornell point light casts shadows; scenes without shadow-casting +// lights compile identical shaders, so every other scenario is unaffected. +renderer.shadowMap.enabled = true; renderer.setPixelRatio(dpr); -renderer.setSize(window.innerWidth, window.innerHeight); -// The FSR output pass already applies tonemapping + sRGB encoding in WGSL, so -// the presentation quad must go to the canvas untouched. -renderer.toneMapping = THREE.NoToneMapping; -renderer.outputColorSpace = THREE.LinearSRGBColorSpace; +renderer.setSize( + automated ? config.dimensions.width / dpr : window.innerWidth, + automated ? config.dimensions.height / dpr : window.innerHeight, + true, +); +// The upscaler stays linear/HDR; presentation belongs to the renderer. +renderer.toneMapping = THREE.ACESFilmicToneMapping; +renderer.outputColorSpace = THREE.SRGBColorSpace; document.body.appendChild(renderer.domElement); await renderer.init(); -const backend = renderer.backend as { isWebGPUBackend?: boolean }; -if (backend.isWebGPUBackend !== true) { +const backend = renderer.backend as { isWebGPUBackend?: boolean; device?: GPUDevice }; +if (backend.isWebGPUBackend !== true || !backend.device) { fatal.style.display = 'grid'; - fatal.textContent = 'three fell back to the WebGL backend — the FSR3 bench needs real WebGPU.'; - throw new Error('WebGL fallback active'); + fatal.textContent = 'three fell back to the WebGL backend — the bench needs real WebGPU.'; + throw new Error('WebGPU backend unavailable'); } +if (automated) { + const animation = (renderer as unknown as { _animation?: { stop(): void } })._animation; + if (!animation || typeof animation.stop !== 'function') + throw new Error('Pinned renderer._animation bridge shape changed.'); + // Automated frames advance the pinned NodeFrame bridge explicitly; leaving + // three's ambient RAF loop active would inject uncontrolled wall-clock ticks. + animation.stop(); +} +const validation: BenchmarkValidationRecord[] = []; +monitorDeviceLoss(renderer, validation); //* Scene & Camera const bench = createBenchScene(); -const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 200); +const camera = new THREE.PerspectiveCamera( + 50, + config.dimensions.width / config.dimensions.height, + 0.1, + 200, +); camera.position.set(9, 6, 12); -const controls = new OrbitControls(camera, renderer.domElement); -controls.target.set(0, 1.6, 0); -controls.enableDamping = true; +camera.lookAt(0, 1.6, 0); +camera.layers.enable(1); +if (scenario.id === 'Q6' || scenario.id === 'Q7' || scenario.id === 'Q8') { + camera.fov = 55; + camera.position.set(0, 4, scenario.id === 'Q6' ? 10 : 10.5); + camera.lookAt(0, 3, -5); + camera.updateProjectionMatrix(); +} +const controls = automated ? null : new OrbitControls(camera, renderer.domElement); +if (controls) { + controls.target.set(0, 1.6, 0); + controls.enableDamping = true; +} //* Pipeline & UI -const pipeline = new BenchPipeline(renderer); +const pipeline = scenario.unsupported + ? null + : new BenchPipeline( + renderer, + (activeRenderer) => registry.create(config.variant, config.ratio, activeRenderer), + variantMetadata, + ); +if (pipeline && (scenario.id === 'Q6' || scenario.id === 'Q7' || scenario.id === 'Q8')) + pipeline.configureEffectScenario(scenario.id, config.subrun, bench.roomScene, camera); +// Q11 drives an app-baked pre-exposure through the scene color + resolver. +if (pipeline && scenario.id === 'Q11') pipeline.enableHostPreExposureDrive(); function reconfigure(): void { + if (!pipeline) return; const { width, height } = displaySize(); pipeline.configure(width, height, state.mode, state.quality); } -createBenchUI(state, reconfigure, () => pipeline.upscaler.resetHistory()); -reconfigure(); - -window.addEventListener('resize', () => { - renderer.setSize(window.innerWidth, window.innerHeight); - camera.aspect = window.innerWidth / window.innerHeight; +function resizeBenchmark(dimensions: BenchmarkDimensions): void { + renderer.setPixelRatio(dimensions.devicePixelRatio); + renderer.setSize( + dimensions.width / dimensions.devicePixelRatio, + dimensions.height / dimensions.devicePixelRatio, + true, + ); + camera.aspect = dimensions.width / dimensions.height; camera.updateProjectionMatrix(); + pipeline?.configureBenchmark( + dimensions.width, + dimensions.height, + config.ratio, + scenario.id === 'Q5', + ); +} + +if (automated) { + resizeBenchmark(config.dimensions); + await pipeline?.prepareEffectReadiness(camera); + pipeline?.reset(bench.scene, camera, 0); +} +else { + createBenchUI(state, reconfigure, () => pipeline?.reset(bench.scene, camera)); reconfigure(); -}); + window.addEventListener('resize', () => { + renderer.setSize(window.innerWidth, window.innerHeight); + camera.aspect = window.innerWidth / window.innerHeight; + camera.updateProjectionMatrix(); + reconfigure(); + }); +} //* Stats Readout const statsEl = document.getElementById('stats')!; +if (automated) statsEl.style.display = 'none'; let frameCount = 0; let fpsAccum = 0; let statsClock = 0; let fps = 0; function updateStats(dt: number): void { + if (!pipeline) return; frameCount++; fpsAccum += dt; statsClock += dt; @@ -100,53 +182,87 @@ function updateStats(dt: number): void { fpsAccum = 0; statsClock = 0; - const u = pipeline.upscaler; + const resolver = pipeline.resolver; const lines = [ `mode ${state.mode}`, - `render ${u.renderWidth}×${u.renderHeight}`, - `display ${u.displayWidth}×${u.displayHeight} (${u.upscaleRatio.toFixed(2)}x)`, - `jitter ${u.jitterPhaseCount} phases`, + `render ${resolver.renderWidth}×${resolver.renderHeight}`, + `display ${resolver.displayWidth}×${resolver.displayHeight} (${resolver.upscaleRatio.toFixed(2)}x)`, + `jitter ${resolver.jitterPhaseCount} phases`, `fps ${fps.toFixed(0)}`, ]; - if (u.gpuTimings.size > 0) { + if (resolver.timings.size > 0) { lines.push('--- gpu (ms) ---'); let total = 0; - for (const [label, ms] of u.gpuTimings) { - lines.push(`${label.padEnd(10)}${ms.toFixed(3)}`); - total += ms; + for (const [label, milliseconds] of resolver.timings) { + lines.push(`${label.padEnd(10)}${milliseconds.toFixed(3)}`); + total += milliseconds; } lines.push(`${'total'.padEnd(10)}${total.toFixed(3)}`); } - statsEl.innerHTML = lines.map((l) => l.replace(/^(\S+)/, '$1')).join('\n'); + statsEl.innerHTML = lines.map((line) => line.replace(/^(\S+)/, '$1')).join('\n'); } +//* Automated API ============================================================= +const manifestDigest = await getManifestDigest(); +const environment = await collectBenchmarkEnvironment(renderer, config); +window.__UPSCALER_BENCH__ = createBenchmarkApi({ + renderer, + camera, + bench, + pipeline, + config, + metadata: variantMetadata, + scenario, + environment, + manifestDigest, + validation, + resize: resizeBenchmark, +}); + +let pageDisposed = false; +window.addEventListener( + 'pagehide', + () => { + if (pageDisposed) return; + pageDisposed = true; + renderer.setAnimationLoop(null); + controls?.dispose(); + if (pipeline) { + pipeline.dispose(); + registry.releaseDisposed(pipeline.resolver); + } + renderer.dispose(); + window.__UPSCALER_BENCH__ = undefined; + }, + { once: true }, +); + //* Main Loop const timer = new THREE.Timer(); +let interactiveFrame = 0; -renderer.setAnimationLoop(() => { - timer.update(); - const dt = Math.min(timer.getDelta(), 0.1); - const time = timer.getElapsed(); +if (!automated && pipeline && controls) { + renderer.setAnimationLoop(() => { + timer.update(); + const dt = Math.min(timer.getDelta(), 0.1); + const time = timer.getElapsed(); - if (state.autoOrbit) { - controls.autoRotate = true; + controls.autoRotate = state.autoOrbit; controls.autoRotateSpeed = 0.6; - } else { - controls.autoRotate = false; - } - controls.update(); - - bench.update(time, state.animate); - pipeline.applySettings({ - sharpness: state.sharpness, - rcasDenoise: state.rcasDenoise, - maxAccumulation: state.maxAccumulation, - exposure: state.exposure, - autoExposure: state.autoExposure, - lockThinFeatures: state.lockThinFeatures, - detectShadingChanges: state.detectShadingChanges, - debugView: state.debugView, + controls.update(); + + bench.update(time, state.animate); + pipeline.applySettings({ + sharpness: state.sharpness, + rcasDenoise: state.rcasDenoise, + maxAccumulation: state.maxAccumulation, + exposure: state.exposure, + autoExposure: state.autoExposure, + lockThinFeatures: state.lockThinFeatures, + detectShadingChanges: state.detectShadingChanges, + debugView: state.debugView, + }); + pipeline.render(bench.scene, camera, dt, interactiveFrame++); + updateStats(dt); }); - pipeline.render(bench.scene, camera, dt); - updateStats(dt); -}); +} diff --git a/bench/src/types/benchmark.d.ts b/bench/src/types/benchmark.d.ts new file mode 100644 index 0000000..fae9ea7 --- /dev/null +++ b/bench/src/types/benchmark.d.ts @@ -0,0 +1,281 @@ +declare type BenchmarkMode = 'interactive' | 'performance' | 'capture'; +declare type BenchmarkVariantId = + | 'baseline' + | 'local-baseline-5d6a65e' + | 'local-baseline-through-e00-harness' + | 'rcas-fsr315-limiter' + | 'rcas-fsr315-numeric' + | 'rcas-hoisted-exposure-v1' + | 'rcas-tonemap-space-v1' + | 'source-filter-bundle-v1' + | 'source-structural-bundle-v1' + | 'source-spd-resolver-bundle-v1'; +declare type BenchmarkScenarioId = + | 'Q0' + | 'Q1' + | 'Q2' + | 'Q3' + | 'Q4' + | 'Q5' + | 'Q6' + | 'Q7' + | 'Q8' + | 'Q9' + | 'Q10' + | 'Q11' + | 'Q12'; +declare type BenchmarkDebugView = + | 'final' + | 'motion-vectors' + | 'disocclusion' + | 'accumulation-age' + | 'locks' + | 'exposure' + | 'shading-change' + | 'reactivity'; + +declare interface BenchmarkDimensions { + width: number; + height: number; + devicePixelRatio: number; +} + +declare interface BenchmarkRunConfig { + experiment: 'E00'; + mode: BenchmarkMode; + variant: BenchmarkVariantId; + comparison: BenchmarkVariantId; + scenario: BenchmarkScenarioId; + subrun: string | null; + ratio: number; + dimensions: BenchmarkDimensions; + timestepSeconds: number; + warmupFrames: number; + sampleFrames: number; + authoritativeTiming: boolean; +} + +declare interface BenchmarkPipelineMetadata { + shaderKey: string; + pipelineKey: string; + assembledChunks: readonly string[]; + wgslOverrides: Readonly>; + timingPassLabels: readonly string[]; +} + +declare interface BenchmarkVariantMetadata { + id: BenchmarkVariantId; + name: string; + supportedRatios: readonly number[]; + settings: Readonly>; + resourceGraph: readonly string[]; + pipeline: BenchmarkPipelineMetadata; +} + +declare interface BenchmarkResolverConfigure { + displayWidth: number; + displayHeight: number; + ratio: number; + path: 'bilinear' | 'spatial' | 'temporal'; +} + +declare interface BenchmarkResolverDispatch { + color: unknown; + depth?: unknown; + velocity?: unknown; + reactive?: unknown; + transparencyAndComposition?: unknown; + preExposureTexture?: unknown; + deltaTime: number; + frameTag: number; +} + +declare interface BenchmarkResolver { + readonly metadata: BenchmarkVariantMetadata; + readonly outputTexture: unknown; + readonly renderWidth: number; + readonly renderHeight: number; + readonly displayWidth: number; + readonly displayHeight: number; + readonly upscaleRatio: number; + readonly jitterPhaseCount: number; + readonly timestampQuerySupported: boolean; + readonly unjitteredProjectionMatrix: unknown; + readonly settings: Record; + readonly timings: ReadonlyMap; + configure(config: BenchmarkResolverConfigure): void; + beginFrame(camera: unknown): void; + endFrame(camera: unknown): void; + dispatch(inputs: BenchmarkResolverDispatch, camera: unknown): void; + reset(): void; + resetTiming(): void; + setAuthoritativeTiming(authoritative: boolean): void; + waitForTimingCapacity(): Promise; + drainTiming(): Promise; + takeTimingSamples(): BenchmarkGpuFrameSample[]; + dispose(): void; +} + +declare type BenchmarkResolverFactory = ( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +) => BenchmarkResolver; + +declare interface BenchmarkVariantDefinition { + metadata: BenchmarkVariantMetadata; + create: BenchmarkResolverFactory; +} + +declare interface BenchmarkGpuPassSample { + label: string; + milliseconds: number; +} + +declare interface BenchmarkGpuFrameSample { + frameTag: number; + sequence: number; + passes: BenchmarkGpuPassSample[]; +} + +declare interface BenchmarkPassSummary { + label: string; + samples: number[]; + median: number | null; + p95: number | null; + missingCount: number; +} + +declare interface BenchmarkTimingSummary { + expectedFrames: number; + receivedFrames: number; + missingFrameCount: number; + duplicateFrameCount: number; + duplicateSequenceCount: number; + unexpectedFrameCount: number; + duplicatePassLabelCount: number; + invalidValueCount: number; + inconsistentPassSetCount: number; + invalidityCount: number; + expectedPassLabels: string[]; + invalidSamples: BenchmarkInvalidTimingSample[]; + passes: BenchmarkPassSummary[]; + computeSum: BenchmarkPassSummary; + raw: BenchmarkGpuFrameSample[]; +} + +declare interface BenchmarkInvalidTimingSample { + frameTag: number; + sequence: number; + reason: + | 'unexpected-frame' + | 'duplicate-frame' + | 'duplicate-sequence' + | 'duplicate-pass-label' + | 'invalid-pass-value' + | 'inconsistent-pass-set'; + detail: string; +} + +declare interface BenchmarkUnsupportedCapability { + code: string; + capability: string; + reason: string; +} + +declare interface BenchmarkScenarioDefinition { + id: BenchmarkScenarioId; + name: string; + endFrame: number; + captures: readonly string[]; + debugViews: readonly BenchmarkDebugView[]; + rois: Readonly>; + subruns: readonly string[]; + unsupported: BenchmarkUnsupportedCapability | null; + frame(frame: number): BenchmarkFrameState; +} + +declare interface BenchmarkFrameState { + frame: number; + time: number; + cameraPosition: readonly [number, number, number]; + cameraTarget: readonly [number, number, number]; + sceneTime: number; + animateScene: boolean; + directionalIntensity: number; + resetHistory: boolean; + resize: BenchmarkDimensions | null; + particlesVisible: boolean; + /** App-baked exposure factor driven into the scene color + resolver (Q11). */ + hostPreExposure?: number; + /** Scene rendered as the upscaler input; defaults to the main torture scene. */ + scene?: 'main' | 'cornell'; +} + +declare interface BenchmarkCaptureRequest { + frame: number; + debugView: BenchmarkDebugView; +} + +declare interface BenchmarkCaptureResult { + scenario: BenchmarkScenarioId; + subrun: string | null; + ratio: number; + frame: number; + debugView: BenchmarkDebugView; + width: number; + height: number; + jitterPeriod: number; +} + +declare interface BenchmarkRunOptions { + warmupFrames?: number; + sampleFrames?: number; +} + +declare interface BenchmarkValidationRecord { + channel: string; + level: string; + text: string; + timestamp: number; +} + +declare interface BenchmarkEnvironment { + browser: string; + operatingSystem: string; + adapter: string; + backend: string; + webgpuFeatures: string[]; + threeVersion: string; + dimensions: BenchmarkDimensions; + ratio: number; + fixedTimestep: number; +} + +declare interface BenchmarkResult { + status: 'ready' | 'complete' | 'unsupported' | 'failed'; + experiment: 'E00'; + manifestDigest: string; + config: BenchmarkRunConfig; + variant: BenchmarkVariantMetadata; + scenario: BenchmarkScenarioId; + unsupported: BenchmarkUnsupportedCapability | null; + environment: BenchmarkEnvironment; + timing: BenchmarkTimingSummary | null; + captures: BenchmarkCaptureResult[]; + validation: BenchmarkValidationRecord[]; +} + +declare interface UpscalerBenchmarkApi { + readonly ready: boolean; + readonly config: BenchmarkRunConfig; + readonly metadata: BenchmarkVariantMetadata; + readonly result: BenchmarkResult; + step(frame?: number): Promise; + reset(): Promise; + capture(request: BenchmarkCaptureRequest): Promise; + run(options?: BenchmarkRunOptions): Promise; +} + +interface Window { + __UPSCALER_BENCH__?: UpscalerBenchmarkApi; +} diff --git a/examples/12-temporal-guides/index.html b/examples/12-temporal-guides/index.html new file mode 100644 index 0000000..b641756 --- /dev/null +++ b/examples/12-temporal-guides/index.html @@ -0,0 +1,53 @@ + + + + + + 12 · Temporal Guides + + + +
+
disocclusion
+
dilated depth (eye-z)
+
final upscale
+
dilated motion (uv delta)
+ + + diff --git a/examples/12-temporal-guides/main.ts b/examples/12-temporal-guides/main.ts new file mode 100644 index 0000000..5dfce73 --- /dev/null +++ b/examples/12-temporal-guides/main.ts @@ -0,0 +1,187 @@ +import * as THREE from 'three/webgpu'; +import { fract, mix, mrt, output, step, texture, uv, vec3, vec4, velocity } from 'three/tsl'; + +import { MomentsPass, Upscaler } from '@pmndrs/upscaler'; + +import { bootRenderer, displaySize } from '../shared/boot'; +import { addStudioLighting, createGridFloor } from '../shared/props'; + +//* Temporal guides — the upscaler as a data-products provider. +// The frame is driven with the SPLIT dispatch: dispatchGuides() right after +// the G-buffer produces the geometry guides (dilated motion/depth, +// disocclusion) that other temporal effects (SSGI temporal passes, denoisers) +// can consume *before* the final color exists; dispatchUpscale() then +// finishes the frame. The 2×2 view samples the published guide textures +// straight from `upscaler.guides` as ordinary TSL texture() nodes — exactly +// the consumer contract (ping-ponged products are re-pointed per frame). + +const { renderer, dpr } = await bootRenderer(); + +//* Scene — moving knot + orbiting spheres over a grid floor, so motion, +//* disocclusion trails, and depth all have something to show. +const scene = new THREE.Scene(); +scene.background = new THREE.Color(0x10141a); +addStudioLighting(scene); +scene.add(createGridFloor()); + +const knot = new THREE.Mesh( + new THREE.TorusKnotGeometry(1.1, 0.34, 220, 28), + new THREE.MeshStandardMaterial({ color: 0xc0c8d8, metalness: 0.9, roughness: 0.22 }), +); +knot.position.y = 2; +scene.add(knot); + +const spheres: THREE.Mesh[] = []; +for (let i = 0; i < 3; i++) { + const sphere = new THREE.Mesh( + new THREE.SphereGeometry(0.5, 48, 24), + new THREE.MeshStandardMaterial({ color: 0xf59e0b, metalness: 0.2, roughness: 0.4 }), + ); + scene.add(sphere); + spheres.push(sphere); +} + +const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 200); + +//* Upscaler — raw driver (no UpscalePass: we need the split dispatch). +const upscaler = new Upscaler({ renderer }); +upscaler.init(); +// Motion vectors must be jitter-free. +velocity.setProjectionMatrix(upscaler.unjitteredProjectionMatrix); + +const mrtFull = mrt({ output, velocity }); +let rt: THREE.RenderTarget | null = null; + +function configure(): void { + const { width, height } = displaySize(dpr); + upscaler.configure({ + displayWidth: width, + displayHeight: height, + customUpscaleRatio: 2, + path: 'temporal', + }); + + rt?.dispose(); + const depthTexture = new THREE.DepthTexture(upscaler.renderWidth, upscaler.renderHeight); + depthTexture.type = THREE.FloatType; + rt = new THREE.RenderTarget(upscaler.renderWidth, upscaler.renderHeight, { + count: 2, // MUST match the MRT output count + type: THREE.HalfFloatType, + depthTexture, + }); + // MRT routes node outputs to attachments BY TEXTURE NAME. + rt.textures[0].name = 'output'; + rt.textures[1].name = 'velocity'; +} +configure(); + +//* Present — one quad, 2×2 quadrants sampling the published guides. +// top-left: disocclusion top-right: dilated depth +// bottom-left: final bottom-right: dilated motion +const tile = uv().mul(2); +const local = fract(tile); +const right = step(1.0, tile.x); +const top = step(1.0, tile.y); + +// Texture nodes are created once; ping-ponged guides (dilated depth) get +// their `.value` re-pointed every frame — the documented consumer pattern. +const finalNode = texture(upscaler.outputTexture, local); +const motionNode = texture(upscaler.guides.dilatedMotion, local); +const disocclusionNode = texture(upscaler.guides.disocclusion, local); +const depthNode = texture(upscaler.guides.dilatedDepth, local); + +function refreshGuideNodes(): void { + const guides = upscaler.guides; + finalNode.value = upscaler.outputTexture; + motionNode.value = guides.dilatedMotion; + disocclusionNode.value = guides.disocclusion; + depthNode.value = guides.dilatedDepth; +} + +const motionVis = vec3(motionNode.xy.mul(40.0).add(0.5), 0.5); +const disocclusionVis = vec3(disocclusionNode.r); +// Linear eye-Z → a readable gradient (near bright, far dark). +const depthVis = vec3(depthNode.r.div(depthNode.r.add(8.0)).oneMinus()); + +// Note: the quad's uv v axis runs top-down on screen, so `top = 1` selects +// the LOWER half — order the rows accordingly. +const quadColor = mix( + mix(disocclusionVis, depthVis, right), // screen top row + mix(finalNode.rgb, motionVis, right), // screen bottom row + top, +); +const quadMaterial = new THREE.NodeMaterial(); +quadMaterial.colorNode = vec4(quadColor, 1.0); +quadMaterial.depthTest = false; +quadMaterial.depthWrite = false; +quadMaterial.fog = false; +const quad = new THREE.QuadMesh(quadMaterial); + +const badge = document.getElementById('badge')!; +function updateBadge(): void { + const reconstruct = upscaler.gpuTimings.get('reconstruct'); + badge.innerHTML = + `@pmndrs/upscaler temporal guides (split dispatch)\n` + + `render ${upscaler.renderWidth}×${upscaler.renderHeight}\n` + + `display ${upscaler.displayWidth}×${upscaler.displayHeight} (${upscaler.upscaleRatio.toFixed(1)}x)\n` + + `guides dispatched post-G-buffer, pre-color\n` + + (reconstruct !== undefined ? `reconstruct ${reconstruct.toFixed(3)} ms` : ''); +} + +window.addEventListener('resize', () => { + renderer.setSize(window.innerWidth, window.innerHeight); + camera.aspect = window.innerWidth / window.innerHeight; + camera.updateProjectionMatrix(); + configure(); + // configure() reallocates the working set — re-point every guide node. + refreshGuideNodes(); +}); + +// Exposed for the headless GPU-verification harness (drives the guides-only +// path against this page's live renderer + render target). +Object.assign(window as unknown as Record, { + __guidesExample: { upscaler, renderer, camera, Upscaler, MomentsPass, THREE, getRenderTarget: () => rt }, +}); + +//* Loop — the split frame. +const timer = new THREE.Timer(); +renderer.setAnimationLoop(() => { + timer.update(); + const dt = Math.min(timer.getDelta(), 0.1); + const t = timer.getElapsed(); + + knot.rotation.y = t * 0.5; + knot.rotation.x = t * 0.35; + for (let i = 0; i < spheres.length; i++) { + const phase = t * 0.9 + (i * Math.PI * 2) / spheres.length; + spheres[i].position.set(Math.cos(phase) * 3.4, 1.1 + Math.sin(t * 1.3 + i) * 0.4, Math.sin(phase) * 3.4); + } + camera.position.set(Math.cos(t * 0.12) * 9, 4.5, Math.sin(t * 0.12) * 9); + camera.lookAt(0, 1.6, 0); + + //* 1. G-buffer: jittered scene render with color + velocity MRT. + upscaler.beginFrame(camera); + renderer.setMRT(mrtFull); + renderer.setRenderTarget(rt); + renderer.render(scene, camera); + renderer.setRenderTarget(null); + renderer.setMRT(null); + upscaler.endFrame(camera); + + //* 2. Early stage: geometry guides, valid from here on. + upscaler.dispatchGuides( + { depth: rt!.depthTexture!, velocity: rt!.textures[1], deltaTime: dt }, + camera, + ); + + // (A real pipeline runs its guide-consuming effects here — SSGI temporal + // reprojection, denoisers — then composites the final color.) + + //* 3. Late stage: accumulate + sharpen into outputTexture. + upscaler.dispatchUpscale({ color: rt!.textures[0], deltaTime: dt }, camera); + + //* 4. Present the quadrants (re-point ping-ponged guides first). + depthNode.value = upscaler.guides.dilatedDepth; + quad.render(renderer); + updateBadge(); +}); diff --git a/examples/13-guides-node/index.html b/examples/13-guides-node/index.html new file mode 100644 index 0000000..6f1c0a4 --- /dev/null +++ b/examples/13-guides-node/index.html @@ -0,0 +1,40 @@ + + + + + + 13 · Guides Node + + + +
+ + + diff --git a/examples/13-guides-node/main.ts b/examples/13-guides-node/main.ts new file mode 100644 index 0000000..911e86c --- /dev/null +++ b/examples/13-guides-node/main.ts @@ -0,0 +1,148 @@ +import * as THREE from 'three/webgpu'; +import { convertToTexture, mix, mrt, output, pass, texture, vec3, vec4, velocity } from 'three/tsl'; + +import { temporalGuides, upscale, type TemporalGuidesNode, type Upscaler } from '@pmndrs/upscaler'; + +import { bootRenderer, displaySize } from '../shared/boot'; +import { addStudioLighting, createGridFloor } from '../shared/props'; + +//* Temporal guides as a TSL node — one computation, two consumers. +// `temporalGuides(depth, velocity, camera)` publishes the upscaler's guide +// products into the post graph; a toy effect tints the pre-upscale color +// wherever `disocclusion` fires (orange trailing silhouettes), and +// `upscale(..., { guides })` SHARES the guides node's upscaler — the frame +// runs split (guides dispatch → effect renders → late upscale), so the +// reconstruct pass runs once and serves both. The imperative twin of this +// wiring is `examples/12-temporal-guides`. + +const { renderer, dpr } = await bootRenderer(); + +//* Scene — orbiting spheres + spinning knot, so disocclusion trails are +//* always live behind the movers. +const scene = new THREE.Scene(); +scene.background = new THREE.Color(0x10141a); +addStudioLighting(scene); +scene.add(createGridFloor()); + +const knot = new THREE.Mesh( + new THREE.TorusKnotGeometry(1.1, 0.34, 220, 28), + new THREE.MeshStandardMaterial({ color: 0xc0c8d8, metalness: 0.9, roughness: 0.22 }), +); +knot.position.y = 2; +scene.add(knot); + +const spheres: THREE.Mesh[] = []; +for (let i = 0; i < 3; i++) { + const sphere = new THREE.Mesh( + new THREE.SphereGeometry(0.5, 48, 24), + new THREE.MeshStandardMaterial({ color: 0xf59e0b, metalness: 0.2, roughness: 0.4 }), + ); + scene.add(sphere); + spheres.push(sphere); +} + +const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 200); + +//* Post graph — rebuilt on resize (the reduced-res sizes are baked in). +const post = new THREE.PostProcessing(renderer); +const RATIO = 2; +let guidesNode: TemporalGuidesNode | null = null; +let fsrNode: ReturnType | null = null; + +function configure(): void { + const { width, height } = displaySize(dpr); + const rw = Math.max(1, Math.floor(width / RATIO)); + const rh = Math.max(1, Math.floor(height / RATIO)); + + //* Reduced-res scene pass with the color + velocity MRT. + const scenePass = pass(scene, camera); + scenePass.setMRT(mrt({ output, velocity })); + scenePass.setResolutionScale(1 / RATIO); + + const beauty = scenePass.getTextureNode('output'); + const depth = scenePass.getTextureNode('depth'); + const vel = scenePass.getTextureNode('velocity'); + + // Dispose the previous graph's upscaler before replacing it. The linked + // guides node shares the upscale node's upscaler, so fsrNode owns it. + (fsrNode as unknown as { dispose?(): void } | null)?.dispose?.(); + (guidesNode as unknown as { dispose?(): void } | null)?.dispose?.(); + + //* The guides node — same depth/velocity/camera as the upscale below. + guidesNode = temporalGuides(depth, vel, camera); + + //* Toy consumer: paint the same-frame disocclusion product into the + //* pre-upscale color. Any guide-fed effect (SSGI temporal reprojection, + //* a denoiser's history rejection) slots in exactly here. + const disocclusion = guidesNode.getTextureNode('disocclusion'); + const tinted = mix(beauty.rgb, vec3(1.0, 0.45, 0.15), disocclusion.r.mul(0.85)); + + // Pin the effected color to render res (a bare convertToTexture would + // render it full-res — see 09), then upscale with the SHARED computation. + const colorTex = convertToTexture(vec4(tinted, beauty.a), rw, rh); + fsrNode = upscale(colorTex, depth, vel, camera, { ratio: RATIO, guides: guidesNode }); + post.outputNode = fsrNode as unknown as THREE.Node; + post.needsUpdate = true; +} +configure(); + +const badge = document.getElementById('badge')!; +function updateBadge(): void { + const u = (fsrNode as unknown as { upscaler?: Upscaler | null })?.upscaler; + const shared = u !== null && u !== undefined && guidesNode?.upscaler === u; + const reconstruct = u?.gpuTimings.get('reconstruct'); + badge.innerHTML = + `@pmndrs/upscaler temporalGuides() + upscale({ guides })\n` + + `orange = disocclusion guide, consumed pre-upscale\n` + + (u + ? `render ${u.renderWidth}×${u.renderHeight}\n` + + `display ${u.displayWidth}×${u.displayHeight} (${u.upscaleRatio.toFixed(1)}x)\n` + + `shared upscaler: ${shared ? 'yes (split frame)' : 'NO'}\n` + + (reconstruct !== undefined ? `reconstruct ${reconstruct.toFixed(3)} ms` : '') + : ''); +} + +window.addEventListener('resize', () => { + renderer.setSize(window.innerWidth, window.innerHeight); + camera.aspect = window.innerWidth / window.innerHeight; + camera.updateProjectionMatrix(); + configure(); +}); + +// Exposed for the headless GPU-verification harness (the tsl/THREE handles let +// it assemble a second, standalone guides graph against this live renderer). +Object.assign(window as unknown as Record, { + __guidesNodeExample: { + renderer, + camera, + scene, + THREE, + temporalGuides, + tsl: { pass, mrt, output, velocity, texture }, + get guidesNode() { + return guidesNode; + }, + get fsrNode() { + return fsrNode; + }, + }, +}); + +//* Loop — the nodes drive the split frame; we just render the post graph. +const timer = new THREE.Timer(); +renderer.setAnimationLoop(() => { + timer.update(); + const t = timer.getElapsed(); + + knot.rotation.y = t * 0.5; + knot.rotation.x = t * 0.35; + for (let i = 0; i < spheres.length; i++) { + const phase = t * 0.9 + (i * Math.PI * 2) / spheres.length; + spheres[i].position.set(Math.cos(phase) * 3.4, 1.1 + Math.sin(t * 1.3 + i) * 0.4, Math.sin(phase) * 3.4); + } + camera.position.set(Math.cos(t * 0.12) * 9, 4.5, Math.sin(t * 0.12) * 9); + camera.lookAt(0, 1.6, 0); + + post.render(); + updateBadge(); +}); diff --git a/examples/README.md b/examples/README.md index f70c1c3..3beb687 100644 --- a/examples/README.md +++ b/examples/README.md @@ -21,14 +21,21 @@ shader/pipeline edits hot-reload here just like in the bench. | 03 | **Split compare** (`03-split-compare`) | Native vs FSR3, same scene and instant, wiped by the mouse. | | 04 | **Aliasing torture** (`04-aliasing-torture`) | A chain-link fence + moiré floor under a moving camera — where naive upscaling shimmers and temporal holds. | | 05 | **Transparency & particles** (`05-transparency`) | The honest limitation: particles/transparents ghost (no depth/motion) — the acceptance test for a future reactive mask. | -| 06 | **Screen-space effects** (`06-screenspace-gi`) | GTAO / SSR / SSGI rendered at reduced resolution, then upscaled by FSR3. | - -Every interactive demo has a **render scale ×** slider (1.0×–3.0×) that sweeps the +| 06 | **Screen-space effects** (`06-screenspace-gi`) | GTAO / SSR / SSGI rendered at reduced resolution, then upscaled — the raw-`Upscaler` reference for imperative effect pipelines. | +| 07 | **TSL node** (`07-tsl-node`) | The whole upscaler as one line: `post.outputNode = upscaleScene(scene, camera)`. | +| 08 | **TSL compose** (`08-tsl-compose`) | The node composed with other TSL effects (`.mul(vignette)`) in the same post graph. | +| 09 | **Kitchen sink** (`09-kitchen-sink`) | The composable `upscale()` node driving a full SSGI+SSR stack rendered small, in one post graph, with jitter A/B. | +| 10 | **SSGI denoise** (`10-ssgi-denoise`) | Experimental documentation, not a feature: why a second temporal denoiser in front of FSR3 can't work (jitter-blind history rejection). | +| 11 | **Reactive mask (node)** (`11-node-reactive`) | The reactive mask through the composable node — an in-graph coverage pass, toggleable to A/B ghost trails. | +| 12 | **Temporal guides** (`12-temporal-guides`) | The upscaler as a data-products provider: the split `dispatchGuides()`/`dispatchUpscale()` frame, guide textures sampled live (raw driver). | +| 13 | **Guides node** (`13-guides-node`) | The same split frame, declaratively: `temporalGuides()` publishes the bundle into the graph, a toy effect consumes disocclusion pre-upscale, `upscale({ guides })` shares one computation. | + +Most interactive demos have a **render scale ×** slider (1.0×–3.0×) that sweeps the base render resolution, with the resulting size + base % shown in the HUD. ## Planned -- **07 · DPR budget** (`07-dpr`) — the mobile win, made explicit. Simulate a device +- **DPR budget** — the mobile win, made explicit. Simulate a device pixel ratio (e.g. a phone at DPR 1.5–3) and compare **native render at that DPR** vs **FSR: render at a lower effective resolution, present at the DPR output**. Show total pixels rendered *and* GPU ms for both sides so the saving is a number, @@ -58,12 +65,14 @@ inspector can't give you per-GPU-pass times. Notes for the DPR demo: ## How they're built -Every demo drives the library through [`shared/UpscalePresenter.ts`](shared/UpscalePresenter.ts), -which encapsulates the whole integration recipe (jitter-free velocity, MRT output -count matched to the render-target attachment count, float depth, the -`NoToneMapping` present). New demos should reuse it rather than re-deriving the -wiring — the one exception is `06`, which drives the raw `Upscaler` directly -so it can feed FSR3 the output of a TSL pass graph. +Demos `01`–`05` drive the library through [`shared/UpscalePresenter.ts`](shared/UpscalePresenter.ts), +which encapsulates the whole imperative integration recipe (jitter-free velocity, +MRT output count matched to the render-target attachment count, float depth, the +linear/HDR output, and renderer-owned presentation). `06` and `12` drive the raw +`Upscaler` directly (an external effect graph, and the split guides frame). +`07`–`11` and `13` are the TSL-node surface — no presenter at all, the node owns +the recipe inside the post graph. New imperative demos should reuse the presenter +rather than re-deriving the wiring; new graph demos should start from `07`. ### The `06` pattern (TSL effect graph → FSR3) diff --git a/examples/index.html b/examples/index.html index 4789b3b..d9ac99b 100644 --- a/examples/index.html +++ b/examples/index.html @@ -221,6 +221,28 @@

@pmndrs/upscaler — Examples

integration + +
12
+
Temporal guides
+
+ The upscaler as a data-products provider: the split + dispatchGuides() / dispatchUpscale() frame, with + the published dilated motion, disocclusion, and depth guides sampled live + as TSL texture nodes. +
+ integration + + +
13
+
Guides node
+
+ The guides bundle as TSL: temporalGuides() publishes the + products into the post graph, a toy effect tints disoccluded pixels + pre-upscale, and upscale({ guides }) shares one computation + across both — the split frame, declaratively. +
+ integration +