From 5d6a65e5681e5e95590f3e9a11ce75e43354ca13 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Thu, 16 Jul 2026 14:53:43 +0900 Subject: [PATCH 01/22] wip: pre-bench --- bench/DENOISING-DIRECTION.md | 526 +++++++++++ bench/THREE-TEMPORAL-COMPARISON.md | 1305 ++++++++++++++++++++++++++++ src/shaders/README.md | 895 +++++++++++++++++-- 3 files changed, 2652 insertions(+), 74 deletions(-) create mode 100644 bench/DENOISING-DIRECTION.md create mode 100644 bench/THREE-TEMPORAL-COMPARISON.md diff --git a/bench/DENOISING-DIRECTION.md b/bench/DENOISING-DIRECTION.md new file mode 100644 index 0000000..590b8b9 --- /dev/null +++ b/bench/DENOISING-DIRECTION.md @@ -0,0 +1,526 @@ +# Denoising direction + +## Status + +This document captures the current denoising discussion and a possible direction +for future experiments. It is an outline, not an approved implementation plan. + +The immediate recommendation is: + +- keep the upscaler as the only final-image temporal resolver; +- keep noisy SSGI and SSR signals separate long enough to denoise them with + effect-specific information; +- make any effect-level temporal reprojection aware of the exact projection + jitter used by the upscaler; +- apply spatial denoising primarily where temporal history is weak; +- consider a learned filter only after a conventional implementation provides a + measured baseline. + + +## The terms that are easy to conflate + +### Projection jitter + +Projection jitter moves the camera projection by a fraction of a pixel. It lets +the final temporal resolver collect different subpixel samples over multiple +frames. + +Exactly one component should own projection jitter for a render. In this +repository, that should normally be the upscaler. + +### Effect sampling noise + +SSGI, GTAO, stochastic SSR, and denoisers can rotate or shift their own sampling +patterns each frame. This changes ray or filter-tap locations without moving the +camera. + +These patterns do not need to use the upscaler's Halton sequence. They only need +predictable frame identity, reset behavior, and rendered cadence. + +### Spatial denoising + +A spatial denoiser combines neighboring pixels from the current frame. Depth, +normals, roughness, albedo, and ray metadata can prevent filtering across +unrelated surfaces. + +Spatial denoising helps immediately, including on the first frame, but excessive +filtering blurs detail and can create halos. + +### Temporal denoising + +A temporal denoiser reprojects an effect's previous result into the current +frame and accumulates it over time. + +It needs motion, depth, history rejection, and reset handling. In a jittered +pipeline it must also account for the current and previous projection offsets. + +### Final temporal AA/upscaling + +TRAA, TAAU, and this library's temporal path accumulate the final image. They +reduce aliasing and can average moderate effect noise, but they do not have the +effect-specific inputs needed to be ideal GI or reflection denoisers. + + +## What the repository currently uses + +### Examples 06 and 09 + +`examples/06-screenspace-gi` and `examples/09-kitchen-sink` use three's ordinary +`DenoiseNode` for SSGI and SSR. + +That node is spatial-only: + +- it filters the current frame; +- it uses depth and normals for edge stopping; +- it does not reproject history; +- it does not consume velocity; +- it does not jitter the camera. + +The resulting signal is composited into scene color and then accumulated by the +upscaler. + +### Example 10 + +`examples/10-ssgi-denoise` is explicitly experimental and compares three SSGI +paths: + +#### `builtin` + +Raw, temporally varying SSGI is composited into scene color. The upscaler owns +all temporal accumulation. + +This is the default path. + +#### `spatial` + +`RecurrentDenoiseNode` is used with: + +`accumulate: false` + +There is no `TemporalReprojectNode`, so the class runs as a spatial denoiser +despite its name. The upscaler remains the only temporal stage. + +#### `recurrent` + +The full chain is: + +```text +Raw SSGI + -> TemporalReprojectNode + -> RecurrentDenoiseNode { accumulate: true } + -> denoised output fed back as effect history + -> scene composite + -> temporal upscaler +``` + +This is genuinely temporal. + +The experiment produced the worst result under the upscaler's projection +jitter. Its velocity-only reprojection did not account for the changing +subpixel projection offset, so effect history could be misaligned before the +final temporal resolve. + +The experiment also exposes a separate integration risk: `UpscalerNode` and +`TemporalReprojectNode` assign the same scalar render-pipeline hooks and touch +shared velocity state. + +The current result does not prove that effect-level temporal denoising is +inherently incompatible with temporal upscaling. It shows that a second resolver +must share the projection-jitter and lifecycle contract. + + +## How SSGI temporal filtering relates to the upscaler + +`SSGINode.useTemporalFiltering` does not maintain SSGI history. It changes the +SSGI ray pattern over time so a downstream temporal resolver can average the +samples. + +The flow is: + +```text +Jittered scene render + -> SSGI produces a different noisy estimate + -> SSGI is composited into scene color + -> final temporal resolver reprojects previous scene color + -> noisy GI contribution is averaged with the rest of the pixel +``` + +SSGI's sample generation does not require velocity. The downstream temporal +resolver uses velocity to align the composited result. + +This can reduce the amount of spatial filtering required, but it does not +guarantee that spatial denoising becomes unnecessary. Newly revealed areas, +moving surfaces, rejected history, and the first frame still have little or no +usable temporal history. + + +## What `RecurrentDenoiseNode` contributes + +The current three.js implementation contains useful effect-specific ideas: + +- separate diffuse and specular modes; +- depth-, normal-, roughness-, and albedo-aware edge stopping; +- SSR ray-length handling; +- spatial radius based on history confidence; +- stronger filtering for young or unreliable history; +- firefly suppression; +- disocclusion smoothing; +- temporally varying analytic R² kernel rotation; +- optional temporal blending. + +Its changing R² kernel is effect sampling noise, not camera projection jitter. + +For temporal operation, the node relies on a reprojected input and feedback +history. The important transferable idea is not to insert this node unchanged, +but to coordinate effect history with the same jitter, motion, reset, and frame +identity used by the upscaler. + + +## What n8AO's neural denoiser does + +Source: + +- [NeuralDenoise.js](https://github.com/N8python/n8ao/blob/master/src/NeuralDenoise.js) +- [PoissionBlur.js](https://github.com/N8python/n8ao/blob/master/src/PoissionBlur.js) + +n8AO's neural stage is not a temporal neural denoiser. It is a compact learned +spatial correction inside the second Poisson blur iteration. + +At a high level it: + +1. gathers 4, 8, or 16 neighboring AO samples; +2. encodes local position, normal, occlusion, and distance features; +3. runs a small int8 attention model; +4. predicts a scalar residual; +5. adds that residual to the conventionally filtered AO result. + +The model does not use: + +- motion vectors; +- previous-frame textures; +- camera projection jitter; +- temporal reprojection. + +n8AO has a separate accumulation path, but it accumulates only while camera +matrices remain unchanged. Camera movement resets that history instead of +reprojecting it. + +The bundled neural weights are specific to n8AO's AO estimator, feature layout, +sample counts, and training distribution. Reusing those weights for SSGI, SSR, +or final scene color would not be valid. + +The implementation also does not receive special neural hardware acceleration. +Its int8 weights are emitted into a generated GLSL program and evaluated as +ordinary shader math. Actual cost depends on shader compilation, register +pressure, generated instruction count, and GPU scheduling. + + +## Where denoising could improve this project + +### Final scene color + +A general denoiser over final scene color is not the recommended first step. + +After SSGI or SSR has been composited into RGB, the upscaler cannot reliably +distinguish stochastic noise from: + +- texture detail; +- foliage; +- thin geometry; +- specular highlights; +- particles; +- intentional film grain. + +A generic filter would risk blur, haloing, and lost material detail across every +application, even when no noisy effect is present. + +### Separate noisy effect signals + +Effect-specific denoising has a clearer potential gain. + +Keeping SSGI or SSR separate allows the resolver to use information that final +RGB no longer contains: + +- normal; +- depth; +- roughness and metalness; +- albedo; +- AO value; +- reflection ray length; +- effect-specific variance; +- effect history age. + +That can improve quality or permit fewer expensive SSGI/SSR samples. + +### Low-confidence pixels + +The most promising spatial-denoise policy is to spend filtering work where +temporal history is weak: + +- newly disoccluded pixels; +- rejected history; +- low accumulation age; +- high local variance; +- reactive or rapidly changing pixels; +- pixels whose history was heavily clipped. + +Stable pixels with strong history should need less spatial filtering and a +smaller radius. + + +## Recommended experimental architecture + +The preferred experimental flow is: + +```text +Raw SSGI or SSR + -> jitter-aware effect reprojection + -> effect-specific temporal history and rejection + -> confidence-driven spatial denoise + -> composite with scene color + -> final temporal AA/upscale +``` + +This keeps effect denoising and final image reconstruction as separate concerns +while placing both on the same temporal timeline. + +### Required effect inputs + +Common: + +- raw effect color or scalar; +- depth; +- normal; +- velocity; +- current and previous projection jitter; +- reset generation; +- render dimensions; +- frame/sample identity. + +For SSGI: + +- albedo or diffuse color; +- optional direct/indirect separation; +- effect variance or confidence. + +For SSR: + +- roughness; +- metalness; +- hit distance or ray length; +- optional environment-hit classification. + +### Required temporal state + +Each independently denoised effect would need: + +- effect history; +- history age or confidence; +- previous depth; +- current and previous camera transforms; +- current and previous projection offsets. + +The effect resolver must use the same motion convention and reset cadence as the +final upscaler. + +### Spatial fallback + +Start with a small conventional kernel: + +- 5 to 8 taps; +- depth/plane-distance rejection; +- normal rejection; +- effect-specific material rejection; +- radius controlled by history confidence; +- stronger firefly suppression for young history; +- no filtering across disocclusions. + +This establishes a readable and tunable baseline before considering a learned +filter. + + +## Integration options + +### Option A: separate jitter-aware effect resolver + +Build a small reusable effect-history stage that consumes the application's +shared temporal context. + +**Advantages** + +- Clear separation from final AA/upscaling. +- Can use effect-specific inputs. +- Easier to compare or disable. +- Matches how dedicated GI/reflection denoisers are commonly structured. + +**Risks** + +- Additional history textures and passes. +- Requires careful scheduling. +- Can still create double-history lag if both stages are overly conservative. + +### Option B: fuse effect history into the upscaler + +Add dedicated SSGI or SSR inputs and resolve their histories inside the +upscaler's temporal pipeline. + +**Advantages** + +- One jitter and motion implementation. +- One reset lifecycle. +- Can share disocclusion and confidence signals. + +**Risks** + +- Couples the general upscaler to specific rendering effects. +- Adds inputs, textures, passes, and tuning to the core path. +- Makes the API harder to use for applications without those effects. +- Diffuse GI and specular reflection need materially different filtering. + +This should be treated as a focused research path, not a default extension. + +### Option C: final-color denoising + +Filter the accumulated or reconstructed final scene color. + +**Advantages** + +- Simple integration. +- No additional effect buffers. + +**Risks** + +- Cannot distinguish noise from detail. +- Applies cost and blur to applications that do not need denoising. +- Lacks normal, roughness, ray, and effect-confidence information. + +This is not recommended as the initial direction. + + +## Neural filtering considerations + +A neural filter becomes interesting if a conventional filter cannot provide the +desired quality at an acceptable tap count. + +Before pursuing it, the project would need: + +- a narrowly defined signal, such as diffuse GI or AO; +- representative training scenes and camera motion; +- noisy input and high-sample reference pairs; +- HDR-aware feature normalization; +- jitter-aware temporal examples; +- disocclusion and transparency coverage; +- separate training or features for diffuse and specular signals; +- a WebGPU implementation with measured shader size and register pressure. + +Potential benefits: + +- better edge preservation at a fixed tap count; +- learned rejection of recurring noise patterns; +- fewer SSGI/SSR rays for similar output quality. + +Potential costs: + +- model-training and dataset maintenance; +- content-dependent failure modes; +- difficult debugging; +- shader compilation and register pressure; +- no guarantee of being cheaper than a small hand-written kernel; +- new provenance, packaging, and model-version responsibilities. + +The n8AO implementation is evidence that a compact shader-resident model is +possible. It is not evidence that its model or architecture will generalize to +this upscaler. + + +## Benchmark outline + +Compare one change at a time. + +### Baselines + +1. Raw SSGI/SSR into the temporal upscaler. +2. Existing spatial `DenoiseNode` into the temporal upscaler. +3. `RecurrentDenoiseNode` with `accumulate: false`. +4. Current full recurrent experiment. + +### Proposed variants + +5. Jitter-aware temporal effect reprojection without spatial filtering. +6. Jitter-aware reprojection plus confidence-driven spatial filtering. +7. Reduced effect sample count plus the proposed denoiser. +8. Learned spatial correction only after the conventional path is understood. + +### Required scenes + +- static convergence; +- slow camera movement; +- fast translation; +- rotating camera; +- independently moving objects; +- disocclusion; +- thin geometry; +- glossy and rough reflections; +- diffuse GI around depth and normal edges; +- bright fireflies; +- camera cuts and reset; +- transparency where relevant. + +### Measurements + +- effect pass GPU time; +- denoiser GPU time; +- final temporal-upscale GPU time; +- combined frame GPU cost; +- temporal variance after convergence; +- disocclusion recovery time; +- visible trail length; +- retained edge/detail contrast; +- required SSGI/SSR sample count; +- memory footprint; +- compile and first-frame cost. + +Quality comparisons should use deterministic camera paths, fixed random seeds, +matched effect settings, clean history, and identical output transforms. + + +## Suggested next steps + +1. Preserve example 10 as the documented baseline for the current recurrent + conflict. + +2. Make a bench-only effect reprojection experiment that consumes exact current + and previous projection jitter. + +3. Verify reprojection in isolation before adding temporal blending or spatial + denoising. + +4. Add effect-specific history age, rejection, and reset behavior. + +5. Add a small confidence-driven spatial kernel. + +6. Compare raw, spatial-only, recurrent, and jitter-aware variants at equal + SSGI/SSR sample counts. + +7. Reduce effect samples and determine whether denoising produces a net GPU + saving rather than only an image-quality improvement. + +8. Decide between a separate reusable effect resolver and a fused experimental + path. + +9. Investigate a learned filter only if the conventional kernel is a measured + quality or performance bottleneck. + + +## Current recommendation + +There is a plausible quality and performance gain in actual denoising, especially +if it permits lower SSGI or stochastic SSR sample counts. + +The highest-value target is not a general denoiser over the upscaler's final +color. It is a jitter-aware, effect-specific temporal resolver with spatial +filtering concentrated on low-confidence pixels. + +The RecurrentDenoise implementation provides useful filtering ideas. The n8AO +model provides a useful example of compact learned spatial correction. Neither +should be inserted into the current temporal path unchanged. diff --git a/bench/THREE-TEMPORAL-COMPARISON.md b/bench/THREE-TEMPORAL-COMPARISON.md new file mode 100644 index 0000000..cdd49b6 --- /dev/null +++ b/bench/THREE-TEMPORAL-COMPARISON.md @@ -0,0 +1,1305 @@ +# three.js TRAA/TAAU comparison + +## 1. Scope and executive answer + +This audit compares the repository's **current** WebGPU temporal upscaler with the implementations installed in **three.js 0.185.1**: + +- `TRAANode`: same-resolution temporal reprojection anti-aliasing. +- `TAAUNode`: reduced-resolution temporal anti-aliasing plus upscaling. + +The exact installed version is pinned by `../package-lock.json:2717-2722` and `../node_modules/three/package.json:1-19`. + +The local implementation facts in this document describe the code as it exists now. They are not claims that every behavior is permanent design intent. + +### Executive answer + +- **TRAA is not the general upscaling comparator.** It resolves at the input resolution, so it compares directly only with local `QualityMode.NativeAA` at a 1× ratio. See `../src/types.ts:12-24`. + +- **TAAU is the closest reduced-resolution comparator.** It expects low-resolution beauty, depth, and velocity, then reconstructs an output-resolution image. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:10-30`. + +- **The local resolver is feature-richer and structurally heavier.** It has a separate reconstruction pass, persistent age and lock state, reactive masks, generated reactivity, shading-change handling, exposure conditioning, debug views, and RCAS. TAAU and TRAA are simpler and are therefore likely lighter, but there is no controlled GPU benchmark here that proves a winner. + +- **The local resolver can replace TRAA or TAAU for many WebGPU applications, but it is not a transparent drop-in.** The output color domain, logarithmic-depth support, render-graph hooks, resource footprint, reset controls, and APIs differ. + +- **Shared jitter is directionally correct only at the pipeline level.** The preferred design is an application- or render-pipeline-owned temporal sampling context. SSGI, SSR, GTAO, and denoisers should not depend directly on the upscaler. + +### Evidence labels used below + +- **Current implementation fact** means the behavior is directly visible in the audited source. +- **Source-level estimate** means operation or resource cost inferred from source structure, not measured GPU performance. +- **Unverified concern** means the source suggests a possible problem that still needs a focused runtime test. +- **Recommendation** means a proposed direction, not current behavior. + +The files under `node_modules` are useful evidence for the installed package, but they are addon/internal source, not a stable public API contract across future three.js releases. + + +## 2. Plain-language primer + +### Projection jitter + +**Projection jitter** moves the camera's projection by a fraction of one pixel before rendering a frame. Geometry then lands at a slightly different subpixel position each frame. + +A temporal resolver combines those differently positioned samples into a cleaner or higher-resolution result. Exactly **one final temporal resolver** should own this camera movement for a given render. + +This repository applies jitter with `camera.setViewOffset()`. The local frame lifecycle captures a stable projection first, applies jitter before rendering, and clears it afterward. See `../src/Upscaler.ts:260-297`. + +### Stochastic shader noise + +**Stochastic shader noise** changes where a shader samples within an AO, GI, reflection, or denoising kernel. It can rotate rays, shift a sampling pattern, or choose a new random direction. + +It does **not** move the camera. It also does not need to use the upscaler's Halton distribution. An SSGI ray pattern and a camera-jitter pattern solve different sampling problems. + +### Temporal history and reprojection + +**Temporal history** is a texture containing information from previous frames. + +**Reprojection** uses motion vectors and camera transforms to estimate where a previous-frame pixel belongs in the current frame. Without reprojection, camera or object motion would cause history to trail behind the scene. + +### Disocclusion + +A **disocclusion** is an area that becomes newly visible, such as background revealed when a foreground object moves away. + +There is no trustworthy history for a newly revealed surface. A temporal resolver uses depth and motion to detect that case and reject or heavily reduce the old history. + +### Rectification + +**Rectification** constrains reprojected history to colors that are plausible in the current neighborhood. + +It is a defense against stale history. If the old color lies far outside the range or variance of nearby current samples, the resolver clips it toward that neighborhood before blending. + +### Reactive mask + +A **reactive mask** identifies pixels where current color should dominate history. + +Transparent surfaces, particles, animated emissives, and additive effects often lack reliable depth or motion. Marking those pixels as reactive reduces ghost trails. The local API accepts a mask or can generate one from opaque-only versus final color. See `../src/types.ts:119-156` and `../src/Upscaler.ts:421-448`. + +### Why TRAA or TAAU must not run before the local resolver + +Stacking TRAA/TAAU before the local temporal upscaler creates two owners of temporal sampling: + +1. The upstream resolver jitters and temporally filters the scene. +2. The local resolver jitters or expects jitter again and temporally filters the already-resolved result. + +That creates **double jitter** or a mismatch between the color buffer and the final resolver's expected sample position. It also creates **double temporal filtering**, which can blur detail, retain stale history longer, and suppress the per-frame variance the final upscaler needs for reconstruction. + +Use one final AA/upscaling resolver per run: local Native AA, local temporal upscale, TRAA, or TAAU. + + +## 3. Exact implementations audited + +### Local repository + +- Low-level compute orchestration and frame lifecycle: `../src/Upscaler.ts:42-63`, `../src/Upscaler.ts:139-205`, and `../src/Upscaler.ts:260-339`. +- Composable graph node: `../src/UpscalerNode.ts:78-115` and `../src/UpscalerNode.ts:172-330`. +- High-level imperative scene driver: `../src/UpscalePass.ts:21-36` and `../src/UpscalePass.ts:88-172`. +- Jitter generation: `../src/math/halton.ts:1-41` and `../src/math/jitter.ts:3-68`. +- Temporal shaders: `../src/shaders/reconstruct.ts:4-105`, `../src/shaders/accumulate.ts:4-291`, and `../src/shaders/luminancePyramid.ts:4-95`. +- Output stages: `../src/shaders/rcas.ts:4-92`, `../src/shaders/blit.ts:4-46`, and `../src/shaders/common.ts:82-120`. +- Canonical bench integration: `./src/BenchPipeline.ts:14-22` and `./src/BenchPipeline.ts:125-162`. +- Visual fixtures: `../examples/04-aliasing-torture/main.ts:11-14`, `../examples/05-transparency/main.ts:12-18`, `../examples/06-screenspace-gi/main.ts:27-32`, `../examples/09-kitchen-sink/main.ts:25-37`, and `../examples/10-ssgi-denoise/main.ts:27-52`. + +### Installed three.js 0.185.1 + +- TRAA: `../node_modules/three/examples/jsm/tsl/display/TRAANode.js`. +- TAAU: `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js`. +- SSGI: `../node_modules/three/examples/jsm/tsl/display/SSGINode.js`. +- GTAO: `../node_modules/three/examples/jsm/tsl/display/GTAONode.js`. +- SSR: `../node_modules/three/examples/jsm/tsl/display/SSRNode.js`. +- Spatial denoise: `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js`. +- Recurrent denoise: `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js`. +- Temporal reprojection for effects: `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js`. +- Analytic R² noise: `../node_modules/three/examples/jsm/tsl/utils/RNoise.js`. +- Render-pipeline hook storage: `../node_modules/three/src/renderers/common/RenderPipeline.js`. +- Shared velocity node: `../node_modules/three/src/nodes/accessors/VelocityNode.js`. + + +## 4. Jitter comparison + +### Local schedule + +**Current implementation fact** + +The local sequence is centered Halton(2,3): each generated value subtracts `0.5`, producing offsets in render-pixel units around zero. See `../src/math/halton.ts:29-41`. + +The phase count adapts to the upscale ratio: + +`round(8 × ratio²)` + +with a minimum of one phase. See `../src/math/jitter.ts:3-17`. + +Examples: + +- 1×: 8 phases. +- 1.5×: 18 phases. +- 2×: 32 phases. +- 3×: 72 phases. + +The upscaler: + +- captures a stable unjittered projection; +- advances the sequence; +- reads the newly advanced sample; +- applies it through `camera.setViewOffset()`. + +See `../src/Upscaler.ts:270-287`. + +Because `advance()` occurs before `current` is read, the runtime order is a one-sample rotation of the generated array. This does not change the set of samples or cycle length, but it matters when comparing frame-by-frame sequences. + +### three.js schedules + +**Current implementation fact** + +Both installed nodes generate 32 uncentered Halton(2,3) offsets and subtract `0.5` when applying them. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:735-754` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:803-822` + +Both increment with: + +`index % (_haltonOffsets.length - 1)` + +That makes the effective runtime cycle **31 samples**, leaving the final generated offset unused. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:328-338` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:361-370` + +TRAA and TAAU each install their own before/after render-pipeline callbacks to apply and clear camera jitter. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:443-464` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:502-527` + +TAAU's source comments say its offset is reduced to an output-pixel footprint, but the installed implementation applies the centered `[-0.5, 0.5]` offset directly through an input-sized view offset. No input-to-output scale is visible in that code. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:318-354`. + +That makes the intended TAAU jitter footprint internally inconsistent in the audited source. The benchmark should record the behavior as installed and treat any corrected output-pixel variant as a separate diagnostic. + +### Practical conclusion + +Both implementations use the same broad method: low-discrepancy Halton projection jitter and stable motion vectors. + +Local adapts phase count to scale ratio, while installed TRAA/TAAU use an effective fixed 31-frame cycle. Installed TAAU also has the input-pixel/output-pixel inconsistency above. + +For a **stock-product comparison**, preserve each resolver's own schedule. That compares what an application actually receives. + +For a **diagnostic comparison**, a bench may inject one identical sequence into both resolvers to isolate reconstruction and accumulation behavior. That result must be labeled as a modified diagnostic configuration, not stock TRAA/TAAU behavior. + + +## 5. Feature comparison + +### 5.1 Inputs and velocity convention + +#### Local + +The temporal path requires render-resolution color, depth, and velocity. Velocity is three's NDC delta, `current - previous`. Local multiplies it by `(0.5, -0.5)` to convert to UV delta and reprojects with `previousUV = uv - motion`. See `../src/types.ts:119-129`, `../src/Upscaler.ts:620-622`, and `../src/shaders/reconstruct.ts:86-100`. + +**Pros** + +- Matches three's velocity convention. +- Supports explicit reactive and exposure inputs. +- The low-level API makes frame reset and cadence explicit. + +**Cons** + +- Requires correct raw texture access and WebGPU-only integration. +- The current composable node mutates the shared three velocity singleton. +- Public documentation is inconsistent: `UpscalerConfig.jitter` says composable `upscale()` defaults off, but the node constructor currently defaults it on. Compare `../src/types.ts:91-111` with `../src/UpscalerNode.ts:129-165`. + +#### three TRAA/TAAU + +Both consume beauty, depth, velocity, and camera. Both use the same NDC-to-UV conversion and `historyUV = uv - offset`. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-681` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:663-679` + +**Pros** + +- Natural TSL graph integration. +- Returns a graph texture that can feed later post-processing. +- TRAA can use a velocity node from builder context rather than always assuming only the exported singleton. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:466-480`. + +**Cons** + +- Camera and shared velocity state are still modified through node-owned hooks. +- TAAU's jitter setup directly uses the exported global `velocity` singleton, even though its shader accepts a velocity input node. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:327-365`. + +#### Replacement implication + +The motion convention is compatible. The ownership and lifecycle APIs are not. Replacing one resolver with another requires rewiring render ownership, reset behavior, and output transformation rather than only swapping a function call. + + +### 5.2 Depth/motion dilation and disocclusion + +#### Local + +Local fuses nearest-depth motion dilation and depth rejection into one render-resolution compute pass. It searches a 3×3 depth neighborhood, chooses the nearest sample, carries that sample's motion, and compares current versus reprojected previous linear depth. See `../src/shaders/reconstruct.ts:4-25` and `../src/shaders/reconstruct.ts:60-104`. + +Local explicitly supports: + +- standard depth; +- reversed depth; +- perspective cameras; +- orthographic cameras. + +See `../src/shaders/common.ts:122-140` and flag staging in `../src/Upscaler.ts:598-645`. + +**Pros** + +- Strong current support for reversed and orthographic depth. +- Dilation is separated from output-resolution accumulation and reused there. +- Debug views expose motion, depth, and disocclusion. + +**Cons** + +- Previous depth is stored as last frame's positive linearized view-space depth and compared with current linearized view-space depth after motion reprojection. +- It does **not** transform the previous depth sample from previous view space to world space and then into current view space. +- It has no logarithmic-depth conversion path. + +The missing cross-frame camera transform is a quality risk during camera translation or rotation, especially on large depth gradients. It is a source-level difference, not a demonstrated failure in every scene. + +#### three TRAA/TAAU + +Both installed nodes reconstruct the previous depth sample using the previous projection, transform previous view position to world space, then transform it into the current camera view before comparing depth. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:536-548` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:572-581` + +TRAA additionally handles: + +- reversed depth; +- logarithmic depth; +- orthographic depth. + +See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:466-486`, `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:498-546`. + +Installed TAAU does not show equivalent branches in its current-depth sampling, and its previous-depth conversion always calls `viewZToPerspectiveDepth()`. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:537-579`. + +**Pros** + +- Matrix-transformed previous depth better accounts for camera motion. +- TRAA has the broadest depth-mode handling of the three audited paths. + +**Cons** + +- TAAU's installed depth path appears perspective-oriented and lacks explicit reversed/logarithmic handling. +- Depth rejection uses mutable scalar properties with fixed defaults; those defaults still require scene validation. + +#### Replacement implication + +Local explicitly handles reversed-depth and orthographic WebGPU scenes in ways that are not visible in installed TAAU's depth branches. + +TRAA remains safer where logarithmic depth is required. Local should not claim log-depth replacement until that path exists and is tested. + + +### 5.3 Current reconstruction and history filtering + +#### Local + +The current frame is reconstructed with a jitter-aware separable Lanczos2 kernel over a 3×3 render-texel footprint. History uses five filtered Catmull-Rom fetches. See: + +- `../src/shaders/accumulate.ts:83-120` +- `../src/shaders/accumulate.ts:149-201` + +**Pros** + +- Explicitly accounts for where the jittered input sample landed. +- Sharper history reconstruction than a single bilinear sample. +- Current reconstruction, moments, and min/max reuse the same 3×3 color loads. + +**Cons** + +- More filtering work and more source texture operations. +- Negative Lanczos lobes require deringing. +- The exact quality/performance tradeoff has not been benchmarked against installed TAAU. + +#### three TAAU + +TAAU reconstructs current color from a 3×3 input neighborhood with `exp(-2.29 × distance²)`, described in source as a Blackman-Harris Gaussian approximation. It reads one filtered history sample. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:629-715` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:717-728`. + +TRAA is same-resolution and therefore samples current color directly rather than running a reduced-to-output reconstruction kernel. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-681`. + +**Pros** + +- Simpler history path. +- TAAU reuses its nine current samples for both reconstruction and variance moments. + +**Cons** + +- One filtered history sample may blur or under-reconstruct compared with a sharper bicubic history filter. +- The fixed 3×3 current filter may not scale optimally across every upscale ratio. + +#### Replacement implication + +TAAU is the correct comparator for local reduced-resolution reconstruction. TRAA is not. + +The visual comparison should focus on thin geometry, moiré, camera motion, and newly revealed surfaces rather than treating filter names as proof of quality. + + +### 5.4 Rectification and accumulation + +#### Local + +Local: + +- computes neighborhood moments in YCoCg; +- clips history toward a variance box; +- stores accumulation age in history alpha; +- caps age with `maxAccumulation`; +- ages history under disocclusion, heavy clipping, shading change, and reactivity; +- lets stable thin-feature locks relax clipping and favor history. + +See `../src/shaders/accumulate.ts:204-290`. + +**Pros** + +- Rich control over stale-history rejection. +- Explicit age makes reset and convergence behavior inspectable. +- YCoCg usually creates a tighter color-aligned rectification box than RGB. + +**Cons** + +- More persistent state and more interacting tuning constants. +- More ways to over-reject history, protect bad history, or trade shimmer for ghosting. +- Many constants are shader constants rather than convenient diagnostic controls. + +#### three TAAU/TRAA + +TAAU computes RGB moments, narrows variance gamma under motion, clips history, and raises current-frame weight with motion. Its baseline current weight is `0.025`. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:123-132` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:717-755`. + +TRAA uses a `0.05` baseline, adds motion weighting, and can add subpixel-motion correction before the same broad variance/flicker-reduction blend. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:613-643` and `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:683-704`. + +Neither node stores the same explicit normalized age used locally. With valid stationary history and stable luminance, their baseline current weights produce an exponential-like response; motion and flicker reduction alter the effective weights. + +**Pros** + +- Fewer interacting state channels. +- Motion-responsive current weighting is direct and easy to tune. +- TRAA's optional subpixel correction is a focused anti-blur mechanism. + +**Cons** + +- Less explicit convergence control. +- No local-equivalent reactive, shading-change, or exposure-conditioned age policy. +- RGB variance can be less color-aligned than YCoCg. + +#### Replacement implication + +Local has broader controls for difficult content. Three's motion weighting and TRAA subpixel correction are still valuable experiments for local variants; they should be tested behind internal flags rather than assumed superior. + + +### 5.5 Reactive masks, locks, shading, and exposure + +#### Local + +These are mostly local-only features: + +- explicit reactive mask; +- generated reactivity from opaque/final color difference; +- persistent luminance locks; +- shading-change detection; +- automatic, fixed, or external exposure conditioning; +- debug views for each major signal. + +See `../src/Upscaler.ts:394-553`, `../src/shaders/accumulate.ts:211-290`, and `../src/shaders/luminancePyramid.ts:4-30`. + +**Pros** + +- Better tools for transparents, particles, thin features, and HDR stability. +- Caller can explicitly reset and inspect temporal state. + +**Cons** + +- Additional passes, textures, branches, and tuning surface. +- The current scalar exposure analysis is deliberately simpler than a full luminance pyramid. + +#### three + +TRAA has no equivalent reactive-mask, exposure, lock, or shading-change input. + +TAAU contains a thin-feature lock calculation and allocates two history attachments: color and lock. See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:153-163` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:736-758`. + +**Unverified concerns: TAAU lock path** + +The installed source deserves a focused GPU check: + +- history has two attachments; +- the resolve target is created with one attachment; +- the resolve material declares two logical outputs; +- the per-frame copy visibly copies only `resolve.texture` into the first history texture. +- lock history is sampled at the current UV while color history is sampled at the reprojected `historyUV`; +- thin-feature normalization divides by mean luminance without a visible zero-luminance guard. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:160-172`, `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:460-462`, and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:626-627`, `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:727-758`. + +This source structure suggests persistence, reprojection, or dark-pixel stability may not behave as intended, but it is **not evidence that the lock path is definitely broken**. three's material/output handling and backend behavior need a runtime attachment inspection and visual lock test. + +#### Replacement implication + +Local offers explicit reactive-content controls that TAAU does not expose. Whether those controls produce the better image in a given scene still requires GPU comparison. + +Do not copy or design around TAAU's current lock behavior until its second-output persistence is verified. + + +### 5.6 Sharpening and output color domain + +#### Local + +The default temporal output runs RCAS. Each RCAS tap is inverse-tonemapped, de-exposed, transformed through fixed ACES plus sRGB, and then sharpened in that display-referred space before the pass writes `rgba8unorm`. See `../src/shaders/rcas.ts:4-30`, `../src/shaders/rcas.ts:36-61`, and `../src/shaders/common.ts:100-120`. + +The public texture is therefore display-ready under the repository's current presentation contract. See `../src/Upscaler.ts:237-247`. + +**Pros** + +- Simple direct presentation. +- Bench paths can share one known transform. +- Built-in sharpening and optional RCAS denoise. + +**Cons** + +- Not a general linear/HDR graph output. +- Prevents later HDR post-processing, alternate tone mapping, wide-gamut output, or caller-controlled exposure after the resolve. +- Makes direct resolver comparisons unfair unless three's output receives the same transform and sharpening policy. + +#### three + +TRAA and TAAU resolve into half-float render targets and return graph textures. The final `RenderPipeline` can apply tone mapping and output conversion later. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:139-172` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:153-216` +- `../node_modules/three/src/renderers/common/RenderPipeline.js:195-225` + +**Pros** + +- Better composability. +- Keeps the temporal result in the linear graph domain. +- Lets one final output transform serve the whole post stack. + +**Cons** + +- Direct presentation requires correct downstream output configuration. +- It does not include a local-equivalent RCAS stage. + +#### Replacement implication + +Linear half-float output is a clear three.js composability advantage. Local's fixed display output is simpler for direct presentation. + +A future local linear/HDR output option would remove one of the largest replacement boundaries without requiring the direct-present mode to disappear. + + +### 5.7 Reset and resize behavior + +#### Local + +Local has an explicit public `resetHistory()` and a per-dispatch `reset` input. Reconfiguration reallocates textures and schedules reset. See `../src/Upscaler.ts:173-205`, `../src/Upscaler.ts:249-258`, and `../src/types.ts:157-160`. + +**Pros** + +- Camera cuts, teleports, cadence gaps, and app-level discontinuities can be explicit. + +**Cons** + +- The caller must use the reset contract correctly. + +#### three + +TRAA and TAAU recreate or resize targets and seed history after dimension changes. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:383-400` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:413-440` + +No equivalent explicit public camera-cut reset is visible in the audited node APIs. + +**Pros** + +- Resize recovery is automatic. + +**Cons** + +- A camera cut at unchanged dimensions has no obvious public history-reset operation. + +#### Replacement implication + +Local is easier to integrate into applications with explicit scene cuts or discontinuous simulation. + + +### 5.8 Debug, tuning, and API surface + +#### Local + +The current API exposes: + +- low-level `Upscaler`; +- high-level `UpscalePass`; +- composable `UpscalerNode` factories; +- runtime settings; +- eight diagnostic views; +- per-compute-pass timing where timestamp queries exist. + +See `../src/types.ts:39-62`, `../src/Upscaler.ts:63-76`, and `../src/internal/GpuTimer.ts:1-24`. + +**Pros** + +- Strong integration diagnostics. +- Multiple ownership models. + +**Cons** + +- More API and more ways to combine incompatible ownership paths. +- Some tuning remains compile-time shader constants. +- Current jitter-default documentation conflicts with code. + +#### three + +TRAA/TAAU expose a smaller set of mutable thresholds and weights directly on node instances. TRAA also exposes subpixel correction. See: + +- `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:88-120` +- `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:98-132` + +**Pros** + +- Compact API. +- Useful tuning values are directly mutable. + +**Cons** + +- No comparable built-in debug-view suite. +- No explicit reactive/exposure/reset API. + + +## 6. Replacement boundaries + +### Correct comparison matrix + +#### 1× input + +Compare: + +- local temporal with `QualityMode.NativeAA`; +- three `TRAANode`. + +Both are same-resolution temporal AA. TAAU may technically operate near 1×, but it is not the primary product comparison for this case. + +#### Reduced-resolution input + +Compare: + +- local temporal at 1.5×, 2×, and 3×; +- three `TAAUNode` at the same input and output dimensions. + +The local preset ratios are defined in `../src/math/resolution.ts:3-13`. + +### When local is a strong replacement + +Local is a strong candidate when the application: + +- is WebGPU-only; +- can provide three-compatible color, depth, and velocity; +- wants Native AA or temporal upscaling under one API; +- needs reactive-mask handling; +- benefits from thin-feature locks or shading-change aging; +- needs explicit reset; +- wants built-in RCAS and direct display output; +- needs reversed-depth or orthographic behavior beyond installed TAAU's visible branches; +- values debug views over graph simplicity. + +### When TRAA/TAAU remain preferable + +TRAA or TAAU may be preferable when the application: + +- wants a native TSL graph texture in linear half-float space; +- needs later HDR effects or caller-owned output transformation; +- wants fewer persistent resources and a simpler resolver; +- wants three's renderer-state save/restore behavior; +- uses logarithmic depth and can choose TRAA at 1×; +- does not need reactive masks, explicit reset, local exposure conditioning, or RCAS; +- wants to avoid raw WebGPU/private-backend integration. + +### What local should consider adopting + +1. **Matrix-transformed previous depth.** Reconstruct previous view position, move it through world space, and compare in current view space. + +2. **Motion-responsive current weighting.** Prototype a current-frame bias based on motion magnitude. + +3. **TRAA subpixel-motion correction.** Test it as an internal experiment, including its documented square-pattern risk. + +4. **Logarithmic-depth support if an actual integration requires it.** + +5. **A linear/HDR output option.** Keep direct display output as an explicit integration mode. + +6. **Renderer-state preservation in imperative helpers.** `UpscalePass.draw()` currently restores MRT and render target to `null`, not necessarily to the caller's previous state. See `../src/UpscalePass.ts:129-156`. + +7. **Mutable internal benchmark thresholds.** Make depth, weighting, and correction parameters adjustable in bench variants before deciding whether any deserve public API. + +### What local should not copy automatically + +- The fixed effective 31-phase schedule. +- A simpler one-sample history path without visual evidence. +- The absence of reactive, exposure, and explicit-reset features. +- TAAU's installed lock persistence behavior before it is verified. +- Source constants merely because they are in three.js; they target a different resolver structure. + + +## 7. Performance comparison + +Everything in this section is a **source-level structural estimate**, not a GPU measurement. + +Texture-operation counts below count visible source sampling/loading expressions. They are not shader ISA counts. Hardware filtering, compiler common-subexpression elimination, cache behavior, format bandwidth, occupancy, and backend scheduling can change actual cost substantially. + +### Local pass structure + +The default temporal path encodes four compute passes: + +1. exposure; +2. reconstruct; +3. accumulate; +4. RCAS, or blit when sharpening is disabled. + +Generated reactivity adds a fifth pass. See `../src/Upscaler.ts:394-553`. + +#### Exposure + +The current exposure pass runs one 8×8 workgroup but only one invocation performs a serial 32×32 loop: + +- 1,024 filtered scene-color samples; +- one previous-exposure load; +- one external-exposure load. + +The 1,024 scene samples occur before the auto/manual/external selection, so they still run when fixed or external exposure is selected. See `../src/shaders/luminancePyramid.ts:42-94`. + +This is a fixed per-frame cost, not a per-output-pixel count. + +#### Reconstruct + +Per render-resolution pixel, the visible source structure is approximately: + +- nine current depth loads; +- one velocity load; +- up to four previous-depth loads for manual bilinear filtering. + +That is about **14 source texture operations per render pixel**, except offscreen reprojections can return before previous-depth sampling. See `../src/shaders/reconstruct.ts:44-57` and `../src/shaders/reconstruct.ts:66-104`. + +#### Accumulate + +With default locks enabled and no reactive mask, the visible source structure is approximately: + +- one motion load; +- one mask load; +- one exposure load; +- nine current-color loads; +- five filtered history samples; +- one lock-history sample. + +That is about **17 source texture operations per output pixel**. An active reactive input adds one more, for about 18. See `../src/shaders/accumulate.ts:131-178`, `../src/shaders/accumulate.ts:191-201`, and `../src/shaders/accumulate.ts:221-250`. + +#### RCAS or blit + +RCAS visibly performs five input loads. Its helper also loads the same exposure texel per tap on the temporal path, giving ten source texture expressions. A compiler may hoist or merge the invariant exposure loads, so **five input plus one-to-five exposure operations** is the honest range. See `../src/shaders/rcas.ts:36-61`. + +Blit has one filtered input sample and one exposure load on the temporal path. See `../src/shaders/blit.ts:30-45`. + +#### Owned textures + +The current temporal allocation owns **13 textures**, excluding caller inputs, scene render targets, sampler, UBO, and timing buffers: + +- one output; +- two history; +- two locks; +- two dilated depth; +- one dilated motion; +- one mask; +- two exposure; +- one reactive dummy; +- one generated-reactive target. + +This count follows `../src/Upscaler.ts:665-720`. Several are tiny or render-resolution resources; counting textures alone does not represent memory cost. The two display-resolution RGBA16F history textures and two display-resolution RGBA16F lock textures dominate this inventory. + +### TAAU structure + +TAAU performs one output-resolution fullscreen resolve, then: + +- copies resolved output color into history; +- copies current input depth into previous-depth storage; +- performs an additional seed render only after resize. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:413-488`. + +The visible resolve is approximately **22 source texture operations per output pixel**: + +- nine current depth loads; +- one velocity load; +- one previous-depth sample; +- nine current beauty loads; +- one history sample; +- one lock-history sample. + +See `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:537-579` and `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:629-755`. + +That estimate does not resolve the lock-attachment concern described earlier. + +### TRAA structure + +At 1×, TRAA's visible resolve is approximately **21 source texture operations per output pixel**: + +- nine current depth loads; +- one velocity load; +- one previous-depth sample; +- one current color sample; +- one history sample; +- eight additional current-neighborhood samples for variance. + +It then copies output color and input depth into history resources. See `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:498-609` and `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:647-704`. + +### Structural interpretation + +Local is likely to have higher pass, state, and resource cost than installed TAAU/TRAA. + +That does **not** prove it is slower: + +- local reconstruct runs at render resolution; +- local compute passes may schedule differently from TSL fullscreen draws; +- TAAU/TRAA perform full texture copies; +- filterable samples and explicit loads have different costs; +- local and three currently produce different output domains and feature sets. + +Local `GpuTimer` measures only the compute passes encoded by `Upscaler`. It does not include the scene render, final present, TAAU/TRAA resolve, or their history/depth copies. See `../src/internal/GpuTimer.ts:45-98`. + +Therefore, a local `gpuTimings` total is **not comparable** with an untimed three resolver. + +### Compute ping-pong versus render/copy + +#### Local compute ping-pong + +**Pros** + +- Explicit pass boundaries and timestamp scopes. +- Direct storage writes into history/output resources. +- Flexible formats and debug-output routing. +- No full output-to-history color copy after accumulation; accumulation writes the next history directly. + +**Cons** + +- More bind groups, dispatches, barriers, and owned intermediate textures. +- Depends on raw WebGPU access through three internals. +- Fixed storage output format constrains composability. + +#### three render/copy path + +**Pros** + +- Fits TSL and renderer resource management. +- Produces linear half-float graph textures. +- Simpler visible pass graph. +- Uses renderer state save/restore. + +**Cons** + +- Fullscreen resolve plus full texture copies. +- Copy cost is easy to omit accidentally from profiling. +- Scalar pipeline hooks and global velocity state complicate nesting. + +### Fair-comparison requirement + +Output transforms must be normalized: + +- same linear/HDR input; +- same exposure; +- same tone map; +- same output transfer function; +- same sharpening policy; +- same target format where practical. + +Without that normalization, both visual quality and performance results mix resolver behavior with output processing. + + +## 8. Other screen-space nodes and temporal ownership + +Projection jitter and effect noise should not be conflated. + +### SSGI + +`SSGINode.useTemporalFiltering` changes ray direction and initial offsets from `frame.frameId`. It does not own camera jitter and does not store or reproject its own image history. See `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:181-193` and `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:352-382`. + +Its shader combines: + +- interleaved gradient noise; +- a hash-like `rand`; +- six temporal rotations; +- four spatial offsets. + +See `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:7-9` and `../node_modules/three/examples/jsm/tsl/display/SSGINode.js:576-610`. + +The installed documentation says temporal filtering expects TRAA, but the local resolver can be the one final temporal resolver if the SSGI signal is composited into its input. + +### GTAO + +GTAO optionally rotates its sampling directions by frame ID. It does not move the camera or maintain image history. See `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:157-169` and `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:265-288`. + +Its spatial pattern comes from a generated magic-square texture, not conventional blue noise. See `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:171-177` and `../node_modules/three/examples/jsm/tsl/display/GTAONode.js:487-521`. + +### SSR + +Installed SSR defaults `stochastic` to `false`. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:15-25` and `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:52-76`. + +The repository's examples 06 and 09 do not enable stochastic SSR. See: + +- `../examples/06-screenspace-gi/main.ts:165-173` +- `../examples/09-kitchen-sink/main.ts:140-147` + +When stochastic mode is enabled, SSR advances an independent noise index and uses analytic R² noise. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:737-760` and `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:863-892`. + +SSR has optional multibounce history through `setHistory()`, but the audited examples do not wire it. See `../node_modules/three/examples/jsm/tsl/display/SSRNode.js:661-678`. + +### DenoiseNode + +`DenoiseNode` is spatial. It uses a 16-sample rotating kernel and a generated simplex-noise texture, but it owns no temporal image history and does not projection-jitter the camera. See `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:118-132`, `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:184-239`, and `../node_modules/three/examples/jsm/tsl/display/DenoiseNode.js:284-319`. + +### RecurrentDenoiseNode and TemporalReprojectNode + +`RecurrentDenoiseNode` can temporally accumulate a reprojected input and uses analytic R² noise to rotate its spatial kernel. See `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js:321-386` and `../node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js:466-518`. + +`TemporalReprojectNode` owns history or accepts external history. It explicitly states that it does not apply camera subpixel jitter; it reprojects with motion and camera matrices. See `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:489-518` and `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:535-609`. + +Neither node moves the camera projection. Their temporal state can still overlap with the final resolver's history, and `TemporalReprojectNode` still participates in velocity-hook ownership. + +It still installs render-pipeline hooks to bind and later clear the shared velocity projection. That ownership collision is discussed next. + +### Noise taxonomy + +No conventional blue-noise texture is used by these audited paths. + +The distinct mechanisms are: + +- **IGN/hash:** SSGI interleaved gradient noise and `rand`. +- **Magic square:** GTAO's tiled direction texture. +- **Simplex:** `DenoiseNode`'s generated noise texture. +- **Analytic R²:** SSR stochastic sampling and recurrent-denoise rotation through `RNoise`. + +See `../node_modules/three/examples/jsm/tsl/utils/RNoise.js:3-49`. + +Sharing temporal identity can coordinate when these patterns advance. It does not imply replacing all of them with the upscaler's Halton samples. + + +## 9. Hook and velocity collision + +### Scalar render-pipeline hooks + +`RenderPipeline` stores one `onBeforeRenderPipeline` callback and one `onAfterRenderPipeline` callback. They are scalar properties, not callback lists. See `../node_modules/three/src/renderers/common/RenderPipeline.js:195-204`. + +The pipeline calls each scalar once around its graph render. See `../node_modules/three/src/renderers/common/RenderPipeline.js:121-150`. + +These nodes assign those scalar properties directly: + +- local `UpscalerNode`: `../src/UpscalerNode.ts:206-217`; +- TRAA: `../node_modules/three/examples/jsm/tsl/display/TRAANode.js:443-464`; +- TAAU: `../node_modules/three/examples/jsm/tsl/display/TAAUNode.js:502-527`; +- `TemporalReprojectNode`: `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:728-748`. + +If more than one is nested in a graph, setup order can overwrite an earlier callback. + +### Global mutable velocity + +three exports one shared singleton named `velocity`, created through `nodeImmutable()`. The wrapper is shared, while its internal `projectionMatrix` remains mutable through `setProjectionMatrix()`. See `../node_modules/three/src/nodes/accessors/VelocityNode.js:24-98` and `../node_modules/three/src/nodes/accessors/VelocityNode.js:218-224`. + +When both lifecycles are coordinated, the required frame ordering is: + +1. capture the stable, unjittered camera projection; +2. bind that projection for velocity generation; +3. apply projection jitter to the scene camera; +4. render all velocity/color/depth inputs; +5. clear camera jitter; +6. release or restore velocity ownership only after all dependent passes are done. + +Another node must not replace or clear the stable velocity projection between steps 2 and 4. + +### Example 10 risk + +Example 10 can nest `TemporalReprojectNode` inside the local `UpscalerNode` graph. See `../examples/10-ssgi-denoise/main.ts:161-192` and `../examples/10-ssgi-denoise/main.ts:213-221`. + +Both nodes assign the same scalar hooks. `UpscalerNode` uses them for camera-jitter begin/end, while `TemporalReprojectNode` uses them to bind and clear the shared velocity projection. + +Setup order deterministically leaves only the last callback pair installed. Depending on build order, either the local camera-jitter lifecycle or `TemporalReprojectNode`'s velocity-projection lifecycle may not run as intended. `TemporalReprojectNode` also has a one-time `updateBefore()` fallback, so the first frame may differ from steady state. + +The exact runtime consequence is an **unverified, source-derived integration risk**, not a confirmed failure from this audit. The example's documented GPU-observed quality conflict concerns stacked temporal accumulation under FSR jitter; it does not prove this hook collision itself occurred. See `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:630-659`, `../node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js:728-748`, and `../examples/10-ssgi-denoise/main.ts:32-52`. + + +## 10. Shared temporal sampling options + +### A. Application/render-pipeline-owned `TemporalSamplingContext` + +The application or render pipeline owns frame identity, projection jitter, reset generation, and ordered begin/end behavior. Resolvers and effects consume the context. + +**Pros** + +- Avoids direct SSGI/SSR dependency on one upscaler. +- Supports local FSR, TRAA, TAAU, or a future resolver as alternatives. +- Creates one authoritative projection-jitter owner. +- Coordinates reset and rendered cadence across effects. +- Provides a natural place to compose hooks. + +**Cons** + +- Requires integration work above individual nodes. +- Needs careful behavior when a pass renders less often than the display loop. +- Public API should wait until the bench proves the contract. + +**Recommendation:** preferred direction to prototype before considering a public API. + +### B. Upscaler-owned provider + +The local upscaler publishes jitter and frame state for SSGI/SSR/denoisers. + +**Pros** + +- Smaller immediate change. +- Reuses data the upscaler already computes. + +**Cons** + +- Couples unrelated effects to one resolver. +- Makes TRAA/TAAU substitution harder. +- Encourages the upscaler to own application scheduling. +- Risks turning implementation details into public API too early. + +**Recommendation:** not preferred. + +### C. Independent noise, one projection-jitter owner + +Keep SSGI/GTAO/SSR/denoiser noise schedules independent. Ensure only the final resolver moves the camera. + +**Pros** + +- Minimal change. +- Preserves each shader's optimized distribution. +- Avoids double projection jitter. + +**Cons** + +- Reset and frame cadence can drift. +- Debugging cross-pass temporal behavior remains harder. +- Effects may advance noise on frames where their output was not rendered. + +**Recommendation:** acceptable short-term state. + +### Conceptual context fields + +A future context should be able to provide: + +- `frameId` and `sampleIndex`; +- current and previous jitter; +- stable unjittered projection; +- current and previous jittered projections; +- render and display dimensions; +- history/reset generation token; +- whether jitter was actually applied for this render; +- `beginFrame()` and `endFrame()` lifecycle boundaries. + +The context should synchronize **identity, reset, and cadence**. It should not require every shader to derive its samples from one distribution. + + +## 11. Integration design + +This is a recommendation, not current API. + +### 11.1 External-jitter mode for `Upscaler` + +Add an external mode in which: + +- the coordinator passes exact current and previous offsets to dispatch; +- offsets are explicitly defined in render-pixel units and top-left convention; +- the upscaler does not internally advance `JitterSequence`; +- `beginFrame()` does not independently move the camera; +- a reset-generation token invalidates history exactly once. + +The internal schedule remains the default for existing imperative users. + +### 11.2 Ordered hook coordinator + +Replace scalar callback assignment with ordered participants: + +1. capture stable camera state; +2. prepare velocity providers; +3. apply projection jitter; +4. render graph dependencies; +5. clear projection jitter; +6. finalize temporal state. + +The coordinator can adapt to three's scalar hook by installing one callback that runs an ordered list. + +### 11.3 Dedicated velocity projection provider + +Do not rely only on the exported global `velocity` singleton. + +Provide an explicit velocity projection/provider per render pipeline or camera so nested temporal effects cannot clear one another's state accidentally. + +### 11.4 Optional identity for noisy effects + +SSGI, GTAO, SSR, and denoisers may receive: + +- sample/frame identity; +- reset generation; +- whether this effect actually rendered this frame. + +They should retain their own IGN, magic-square, simplex, R², or domain-specific distributions. + +### 11.5 TemporalReproject integration + +Allow `TemporalReprojectNode` to consume a stable projection provider and disable its own before/after hook ownership. + +It can still own effect history without becoming a projection-jitter owner. + +### 11.6 TRAA/TAAU diagnostics + +TRAA/TAAU may optionally consume externally supplied jitter in bench-only diagnostic variants. + +They remain **alternative final resolvers**. They must never run upstream of local FSR in a product comparison. + +### 11.7 What must synchronize + +Synchronize: + +- frame identity; +- reset generation; +- rendered cadence; +- current/previous projection state; +- exactly one camera-jitter application. + +Do not automatically synchronize every noise distribution. + + +## 12. Benchmark plan + +Use `bench` as the canonical timing harness. It already controls render/display resolution, shares one display transform across its current modes, and surfaces local timestamp results. See `./src/BenchPipeline.ts:75-161` and `./src/main.ts:86-120`. + +Use examples as visual fixtures: + +- `../examples/04-aliasing-torture`: thin geometry, moiré, slow camera motion. +- `../examples/05-transparency`: transparents, particles, reactive-mask behavior. +- `../examples/06-screenspace-gi`: reduced-resolution GTAO/SSR/SSGI feeding the raw upscaler. +- `../examples/09-kitchen-sink`: in-graph SSGI/SSR composition and local node jitter. +- `../examples/10-ssgi-denoise`: stacked effect-history experiments and hook-collision characterization. + +### 12.1 Two comparison classes + +#### Stock-product comparison + +Preserve: + +- each resolver's stock jitter schedule; +- stock reconstruction and history filtering; +- stock public defaults; +- one resolver per run. + +Normalize only the surrounding application conditions needed for a fair output. + +#### Normalized diagnostic comparison + +Allow controlled substitutions such as: + +- identical injected jitter sequence; +- identical current-frame input; +- identical output transform; +- sharpening disabled on both, or equivalent sharpening added to both; +- internal parameter sweeps. + +Label these results as diagnostics, not stock product behavior. + +### 12.2 Modes + +At 1×: + +- local Native AA; +- three TRAA. + +At reduced resolution: + +- local temporal at 1.5×; +- local temporal at 2×; +- local temporal at 3×; +- three TAAU at the same three ratios. + +Do not put TRAA in the reduced-resolution result set as if it were an upscaler. + +### 12.3 Deterministic setup + +For each capture: + +- fixed canvas physical dimensions and DPR; +- fixed camera path and animation time; +- fixed scene random seeds; +- fixed effect settings; +- fixed renderer features; +- MSAA off; +- one final temporal resolver; +- clean history at the start of each block; +- fixed warm-up frame count; +- fixed measured frame count; +- no UI interaction during the sample window. + +Record: + +- browser and version; +- OS; +- adapter name; +- WebGPU backend; +- three version; +- commit SHA; +- display and render dimensions; +- timestamp-query availability; +- power/thermal state where practical. + +Local can use its explicit reset API. TRAA/TAAU do not expose an equivalent public camera-cut reset, so recreate their node/graph for each capture block or document a deliberately forced resize/reseed procedure. + +### 12.4 Output normalization + +For visual comparison: + +- feed the same linear HDR scene; +- use the same tone map and sRGB conversion; +- compare the same output format where possible; +- disable local RCAS or add a matched sharpen after TAAU; +- keep exposure behavior equivalent; +- capture before browser scaling. + +Keep a separate product-default gallery showing local RCAS/direct output versus three's normal graph output. Do not mix that gallery with resolver-isolation conclusions. + +### 12.5 Timing + +Measure at least: + +- scene/G-buffer render; +- resolver work; +- history/depth copies; +- output transform; +- sharpening; +- final present where relevant; +- whole-frame GPU time. + +Local `GpuTimer` alone is insufficient because it covers only local compute passes. Instrument three's resolve and copies with compatible GPU timestamps, or use a GPU trace that includes both implementations. + +If per-copy timestamping is unavailable, use labeled command scopes or frame captures and report the limitation explicitly. + +### 12.6 Sampling method + +- Warm all pipelines and histories before measured sequences. +- Use repeated A/B/B/A or **ABBA** ordering to reduce thermal and drift bias. +- Report median and p95, not only average. +- Keep raw frame samples. +- Repeat enough blocks to show run-to-run spread. +- Treat CPU FPS as supporting information, not a substitute for GPU timing. + +### 12.7 Scene scenarios + +Include: + +- static camera and static scene for convergence; +- slow camera pan; +- fast camera translation; +- rotating camera; +- independently moving foreground object; +- newly revealed background/disocclusion; +- thin fences and wires; +- high-frequency floor or foliage; +- bright HDR highlights; +- transparent/additive particles; +- noisy SSGI; +- glossy stochastic SSR if enabled; +- camera cut and explicit reset; +- resize and history seed. + +### 12.8 Acceptance criteria + +A benchmark configuration is valid only when: + +- exactly one projection-jitter owner is active; +- velocity is generated from the stable projection; +- color, depth, velocity, and reactive inputs are aligned; +- history is clean at test start, using explicit local reset or three node/graph recreation; +- output transforms are documented and matched; +- no WebGPU validation errors occur; +- TAAU lock persistence has been independently characterized; +- hook assignment and clearing order are logged or inspected; +- timings include equivalent work or clearly identify exclusions. + +Quality review should record: + +- static convergence; +- edge shimmer; +- detail retention; +- motion blur; +- ghost trails; +- disocclusion recovery; +- thin-feature stability; +- transparency behavior; +- SSGI/SSR noise convergence; +- camera-cut recovery. + +No winner should be declared from source operation counts alone. + + +## 13. Recommended next actions + +### 1. Fix or document integration inconsistencies before benchmarking + +- Correct the `../src/types.ts:91-111` jitter-default text to match current `UpscalerNode` behavior, or intentionally change the behavior and document that decision. +- Verify installed TAAU lock persistence on GPU. +- Characterize scalar hook overwrite and velocity clearing in example 10. + +### 2. Add a bench-only TRAA/TAAU comparator + +- Instantiate one final resolver at a time. +- Use identical scene inputs and output transform. +- Keep TRAA at 1× and TAAU at reduced resolution. +- Preserve stock schedules for the product test. + +### 3. Harden timing and capture + +- Time three resolve and copy work, not only local compute. +- Add deterministic camera/animation scripts. +- Record environment metadata. +- Store raw median/p95 samples and screenshots. + +### 4. Prototype selected local algorithm variants + +Behind internal or bench-only flags: + +- matrix-transformed previous depth; +- motion-responsive current weighting; +- TRAA-style subpixel correction. + +Do not expose public tuning until the variants have visual and performance evidence. + +### 5. Prototype `TemporalSamplingContext` internally + +Build it in bench/internal scope with: + +- one projection-jitter owner; +- ordered hook composition; +- explicit stable velocity state; +- frame/reset generation; +- rendered cadence. + +Avoid committing to public API shape yet. + +### 6. Feed optional temporal identity to one noisy node + +Start with SSGI: + +- share frame/sample identity and reset generation; +- preserve SSGI's own ray distribution; +- verify that skipped renders do not advance its pattern incorrectly. + +### 7. Decide public API after evidence + +Only then decide whether to expose: + +- external jitter; +- linear/HDR output; +- a temporal context; +- velocity providers; +- benchmark-proven tuning controls. + +### Direct answer: is shared jitter the right move? + +**Yes** to exactly one projection-jitter owner. + +**No** to forcing SSGI, SSR, GTAO, or denoiser noise to derive from the FSR Halton sequence. + +Shared frame/reset identity is a reasonable prototype for coordinating rendered cadence and history invalidation, but its benefit still needs to be demonstrated. Each shader should keep the sampling distribution suited to its own problem. diff --git a/src/shaders/README.md b/src/shaders/README.md index 8521099..0e597b1 100644 --- a/src/shaders/README.md +++ b/src/shaders/README.md @@ -7,82 +7,829 @@ Every pass is a WGSL compute module assembled from shared chunks (`common.ts` + - **Coordinate space** — texel coordinates and UVs are top-left origin. Motion vectors arrive from three's `velocity` node as NDC deltas (`current − previous`) and are converted to UV deltas with `motionScale = (0.5, −0.5)`; reprojection is `prevUV = uv − motion`. - **Jitter** — applied via `camera.setViewOffset`, so a render texel at index `i` holds scene content from unjittered position `i + jitter`. The accumulate pass measures kernel distances against `srcPos = uv·renderSize − 0.5 − jitter`. - **Depth** — supports three's standard and reversed WebGPU depth conventions (flag bit + `linearizeDepth`, derived from `Matrix4.makePerspective`). Comparisons happen on positive view-space distances. -- **Color spaces** — the temporal pipeline accumulates in _invertible-tonemap space_ (`c / (1 + max(c))`, FSR2's trick) so a single HDR firefly can't swamp the history average. EASU/RCAS run display-referred per the FSR1 spec. Every path exits through the same ACES + sRGB `displayTransform`, so bench modes are comparable. - -## Passes vs. the FidelityFX reference - -| Pass | Fidelity to AMD's source | Simplifications (→ Phase 3+) | -| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `easu.ts` | Faithful port of `FsrEasuF` (12-tap edge-rotated anisotropic Lanczos, deringing) | Exact `1/x`/`inverseSqrt` instead of `APrxLo*` bit tricks; `textureLoad` instead of packed gathers | -| `rcas.ts` | Faithful port of `FsrRcasF` (analytic lobe bound, `exp2` sharpness mapping) + optional `FSR_RCAS_DENOISE` variant | — | -| `reconstruct.ts` | _reconstruct & dilate_ + _depth clip_ fused into one render-res pass | No scatter-based "reconstructed previous depth" (outputs linearized depth directly); disocclusion compares against last frame's dilated depth (classic TAA style) with a relative-depth threshold instead of a plane-fit `Ksep` | -| `accumulate.ts` | Same structure as _reproject & accumulate_ (Lanczos2 upsample, kernel-confidence weighting, history rectification, accumulation counter in alpha) + luminance-stability **locks** protecting thin features + **auto-exposed** pre-tonemap + **shading-change** history aging | Variance clipping (Playdead) as the base rectifier; shading-change measured on the 3×3 neighborhood mean vs history luma rather than a dedicated coarse pyramid mip; Catmull-Rom (Jimenez 5-fetch) history filter | -| `luminancePyramid.ts` | Same intent as _compute luminance pyramid_ (log-average luminance → auto-exposure with eye-adaptation) | Single-workgroup reduction of a 32×32 tap grid instead of an atomic SPD mip chain; intermediate mips not yet produced (the shading-change detector will need them) | -| `generateReactive.ts` | Same intent as _GenerateReactiveMask_ (opaque-vs-final color diff → reactive mask) | Fixed threshold/scale constants rather than per-call params | -| `blit.ts` / `debug.ts` | — (bench/output utilities) | | - -**Luminance-stability locks** (Phase 3, done) protect thin sub-pixel features (wires, -fence pickets, foliage) from the variance clip that would otherwise drag their bright/dark -history toward the neighborhood mean — the cause of thin features dimming and shimmering -under motion. The accumulate pass keeps a persistent display-res lock buffer (r = lock -lifetime, g = locked luma), reprojected through motion like the color history: it detects -a thin feature as a luminance outlier vs its neighborhood (`peakiness` × `contrast`), grows -a lock while the feature is present, and breaks it on disocclusion or a shading change. -A locked pixel widens its rectification AABB (`LOCK_CLAMP_RELAX`) and leans on history in -the blend (`LOCK_HISTORY_BOOST`). Toggle via `settings.lockThinFeatures` (the `FLAG_LOCKS` -bit); inspect via `DebugView.Locks`. The tuning constants at the top of `accumulate.ts` -are sensible defaults, not final — tighten if you see thin features ghost, loosen if they -still dim. - -**Auto-exposure** (Phase 3, done) conditions the invertible-tonemap accumulation. -`luminancePyramid.ts` reduces the scene to a single log-average luminance in one workgroup, -maps it to a pre-exposure that lands the average on middle grey, and eases toward it over -time (eye-adaptation). `accumulate.ts` multiplies the input by that exposure before the -invertible tonemap; the output pass (`rcas.ts` / `blit.ts`) divides it back out before the -display transform — so a very bright or very dark HDR scene accumulates in the same working -range (steadier variance clip + firefly guard) **without changing final brightness**. Toggle -via `settings.autoExposure` (the `FLAG_AUTO_EXPOSURE` bit); with it off, the fixed -`settings.exposure` is published through the same path. Inspect via `DebugView.Exposure` -(the exposed scene luminance should read near mid-grey everywhere). - -**Shading-change detection** (Phase 3, done) tells a genuine shading change (a light -turning on, an animated material) apart from mere motion, so the changed surface -re-converges to its new look instead of ghosting the old one. It compares the reprojected -history's luma against the current 3×3 neighborhood mean, normalized by how much the -neighborhood itself varies — a coherent disagreement the local variance can't explain. -Where it fires, non-locked history is aged (`SHADING_AGE`); a lock fully suppresses the -aging (aliasing on a thin feature is not a shading change — and by construction the -background-dominated neighborhood mean always disagrees with a thin feature's history, so -the detector must not touch locked pixels or drive lock-breaking; locks break on their own -self-referential luma term). Toggle via `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`); inspect via -`DebugView.ShadingChange`. **Simplification vs FSR2:** the comparison uses the 3×3 -neighborhood mean rather than a dedicated coarse luminance-pyramid mip — cheaper, and -jitter-robust enough in practice, but a true SPD mip would be steadier on high-frequency -content (a Phase-5 refinement). Constants (`SHADING_LO/HI/AGE` at the top of -`accumulate.ts`) are conservative defaults — raise `SHADING_LO` if stable surfaces shimmer, -lower it if changed shading ghosts. - -**Reactive mask** (Phase 3, done) is the caller-authored escape hatch for geometry that has -no reliable depth or motion — additive particles, transparent/animated surfaces — which -would otherwise ghost through the history. Pass a render-res mask as `dispatch({ reactive })` -(red channel `[0,1]`); flagged pixels suppress lock formation, keep almost no accumulation, -and snap toward the current frame in the blend (`REACTIVE_STRENGTH`). No mask → a 1×1 zero -texture is bound and the whole path is flag-gated off, so there's zero cost when unused. -Author the mask however you like — render your transparents' coverage, or let the library -generate it: pass `dispatch({ reactiveOpaqueColor })` (an opaque-only render at render res) -and `generateReactive.ts` diffs it against the final `color` (FSR2's `GenerateReactiveMask`) -to produce the mask automatically. `examples/05-transparency` demonstrates both. Inspect via -`DebugView.Reactivity`. (Auto-gen note: render the opaque pass with the same jitter as the -final frame, or high-contrast edges leave faint reactivity from sub-pixel misalignment.) +- **Color and exposure domains** — the local temporal pipeline multiplies input by the + selected local conditioning exposure (auto, fixed, or external), then accumulates in + _invertible-tonemap space_ (`c / (1 + max(c))`). Before RCAS it inverse-tonemaps, divides + by the current local exposure, and applies ACES + sRGB through callbacks. FSR Upscaler + 3.1.5 instead keeps three concepts separate: the host's input `preExposure`, + `DeltaPreExposure()` for moving reprojected history into the current host pre-exposure + domain, and internal/app `Exposure()` for conditioning. It removes only `Exposure()` + before storage/output, so the result remains in the same color and host pre-exposure + domain as the caller's input. + +## FSR1 spatial fallback + +`easu.ts` is the library's separate spatial fallback, not a stage in the FSR 3.1.5 +temporal graph. **Current status — Source-aligned FSR1 port:** its 12-tap EASU +implementation retains the reference edge analysis, anisotropic Lanczos reconstruction, +tap placement, and deringing. + +The WebGPU port uses native WGSL division and `inverseSqrt`, plus per-tap `textureLoad` +calls, instead of AMD's approximation helpers and packed gathers. Those are implementation +and profiling differences, not known algorithm gaps. The local path also assumes an +exact-sized input resource and applies the library's display conversion through its load +callback. **Next action — Keep / Benchmark:** keep EASU as the documented FSR1 fallback; +benchmark the math/load variants before changing them, and generalize viewport or output +handling only when an integration requires it. + +## Temporal pipeline vs. FSR Upscaler 3.1.5 + +This audit is pinned to AMD FidelityFX SDK +[commit `60f4ea81909200d8542eca14dccb2628b763a9a3`](https://github.com/GPUOpen-LibrariesAndSDKs/FidelityFX-SDK/commit/60f4ea81909200d8542eca14dccb2628b763a9a3), +whose `ffx_fsr3upscaler.h` declares **FSR Upscaler 3.1.5**. SDK package tag `v2.3.0` +points to that commit, but `2.3.0` is the package version, not the upscaler algorithm +version. Temporal comparisons below use only the FSR3-named implementation. RCAS is the +exception: FSR 3.1.5 directly includes `fsr1/ffx_fsr1.h`, so that shared helper is part of +the authoritative path. The sibling FSR2 temporal resolver is not used as a baseline. + +The temporal resolver began as a direct FSR port and retains FSR's pipeline intent, +resource flow, and algorithmic lineage. The current implementation combines necessary +WebGPU/three.js adaptations with custom replacements and stages that are absent relative +to FSR Upscaler 3.1.5. This audit records those differences so the project can converge +where source parity remains the goal; a divergence below is not automatically an accepted +design choice. + +### How to read the audit + +Each item separates two questions: + +- **Current status** describes the implementation today: + - **Source-aligned** — the reference algorithm is retained with language-level porting. + - **Adapted port** — the reference behavior remains recognizable, with integration-specific changes. + - **Diverges** — the reference stage exists locally but differs in material behavior. + - **Custom replacement** — a local algorithm stands in for the reference stage. + - **Missing** — no local equivalent currently exists. +- **Next action** describes the recommendation: + - **Keep** — retain the current implementation. + - **Benchmark** — compare quality/performance before choosing either implementation. + - **Target parity** — change the local implementation toward the FSR 3.1.5 behavior. + - **Target parity selectively** — converge through coordinated work or an optional API. + +### Reference pass graph + +FSR Upscaler 3.1.5 dispatches: + +`prepare inputs` → `luma SPD` → `shading-change SPD` → `shading change` → +`prepare reactivity` → `luma instability` → `accumulate` → optional `RCAS` + +### Core recommendation + +Do not replace the local resolver wholesale or preserve divergences by default. Build a +controlled A/B harness around `bench/`, test one compatible parity change at a time, and +keep whichever path gives the better measured quality/performance tradeoff. + +Start with domain and math mismatches that fit the current graph, then compare filters, +then add structural parity as separately configured pipeline variants. Keep a local +divergence only when measurements demonstrate a performance advantage or a verified +WebGPU/three.js constraint justifies it. The full experiment sequence and decision rules +are in [Parity evaluation plan](#parity-evaluation-plan). + +### Audit + +#### RCAS bounds and denoise (`rcas.ts`) + +- **Current status:** Diverges. The lower-limiter omission and denoise mismatch are likely + incomplete parity; native WGSL division and dispatch layout are separate implementation + differences. +- **Local implementation:** Omits + `lowerLimiterMultiplier = saturate(eL / min(neighbor lumas))` from `hitMin`. Denoise uses + green only, excludes the center from its denoise range, and defaults off. Lobe math uses + native WGSL division, and dispatch is one pixel per invocation in 8×8 workgroups. +- **FSR 3.1.5 behavior:** Applies the green-weighted lower limiter. Denoise uses + `L = 0.5R + G + 0.5B`, includes the center in the denoise min/max, and is enabled by the + temporal RCAS pass. Center RGB remains excluded from `mn4`, `mx4`, and `hitMax`. The + source uses high-precision reciprocals for the lobe bounds, an approximate-medium + reciprocal for final normalization, and quad-remapped 16×16 coverage. +- **Why it differs / evidence confidence — Unclear:** No platform requirement has been + identified for the missing lower limiter or different denoise math/default. Local + division matches the source's high-precision intent for the lobe bounds; final + normalization and dispatch layout could affect GPU cost, but no local measurement + currently demonstrates an advantage. + +**Keep the local path** + +- **Pros:** Preserves the current output and opt-in denoise policy. Native WGSL division is + straightforward and retains high-precision lobe-bound behavior. +- **Cons:** Default sharpening and isolated-luma attenuation do not match the source. + Native final normalization and the local dispatch may cost more or less depending on the + GPU; that has not been measured. + +**Adopt FSR parity** + +- **Pros:** Restores the source limiter, denoise response, and temporal default. Reduces an + obvious math divergence before evaluating larger temporal changes. +- **Cons:** The approximate-medium final normalization and quad remapping may behave + differently across WebGPU implementations. Matching those implementation details + without timing evidence could change cost without a quality benefit. + +**Next action:** **Target parity.** Restore the lower limiter, source denoise luma/range, +and temporal denoise default. Separately benchmark native versus approximate-medium final +normalization and 8×8 per-pixel dispatch versus quad-remapped coverage. + +#### RCAS color domain + +- **Current status:** Diverges materially in color space. +- **Local implementation:** Inverse-tonemaps, divides by the current local exposure, and + applies ACES + sRGB per tap before RCAS. +- **FSR 3.1.5 behavior:** Filters color conditioned by `Exposure()`, reverses + `Exposure()`, and applies no presentation transform. Host `preExposure` remains, so + linear HDR input remains linear HDR output. +- **Why it differs / evidence confidence — Unclear:** Folding presentation into the + compute output produces a directly presentable texture, but no evidence establishes + that this was an intended departure from the original direct port or that it improves + performance. + +**Keep the local path** + +- **Pros:** Produces the library's current directly presentable output without requiring a + separate caller-managed presentation stage. +- **Cons:** RCAS operates on presentation-transformed neighborhoods, limiting HDR, + alternate tone mapping, gamut choices, and later linear post-processing. + +**Adopt FSR parity** + +- **Pros:** Keeps RCAS and output in the caller's linear color/pre-exposure domain. Improves + composability and makes the temporal color pipeline source-aligned. +- **Cons:** Requires presentation to happen elsewhere. A careless migration could + duplicate or omit the final display transform. + +**Next action:** **Target parity.** Add a linear/HDR resolver output and route it through +the same existing final presentation transform for comparison. Keep directly presentable +output only as an explicit integration mode if it remains useful. + +#### Reconstruction and disocclusion (`reconstruct.ts`) + +- **Current status:** Custom replacement. +- **Local implementation:** Ping-pongs bilinearly sampled linear dilated depth and applies + fixed relative thresholds in one fused render-resolution pass. +- **FSR 3.1.5 behavior:** Scatters nearest current depth into previous-frame positions with + atomics, then evaluates reconstructed samples with viewport- and depth-scaled thresholds. +- **Why it differs / evidence confidence — Rationale unclear; structural effect + verified:** The local fusion demonstrably avoids one dispatch boundary, intermediate + resource traffic, and source scatter atomics. No evidence establishes that performance + motivated the original divergence or that the reduced work is faster on target devices. + It does not preserve scatter coverage semantics; those writes require synchronization + or atomics. + +**Keep the local path** + +- **Pros:** Avoids a dispatch and intermediate traffic. Avoids atomic scatter requirements + that may be costly or awkward on some WebGPU devices. +- **Cons:** Cannot reconstruct previous-depth coverage around motion in the same way as the + source. Fixed thresholds may classify disocclusion differently by depth and resolution. + +**Adopt FSR parity** + +- **Pros:** Restores source coverage and threshold scaling, which may improve history + rejection around moving silhouettes and depth discontinuities. +- **Cons:** Adds synchronized scatter work, resources, and a pass boundary. The actual GPU + cost and quality gain are unmeasured locally. + +**Next action:** **Benchmark.** Build a distinct synchronized reconstruction variant, +capture disocclusion and accumulation-age results on controlled motion, and report its +per-pass distributions and upscaler compute-pass sum against the fused pass. + +#### Farthest depth and motion divergence + +- **Current status:** Missing. +- **Local implementation:** Provides neither farthest depth nor motion-divergence state. +- **FSR 3.1.5 behavior:** Prepares farthest depth and motion divergence for confidence, + reactivity, and later temporal decisions. +- **Why it differs / evidence confidence — Unclear:** The local graph is reduced to color, + nearest depth, motion, and one reactive input. No verified platform or performance reason + for omitting these signals has been recorded. + +**Keep the local path** + +- **Pros:** Avoids the additional resources, calculations, and graph coupling. +- **Cons:** Later stages cannot use source confidence signals near depth or motion + discontinuities. Any performance advantage remains unmeasured. + +**Adopt FSR parity** + +- **Pros:** Supplies the data expected by source reactivity and temporal-confidence logic. + May improve difficult motion and silhouette cases. +- **Cons:** Expands the prepare-inputs graph and resource set. Benefits cannot be isolated + fully until downstream consumers also exist. + +**Next action:** **Target parity selectively.** Add both signals in the structural +prepare-inputs variant, then evaluate them with reconstructed-depth and reactivity parity +rather than as disconnected accumulation constants. + +#### Motion, depth, and integration conventions + +- **Current status:** Adapted port with a legitimate three.js integration tradeoff. +- **Local implementation:** Expects render-resolution, jitter-free NDC velocity in + current-minus-previous form, applies `motionScale = (0.5, -0.5)`, and derives finite + camera plus standard/reversed-depth behavior from three. +- **FSR 3.1.5 behavior:** Supports configurable motion scale, render- or + display-resolution vectors, jitter cancellation, infinite depth, and dynamic render + size. +- **Why it differs / evidence confidence — Verified:** The adapter is matched to three's + velocity node and renderer conventions. That makes the common path simple and correct, + but does not cover all valid external inputs. + +**Keep the local path** + +- **Pros:** Minimal setup for three users and a known convention for the built-in velocity + node. Avoids exposing source-level integration complexity in the common API. +- **Cons:** External velocity/depth resources can use incompatible scale, sign, jitter, + resolution, or depth conventions. + +**Adopt FSR parity** + +- **Pros:** Enables correct integration of externally authored resources and broader camera + configurations. +- **Cons:** More configuration increases misuse risk. Replacing the three-specific defaults + would make the primary integration less convenient without improving its correctness. + +**Next action:** **Target parity selectively.** Keep the three adapter unchanged and +prototype a lower-level configurable dispatch path for external motion/depth inputs, +including explicit scale, resolution, jitter, and depth conventions. + +#### Reactive and T&C handling + +- **Current status:** Custom replacement with missing source inputs. +- **Local implementation:** Samples one reactive mask directly in accumulation and strongly + reduces history and lock influence. +- **FSR 3.1.5 behavior:** Folds max-dilated application reactive into the + shading-change/accumulation-reset channel. It samples T&C separately, combines it with + motion divergence, and stores that result in the reactive channel consumed by lock and + rectification behavior. +- **Why it differs / evidence confidence — Likely:** One mask avoids T&C resources and + simplifies the public integration. This is a plausible platform/API rationale, not + measured evidence or proof that the divergence was intended. + +**Keep the local path** + +- **Pros:** Simple authoring and resource binding. Existing transparency integrations need + only one mask. +- **Cons:** Raw reactive lacks source dilation/reset coupling. T&C and motion divergence + cannot contribute with their distinct source behavior. + +**Adopt FSR parity** + +- **Pros:** Restores spatial coverage and coordinated temporal reset behavior. Supports + softer T&C influence instead of incorrectly aliasing it to aggressive reactive handling. +- **Cons:** Adds resources, pass work, and API complexity. The total cost and benefit need + representative transparency measurements. + +**Next action:** **Target parity selectively.** Implement prepare-reactivity in a distinct +graph variant first. Add public T&C configuration only after its core behavior is present +and validated on `examples/05-transparency`. + +#### Reactive generation (`generateReactive.ts`) + +- **Current status:** Adapted port with fixed policy. +- **Local implementation:** Uses component-max opaque-versus-final color difference with + fixed threshold, scale, and cap. +- **FSR 3.1.5 behavior:** Selects component-max or vector-length difference, supports + optional tone-map/inverse-tone-map transforms, and configures scale, threshold, and + binary output. +- **Why it differs / evidence confidence — Likely:** Fixed settings keep the helper and API + compact. No benchmark or platform requirement shows that the reduced policy is + preferable. + +**Keep the local path** + +- **Pros:** Small API surface and predictable existing masks. Avoids configuration that + most integrations may not need. +- **Cons:** Coverage varies from source behavior by color direction and HDR domain, and the + fixed policy cannot be tuned for different content. + +**Adopt FSR parity** + +- **Pros:** Allows source-equivalent mask generation and content-specific tuning. Makes HDR + and binary-mask behavior explicit. +- **Cons:** More controls can be misconfigured. Optional transforms add shader work when + enabled; the cost is unmeasured. + +**Next action:** **Target parity selectively.** Add bench-only generation controls and +compare mask captures plus temporal artifacts on `examples/05-transparency`; retain the +current policy as a convenience preset if it remains competitive. + +#### Current-frame upsample (`accumulate.ts`) + +- **Current status:** Custom replacement. +- **Local implementation:** Jitter-positions an analytic separable Lanczos2 kernel over a 3×3 + footprint. +- **FSR 3.1.5 behavior:** Uses a radial approximate Lanczos2 kernel with adaptive bias. +- **Why it differs / evidence confidence — Unclear:** The local kernel has an observably + compact footprint, but no evidence establishes why it replaced the direct-port behavior + or whether performance motivated the divergence. + +**Keep the local path** + +- **Pros:** Straightforward implementation with a bounded footprint and analytic kernel math. +- **Cons:** Sharpness, ringing, and subpixel response differ from the source. Compactness + does not itself prove better GPU performance. + +**Adopt FSR parity** + +- **Pros:** Restores source reconstruction behavior and adaptive bias across scale ratios. +- **Cons:** Different math and sampling may increase or decrease GPU cost depending on the + device; that must be measured. + +**Next action:** **Benchmark.** Precompile local and source-style kernels, compare fixed +visual captures on `examples/04-aliasing-torture`, and measure equivalent workloads in +`bench/` at every supported scale. + +#### History reconstruction (`accumulate.ts`) + +- **Current status:** Custom replacement. +- **Local implementation:** Uses five-fetch Catmull-Rom history sampling. +- **FSR 3.1.5 behavior:** Uses custom bicubic Lanczos reconstruction. +- **Why it differs / evidence confidence — Likely:** Five fetches plausibly reduce texture + sampling work, but no local measurement demonstrates a performance advantage or + acceptable quality tradeoff. + +**Keep the local path** + +- **Pros:** Uses a small known fetch count and preserves current history appearance. +- **Cons:** Blur, ringing, and rejection input can differ from the source. The expected + sampling savings have not been measured. + +**Adopt FSR parity** + +- **Pros:** Aligns history reconstruction with the source resolver and may improve temporal + detail or stability. +- **Cons:** Could increase sampling/instruction cost and may expose different ringing. Both + effects require A/B evidence. + +**Next action:** **Benchmark.** Compare precompiled filters under camera motion, +disocclusion, and thin-feature stress; report per-pass distributions and upscaler +compute-pass sum, not fetch count alone. + +#### Accumulation model (`accumulate.ts`) + +- **Current status:** Custom replacement. +- **Local implementation:** Stores a capped display-resolution sample counter in history + alpha and derives blending around that counter. +- **FSR 3.1.5 behavior:** Uses render-resolution-informed, motion-dependent history/current + weights and stores lock lifetime in history alpha. +- **Why it differs / evidence confidence — Unclear:** The local state and weights are + internally coupled, but no verified rationale explains replacing the source model. + Since this began as a direct port, the replacement should not be treated as accepted + solely because it exists. + +**Keep the local path** + +- **Pros:** Already integrated with local locks, rectification, and debug views. Avoids a + coordinated resource/state rewrite. +- **Cons:** Convergence and motion response differ from the source, and isolated tuning + cannot recover source behavior safely. + +**Adopt FSR parity** + +- **Pros:** Restores the source's coordinated weighting and render-resolution confidence + model. +- **Cons:** Requires accumulation, locks, luma instability, rectification, and packing to + change together. Cost and quality cannot be attributed to one sub-change. + +**Next action:** **Target parity selectively.** Build a coordinated resolver variant after +its input signals exist; compare the complete local and parity models rather than mixing +incompatible alpha/state semantics. + +#### Rectification (`accumulate.ts`) + +- **Current status:** Custom replacement. +- **Local implementation:** Uses an unweighted fixed-gamma YCoCg box, widened strongly by + custom locks. +- **FSR 3.1.5 behavior:** Uses a weighted, dynamically scaled box driven by motion, depth, + accumulation, reactivity, shading, locks, and luma instability. +- **Why it differs / evidence confidence — Unclear:** The local variance clip is compact, + but no measured performance rationale or evidence of intentional acceptance is recorded. + +**Keep the local path** + +- **Pros:** Fewer dependent source signals and simpler tuning/debugging. +- **Cons:** Can over-clamp or over-protect history where source bounds adapt to scene + conditions. + +**Adopt FSR parity** + +- **Pros:** Makes history bounds respond to the same confidence signals as the source, + potentially improving stability/detail balance. +- **Cons:** Adds dependencies and coordinated tuning. Evaluating it before the required + source signals exist would produce a misleading result. + +**Next action:** **Target parity selectively.** Implement it only in the coordinated +resolver variant, with debug visualization for every signal that scales the box. + +#### Exposure history + +- **Current status:** Diverges and has a domain-consistency gap. +- **Local implementation:** Stores locally exposed, invertible-tonemapped history without + correcting reprojected history when auto, fixed, or external conditioning exposure + changes. `exposureTexture` is a conditioning override, not host `preExposure`. +- **FSR 3.1.5 behavior:** Applies `DeltaPreExposure()` and `Exposure()` while moving history + into the current working domain, then removes only `Exposure()` before output and + preserves host `preExposure`. +- **Why it differs / evidence confidence — Unclear:** The local representation lacks + previous/current exposure metadata. This is not a demonstrated optimization; it is a + domain mismatch that can compare current and history samples under different effective + exposures. + +**Keep the local path** + +- **Pros:** Requires no new exposure state and preserves the current API behavior. +- **Cons:** Changing exposure can cause pumping or trails. Treating `exposureTexture` as + host pre-exposure also divides out a domain the caller may expect preserved. + +**Adopt FSR parity** + +- **Pros:** Makes history comparisons domain-consistent and supports a proper host + pre-exposure contract. +- **Cons:** Requires explicit previous/current exposure state and careful migration of the + existing conditioning-exposure API. + +**Next action:** **Target parity.** First add deterministic tests/captures for step and +ramped exposure changes. Then track the host pre-exposure ratio separately, correct +reprojected history, and remove only internal/app conditioning exposure on output. + +#### Locks + +- **Current status:** Custom replacement. +- **Local implementation:** Detects display-resolution peakiness/contrast, writes separate + RGBA16F lock ping-pong textures, and applies custom luma-break and history-boost behavior. +- **FSR 3.1.5 behavior:** Creates locks from render-resolution ridge patterns, carries + lifetime in history alpha, and uses different decay and break signals. +- **Why it differs / evidence confidence — Unclear:** The local system addresses + thin-feature stability, but no verified reason shows why source lock semantics were + replaced or that the memory/quality tradeoff is preferable. + +**Keep the local path** + +- **Pros:** Already protects some thin features and has dedicated debug visibility. It is + compatible with the current local accumulation model. +- **Cons:** Coverage, persistence, memory traffic, and rectification coupling differ + materially; permissive locks can preserve stale history. + +**Adopt FSR parity** + +- **Pros:** Restores source ridge detection and lock lifetime semantics within the intended + resolver. +- **Cons:** Cannot be transplanted independently because source history alpha has a + different meaning. The net resource/performance result needs whole-resolver measurement. + +**Next action:** **Target parity selectively.** Keep local locks for the current resolver, +and replace them only inside the coordinated parity resolver tested on +`examples/04-aliasing-torture`. + +#### Shading change + +- **Current status:** Custom replacement. +- **Local implementation:** Compares reprojected history luma with a current 3×3 + neighborhood mean and variance. +- **FSR 3.1.5 behavior:** Builds a dedicated signed-difference SPD from corrected + current/previous luma and evaluates multiple mips. +- **Why it differs / evidence confidence — Likely:** The local heuristic avoids a pyramid, + its resources, and extra history inputs. That architectural saving is visible, but its + GPU benefit and quality tradeoff have not been measured. + +**Keep the local path** + +- **Pros:** Avoids the dedicated SPD resources/work and is already observable through a + debug view. +- **Cons:** Lacks scale separation and can confuse high-frequency motion or aliasing with a + real shading change. + +**Adopt FSR parity** + +- **Pros:** Provides source multi-scale detection and corrected history/current luma + comparison. +- **Cons:** Adds a dedicated pyramid and dependencies. It may cost more GPU time; the + amount is unknown. + +**Next action:** **Benchmark.** Add the signed-difference SPD as a structural variant and +compare false positives, response to controlled lighting changes, per-pass distributions, +and upscaler compute-pass sum. Do not reuse nonexistent local exposure mips. + +#### Luma instability + +- **Current status:** Missing. +- **Local implementation:** Has no equivalent persistent signal; custom locks overlap with + only some symptoms. +- **FSR 3.1.5 behavior:** Maintains a separate four-frame render-resolution luma history to + protect recurring subpixel luminance. +- **Why it differs / evidence confidence — Unclear:** No verified platform or performance + rationale is recorded. Local locks are not behaviorally equivalent evidence. + +**Keep the local path** + +- **Pros:** Avoids the four-frame luma-history resource and processing. +- **Cons:** Recurring subpixel luma may be rectified or accumulated incorrectly even with + locks enabled. Any savings are unmeasured. + +**Adopt FSR parity** + +- **Pros:** Restores the source signal used to protect temporally unstable luminance. +- **Cons:** Adds state and bandwidth and only has full meaning when source rectification + and accumulation consume it. + +**Next action:** **Target parity selectively.** Implement and evaluate this signal with the +SPD and coordinated resolver variants, not as a standalone toggle in the local blend. + +#### Luma and exposure analysis (`luminancePyramid.ts`) + +- **Current status:** Custom scalar replacement; despite its filename, it is not a pyramid. +- **Local implementation:** Serially samples a fixed 32×32 grid and reduces it to one + log-average exposure value. +- **FSR 3.1.5 behavior:** Prepare-inputs writes current luma and farthest depth. Luma SPD + writes `FrameInfo`, stored coarse log-luma/luma, and half-resolution farthest depth. + Shading-change SPD is a separate signed-difference pyramid. +- **Why it differs / evidence confidence — Rationale unclear; structural effect + verified:** The scalar path avoids SPD resources and work. It is also serial and + undersampled, and cannot feed source spatial luma/depth signals. No evidence establishes + that performance motivated the replacement, and avoided work does not establish a + measured speedup. + +**Keep the local path** + +- **Pros:** Minimal resource graph for basic exposure metering and no SPD allocation. +- **Cons:** Fixed sparse sampling can miss content, serial reduction may be inefficient, + and one scalar cannot support source shading/reactivity signals. + +**Adopt FSR parity** + +- **Pros:** Supplies spatial luma/depth data and the frame state expected by later source + stages. +- **Cons:** Adds resources and dispatch work. If only scalar exposure is needed, full SPD + may not provide enough quality value to justify its cost. + +**Next action:** **Target parity selectively.** Keep scalar metering available for the +local resolver while building luma SPD for the parity graph; compare exposure stability, +source-signal quality, resource footprint, per-pass distributions, and upscaler +compute-pass sum. + +#### State packing + +- **Current status:** Custom replacement coupled to the local resolver. +- **Local implementation:** Stores sample count in history alpha and uses two RGBA16F + textures for lock state. +- **FSR 3.1.5 behavior:** Uses presentation-resolution RGBA16F history with lock lifetime + in alpha, render-resolution ping-ponged R8 accumulation, render-resolution RGBA16F + four-frame luma history, and transient presentation-resolution R8 new locks. +- **Why it differs / evidence confidence — Unclear:** Each layout follows its resolver's + data flow. No measured memory/bandwidth rationale establishes that local packing is + better, and repacking alone would not restore source behavior. + +**Keep the local path** + +- **Pros:** Matches current alpha semantics and lock implementation without migration. +- **Cons:** Uses materially different memory and bandwidth, including large lock textures; + its relative cost is unknown. + +**Adopt FSR parity** + +- **Pros:** Aligns resources with source accumulation, locks, and luma instability and may + reduce some state sizes. +- **Cons:** Adds other source resources and changes lifetimes/resolutions. Format-level + comparisons outside the complete graph would be misleading. + +**Next action:** **Benchmark.** Derive packing from each complete resolver variant, then +measure memory footprint, per-pass distributions, and upscaler compute-pass sum; do not add +public packing flags. Use a separate frame-level profiler if total frame GPU time is +required. + +#### Output (`rcas.ts`, `blit.ts`) + +- **Current status:** Adapted port with a material output-domain divergence. +- **Local implementation:** Fixes output to ACES + sRGB in `rgba8unorm`. +- **FSR 3.1.5 behavior:** Removes internal/app `Exposure()` while preserving host + `preExposure`, returning the caller's input color/pre-exposure domain and leaving + presentation to integration. +- **Why it differs / evidence confidence — Likely:** A fixed display transform creates a + directly presentable three texture. This is a plausible integration convenience, not a + measured optimization or evidence that source-domain output was intentionally rejected. + +**Keep the local path** + +- **Pros:** Simple direct presentation and consistent current demo output. +- **Cons:** Prevents alternate tone mapping, gamut/transfer choices, HDR output, and later + linear post-processing. + +**Adopt FSR parity** + +- **Pros:** Preserves caller color semantics and supports composition before one final + presentation transform. +- **Cons:** Requires integrations to own or select the final transform and may require a + higher-precision output resource. + +**Next action:** **Target parity selectively.** Add a linear/HDR output variant and compare +it through exactly the same final presentation transform as the local output; retain fixed +display output only as an explicit convenience mode. + +#### Raw WGSL and three.js integration + +- **Current status:** Adapted port and required platform integration. +- **Local implementation:** Runs hand-written WGSL directly on three's WebGPU device and + returns a three texture. +- **FSR 3.1.5 behavior:** Uses the FidelityFX host/backend abstraction around the source + shaders and resources. +- **Why it differs / evidence confidence — Verified:** The library targets WebGPU and + three.js directly. Browsers do not expose the native FidelityFX backend contract, so this + adaptation is necessary. This is comparable to frame generation remaining out of scope + because browser swapchain pacing is unavailable. + +**Keep the local path** + +- **Pros:** Fits the supported platform, keeps WGSL inspectable, and integrates with three + textures and command ordering. +- **Cons:** Depends on private three backend access and requires local responsibility for + bindings, resource layouts, validation, and GPU compatibility. + +**Adopt FSR parity** + +- **Pros:** Algorithm/resource semantics can still be matched within WGSL. +- **Cons:** Adopting the native FidelityFX host/backend packaging is not directly available + in this WebGPU/three environment and would not remove the need for an adapter. + +**Next action:** **Keep.** Retain raw WGSL and the three adapter, while validating every +parity experiment on a real WebGPU device and keeping algorithm differences separate from +platform glue. + +### Parity evaluation plan + +#### Principle + +Choose the quality/performance Pareto result. Source parity is a hypothesis to test, not an +end by itself. + +Parity is preferred when it improves quality without unacceptable measured performance or +platform cost. If a local divergence remains, document the measured performance or +verified platform justification here rather than inferring intent from the implementation. + +#### Canonical harness + +- Use `bench/` for controlled scene, timing, resolution, and presentation comparisons. +- Use `examples/04-aliasing-torture` as a visual fixture for reconstruction, history, + locks, luma instability, and rectification stress. +- Use `examples/05-transparency` as a visual fixture for reactive generation, dilation, + T&C, and motion-divergence interactions. +- Keep authoritative GPU timing in `bench/`; the specialized examples do not currently + provide equivalent timing coverage. + +#### Experiment architecture + +- Keep experimental controls bench-only and unexported until a decision is made. Do not + add provisional controls to public `RuntimeSettings` or exported types. +- Small math changes may use runtime flags for rapid visual A/B. +- Final performance comparisons must use separate precompiled local/parity pipelines. + Dynamic branches can bias instruction count, register pressure, and timing. +- Resource or pass-graph changes require distinct pipeline variants and reconfiguration, + not one runtime branch. +- Split-screen is useful for visual comparison, but its timings are invalid because both + variants share the frame workload. + +Not every recommendation can be implemented as “one flag”: math variants can use temporary +runtime flags, while structural changes need separate resource graphs and pipelines. + +#### Stage experiments + +1. **Harden measurements first.** Add a deterministic camera/frame sequence, fixed + timestep and resolution, clear stale timer labels, collect distributions, and disable + debug views for timing. +2. **Math and domain fixes.** Evaluate exposure-history correction, RCAS math/defaults, and + reactive generation/dilation. +3. **Reconstruction filters.** Compare the current-frame kernel and history filter. +4. **Structural input/reactivity.** Compare reconstructed-depth scatter, farthest depth, + motion divergence, and T&C. +5. **Temporal stability graph.** Add luma SPD, shading-change SPD, and luma instability. +6. **Coordinated resolver variant.** Evaluate accumulation, rectification, locks, and state + packing as one compatible model. +7. **Output domain.** Route a linear/HDR variant through the same final presentation + transform for a fair visual comparison. + +#### Measurement protocol + +- Authoritative GPU timing requires an adapter with WebGPU `timestamp-query` support. + Record the GPU, adapter, browser/version, physical resolution, and DPR with every result. + If timestamp queries are unavailable, report GPU timing as unavailable; FPS is not a + substitute. +- Treat the current `GpuTimer` output as a spot-check, not authoritative statistics. It + exposes only the latest asynchronous result map, can retain stale values while readback + is pending, and does not currently collect distributions. +- Before comparing variants, add fresh-sample identifiers, clear removed pass labels on + graph changes, and store per-frame samples for each pass. +- Fix physical resolution, DPR, timestep, seeded scene state, and camera path. Disable or + fix exposure adaptation unless exposure behavior is the variable under test. +- Reset history and jitter so variants begin from identical state. +- Use 240 warm-up frames, then collect 600 fresh timer samples per block. +- Test 1× Native AA, 1.5× Quality, 2× Performance, and 3× Ultra Performance. +- Run four alternating ABBA repetitions to reduce order and thermal bias. +- Run one variant per timing block. +- Report median and p95 per pass plus the **upscaler compute-pass sum**. Do not call that + sum total frame GPU time: it excludes scene rendering, presentation, and other graph + overhead. Treat FPS as secondary. +- Capture the same post-reset frames for both variants. +- Use debug views and fixed captures for quality evidence. Add image-difference or FLIP + analysis later if visual decisions remain ambiguous. + +#### Decision rule + +- Before each experiment, declare its artifact gates and numeric median/p95 GPU budget. +- Adopt parity only when it passes the visual gates and stays within that declared budget. +- Keep local behavior only when it demonstrates a measured performance or platform + advantage and introduces no fixture regression. +- If each path wins different scenes, keep them as explicit variants or iterate on a + hybrid. Do not average away a visible regression; record the evidence and rationale in + this README. + +## Operational notes + +### Custom thin-feature locks + +The local lock heuristic is intended to reduce thin-feature dimming and shimmer from +rectification. `accumulate.ts` keeps display-resolution lock state (r = lifetime, +g = locked luma), reprojects it with motion, and derives candidates from neighborhood +`peakiness × contrast`. A lock widens the local rectification box +(`LOCK_CLAMP_RELAX`) and increases history influence (`LOCK_HISTORY_BOOST`). + +This is custom behavior, not FSR 3.1.5's ridge-pattern lock system. It can preserve detail +or preserve stale history depending on content and tuning. Toggle +`settings.lockThinFeatures` (`FLAG_LOCKS`) and inspect `DebugView.Locks`. Tune cautiously: +more permissive locks can increase trails; stricter locks can reduce their intended effect. + +### Local exposure conditioning + +`luminancePyramid.ts` reduces a fixed 32×32 scene sample grid to one log-average value, +maps it to an auto-exposure target, clamps that target, and eases toward it. Fixed +`settings.exposure` and external `exposureTexture` values are selected as supplied rather +than passed through the auto-target clamp. This local conditioning `exposure` should not be +conflated with FSR's caller-provided host `preExposure`. `accumulate.ts` applies the local +factor before the invertible tonemap; `rcas.ts` or `blit.ts` later divides by the current +factor before the local display transform. + +This conditions the local accumulation range, but it does not guarantee unchanged final +brightness. In particular, stored history is not corrected when exposure changes, so +adaptation can cause pumping or trails. Toggle `settings.autoExposure` +(`FLAG_AUTO_EXPOSURE`); with it off, `settings.exposure` follows the same local path. +Passing `dispatch({ exposureTexture })` overrides both values with the texture's red +channel, but still feeds this local conditioning/history/display path. It is not a way to +declare AMD-style host `preExposure`; using it as one will divide that factor back out +during local output and will not provide `DeltaPreExposure()` history correction. +`DebugView.Exposure` visualizes clamped exposed luma, not the selected exposure scalar. + +### Custom shading-change heuristic + +The local heuristic compares reprojected history luma with the current 3×3 neighborhood +mean, normalized by neighborhood variance. When it responds, non-locked history is aged by +`SHADING_AGE`; locked pixels suppress this aging. This can identify some lighting/material +changes, but it can also respond to high-frequency motion or aliasing. It is not FSR +3.1.5's signed-difference SPD and multiple-mip analysis. + +Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`) and inspect +`DebugView.ShadingChange`. `SHADING_LO`, `SHADING_HI`, and `SHADING_AGE` are heuristic +tuning controls, not source constants or guarantees. + +### Reactive masks + +Pass a render-resolution red-channel mask in `[0, 1]` as `dispatch({ reactive })`. +Locally, stronger values suppress lock formation, sharply reduce accumulation, and bias +the blend toward the current frame. If no mask is supplied, a 1×1 zero texture is bound and +the shader branch is flag-gated; this avoids the full reactive behavior but is not a +literal guarantee of zero overhead. + +Alternatively, pass `dispatch({ reactiveOpaqueColor })` with an opaque-only render and +`generateReactive.ts` will derive a mask using a fixed component-max color difference, +threshold, scale, and cap. `examples/05-transparency` demonstrates both inputs. Render the +opaque pass with the same jitter as the final frame or high-contrast edges can produce +false reactivity from subpixel misalignment. Unlike the FSR 3.1.5 helper, the local API +does not select component-max versus vector-length difference, optional tone-map/ +inverse-tone-map transforms, or binary output value. It also does not reproduce the full +prepare-reactivity, T&C, or motion-divergence behavior. ## Debugging Set `settings.debugView` (`DebugView`) to render pipeline internals instead of the final image: motion vectors, disocclusion mask, linearized depth, accumulation age, locks, auto-exposed luminance, or the shading-change factor. When integrating a new scene, check in this order: -1. **Motion vectors** — static scene + moving camera should produce smooth gradients, no per-object noise (if objects flash, their previous model matrices aren't tracked — did you bypass the `velocity` node?). -2. **Disocclusion** — should outline moving silhouettes, thin and stable. Full-screen flashing means depth linearization flags are wrong (reversed-depth mismatch). -3. **Accumulation age** — should saturate to white within ~a second when still, and reset along disocclusion trails. -4. **Locks** — should light up on thin high-contrast features (grid lines, wire/fence edges, specular silhouettes) and stay black on flat surfaces. Locks everywhere ⇒ thresholds too low (expect ghosting); nothing lit ⇒ thresholds too high (thin features will dim). -5. **Exposure** — the exposed scene luminance should read near an even mid-grey regardless of how bright/dark the scene is (that is auto-exposure normalizing it). All-black ⇒ exposure driven to its floor (scene far too bright), all-white ⇒ driven to its ceiling (scene far too dark). -6. **Shading change** — black on a static, steadily-lit scene; lights up (and fades over a few frames) on surfaces whose shading actually changes — a moving specular highlight, a light animating, a material shifting. Lit everywhere on a still scene ⇒ `SHADING_LO` too low (stable surfaces will re-converge needlessly and shimmer); never lighting up on an obvious lighting change ⇒ too high. -7. **Reactivity** — the caller's reactive mask, as accumulate sees it: white where you flagged transparents/particles, black on opaque geometry. If it's misaligned or empty, the mask isn't being authored/passed correctly (wrong resolution, not set before `dispatch`). +1. **Motion vectors** — a static scene with a moving camera should produce smooth + gradients and no per-object noise. Per-object flashing often points to previous-model + tracking or a bypassed `velocity` node. + +2. **Disocclusion** — should outline moving silhouettes, thin and stable. Full-screen + flashing often points to incorrect depth linearization flags or a reversed-depth + mismatch. + +3. **Accumulation age** — should trend toward white when still and reset along + disocclusion trails. Whitening time depends on frame rate and `maxAccumulation`. + +4. **Locks** — should usually concentrate on thin high-contrast features and remain low on + flat surfaces. Locks everywhere suggests permissive thresholds and trail risk; nothing + lit suggests the heuristic is not engaging. + +5. **Exposure** — shows clamped exposed luma, not the exposure scalar. Under auto + exposure, the metered geometric-mean reference should trend toward mid-grey; individual + pixels are not expected to. All-black or all-white can indicate view saturation, + invalid luma input, or a metering-range mismatch, but does not show that a fixed or + external exposure value was clamped. + +6. **Shading change** — should remain mostly dark on a static, steadily lit scene and + respond temporarily to changed lighting or materials. Broad response while still + suggests `SHADING_LO` is too low; no response to an obvious change suggests it may be + too high. + +7. **Reactivity** — shows the mask as accumulation sees it: white where transparents or + particles were flagged, black on opaque geometry. If it is unexpectedly misaligned or + empty, check its resolution, authoring pass, and whether it was set before `dispatch`. From 1b1be7b46019bcb01c17086509d86fb699af469a Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 11:38:38 +0900 Subject: [PATCH 02/22] feat: parity candidate bundles GPU-verified, benchmarked, and documented Fix five defects that prevented the authored source-parity bundles from ever compiling on a device (isInf, reserved keyword, r8unorm storage, unused sampler binding, /0.5 velocity falloff), fix the cross-variant analyzeAbba label crash so production-vs-candidate A/B runs work, and add --help to the benchmark runner. Measured (ratio 2, ABBA, noise floor <=2%): filter bundle +36%, structural +6.5% incremental, full SPD resolver +76% GPU compute vs production, with no visual regression or win on Q0/Q1/Q3 captures. No bundle adopted; evidence in bench/PARITY-DECISIONS.md and PARITY.md (new consumer-facing parity rationale). Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 6 +- PARITY.md | 103 ++ README.md | 12 +- bench/PARITY-CANDIDATES.md | 296 ++++ bench/PARITY-DECISIONS.md | 52 + bench/PARITY-PROGRESS.md | 116 ++ bench/THREE-TEMPORAL-COMPARISON.md | 15 +- bench/results/.gitignore | 4 + bench/results/README.md | 66 + bench/results/experiments/README.md | 120 ++ bench/results/experiments/e00-harness.json | 1764 ++++++++++++++++++++ bench/src/BenchPipeline.ts | 913 +++++++++- bench/src/BenchScene.ts | 164 +- bench/src/benchmark/BenchmarkResolver.ts | 211 +++ bench/src/benchmark/api.ts | 235 +++ bench/src/benchmark/clock.ts | 34 + bench/src/benchmark/collector.ts | 236 +++ bench/src/benchmark/config.ts | 89 + bench/src/benchmark/environment.ts | 78 + bench/src/benchmark/scenarios.ts | 320 ++++ bench/src/benchmark/variants.ts | 257 +++ bench/src/main.ts | 231 ++- bench/src/types/benchmark.d.ts | 273 +++ examples/README.md | 2 +- examples/shared/boot.ts | 9 +- package.json | 3 + scripts/benchmark-contract.mjs | 145 ++ scripts/benchmark-contract.test.mjs | 171 ++ scripts/compare-rcas.mjs | 300 ++++ scripts/run-benchmark.mjs | 1746 +++++++++++++++++++ src/UpscalePass.ts | 6 +- src/Upscaler.ts | 713 +++++++- src/UpscalerNode.ts | 6 +- src/internal/ComputePass.ts | 30 +- src/internal/GpuTimer.ts | 279 +++- src/shaders/README.md | 173 +- src/shaders/blit.ts | 18 +- src/shaders/candidateDebug.ts | 108 ++ src/shaders/candidateFilters.ts | 280 ++++ src/shaders/candidateInputs.ts | 392 +++++ src/shaders/candidateTemporal.ts | 521 ++++++ src/shaders/common.ts | 22 - src/shaders/debug.ts | 4 +- src/shaders/easu.ts | 15 +- src/shaders/rcas.ts | 96 +- src/shaders/shaders.test.ts | 449 ++++- src/types.ts | 17 + 47 files changed, 10723 insertions(+), 377 deletions(-) create mode 100644 PARITY.md create mode 100644 bench/PARITY-CANDIDATES.md create mode 100644 bench/PARITY-DECISIONS.md create mode 100644 bench/PARITY-PROGRESS.md create mode 100644 bench/results/.gitignore create mode 100644 bench/results/README.md create mode 100644 bench/results/experiments/README.md create mode 100644 bench/results/experiments/e00-harness.json create mode 100644 bench/src/benchmark/BenchmarkResolver.ts create mode 100644 bench/src/benchmark/api.ts create mode 100644 bench/src/benchmark/clock.ts create mode 100644 bench/src/benchmark/collector.ts create mode 100644 bench/src/benchmark/config.ts create mode 100644 bench/src/benchmark/environment.ts create mode 100644 bench/src/benchmark/scenarios.ts create mode 100644 bench/src/benchmark/variants.ts create mode 100644 bench/src/types/benchmark.d.ts create mode 100644 scripts/benchmark-contract.mjs create mode 100644 scripts/benchmark-contract.test.mjs create mode 100644 scripts/compare-rcas.mjs create mode 100644 scripts/run-benchmark.mjs create mode 100644 src/shaders/candidateDebug.ts create mode 100644 src/shaders/candidateFilters.ts create mode 100644 src/shaders/candidateInputs.ts create mode 100644 src/shaders/candidateTemporal.ts diff --git a/CLAUDE.md b/CLAUDE.md index 73650a4..201a742 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -109,7 +109,7 @@ Read `Upscaler.ts` and `bench/src/BenchPipeline.ts` together first — the secon - These are **private**. `threeWebGPU.ts` throws loudly if the shape changes. **If you bump three, re-verify these two accessors first** — they're the most likely thing to break on upgrade. - **Output is a three `StorageTexture`** — the upscaler writes into it, the caller samples it like any texture. Presenting is ordinary three code (a fullscreen quad). - **One shared 256-byte constants UBO** (`ConstantsBuffer` ↔ `WGSL_CONSTANTS` in `common.ts`), bound at `@group(0) @binding(0)` in every pass, written once per frame. **The two layouts must stay byte-for-byte in sync** — f32/u32 indices in `ConstantsBuffer.ts` map to documented byte offsets in the WGSL struct. If you add/reorder a field, change **both** or everything silently corrupts. -- **Color spaces:** temporal accumulation happens in invertible-tonemap space (`c/(1+max(c))`, FSR2's firefly guard); EASU/RCAS run display-referred; every path exits through one shared ACES+sRGB `displayTransform`. That's why the present quad must NOT re-tonemap (see landmines). +- **Color spaces:** temporal accumulation happens in invertible-tonemap space (`c/(1+max(c))`, FSR2's firefly guard), then RCAS/blit reverse that conditioning and write `rgba16float` in the caller's linear/HDR domain. The upscaler never applies ACES, an output transfer function, or another presentation transform. EASU leaves its caller-provided input domain unchanged. Presentation belongs to the consuming renderer/post graph. - **Jitter/velocity:** projection is jittered via `camera.setViewOffset` (same as three's TRAA). Motion vectors must be **jitter-free**, so the scene's `velocity` node gets `upscaler.unjitteredProjectionMatrix` (a stable Matrix4 instance whose contents refresh each `beginFrame`). --- @@ -122,7 +122,7 @@ These were discovered by reading three's source; they're non-obvious and easy to 2. **`StorageTexture` defaults `generateMipmaps = true`** → three allocates a mip chain and the storage view breaks. The output texture sets `generateMipmaps = false`, and storage views are pinned to `{ baseMipLevel: 0, mipLevelCount: 1 }` (`_outputView()`). Keep both. 3. **Combined depth-stencil formats need a depth-only view** (`{ aspect: 'depth-only' }`) to bind as `texture_depth_2d`. `_encodeTemporal` branches on `format.includes('stencil')`. 4. **Reversed depth:** we read `renderer.reversedDepthBuffer` and set a flag; `linearizeDepth` in `common.ts` handles both conventions + ortho. If depth debug view is full-screen-flashing, this flag is the suspect. -5. **Present quad setup:** `NoToneMapping` + `LinearSRGBColorSpace` on the renderer, and `depthTest/depthWrite/fog = false` on the quad material. The FSR output is already display-referred sRGB; re-encoding double-applies the transform. +5. **Present quad setup:** `depthTest/depthWrite/fog = false` on the quad material. The FSR output is linear/HDR, so the renderer's normal output transform should run when presenting to screen. The examples choose `ACESFilmicToneMapping` + `SRGBColorSpace`; library users own that policy. 6. **Bench Vite needs `build.target: 'esnext'`** — `main.ts` uses top-level `await renderer.init()`. 7. **MRT output count must match the RT attachment count.** Rendering the scene into a `count: 2` `RenderTarget` while `setMRT(null)` (or an MRT without the velocity output) leaves **color attachment 0 unwritten → black output**. This is why non-temporal bench modes were black on first GPU boot. Fix: size the RT to the mode (temporal → `count: 2` + `mrt({ output, velocity })`; everything else → `count: 1` + `mrt({ output })`). See `BenchPipeline.configure`/`render`. **Any integration (incl. examples / `UpscalePresenter`) must keep these two counts in lockstep.** 8. **WGSL parses `a < b, c > d` as a template argument list.** `select(d < bestDepth, d > bestDepth, reversed)` failed to compile (`parsed as template list`). Wrap comparisons that put `<` … `>` in one expression in parentheses: `select((d < bestDepth), (d > bestDepth), reversed)`. Watch for this whenever a shader compares with both `<` and `>` nearby. @@ -182,7 +182,7 @@ variant on `06-screenspace-gi`, and expose new toggles in `02-fsr1-vs-fsr3`. Rem - **`upscaleScene(scene, camera, options)`** — a thin convenience, **not** a separate class: it builds `pass(scene, camera)` with a `{ output, velocity }` MRT at `1/ratio` and hands the texture nodes to `upscale(...)` — the same shape as three's `taau(pass.getTextureNode('output'), …)`. So the scene renders in-graph as an FSR3 input, jitter and all. `post.outputNode = upscaleScene(scene, camera)`. Examples `07-tsl-node`, `08-tsl-compose` (`.mul(vignette)`); the full SSGI/SSR stack is `09-kitchen-sink`. - **`upscaleSpatial(color, options)`** — color-only **spatial** (FSR1/EASU) node for inputs with no motion data: no depth/velocity/camera, no history, no reconstruction. A thin facade over `path: 'spatial'` with a stand-in camera (EASU reprojects nothing; `_writeConstants` still stages `near`/`far`, which the shader ignores, so any finite values keep NaN out of the UBO). It exists so the "I only have a color texture" case has a clean door instead of `upscale(color, null, null, camera, { path: 'spatial' })` — and so defaulting the *temporal* node to jitter-on isn't a trap. - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity now `console.warn`s once (was a silent no-op). `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. - - Color path (both): the boot renderer's `NoToneMapping` + `LinearSRGBColorSpace` makes the PostProcessing output transform identity, so the node passes its already-sRGB texture straight through. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). + - Color path (both): the node emits linear/HDR color. When it is the final graph node, three's RenderPipeline applies the renderer's configured tone mapping and output color space; otherwise it can feed later linear post-processing. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). - **Resolved** (was an open question): the composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. - ~~MSAA-input support~~ — **dropped by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). - **Phase 5 — performance & the one remaining quality item:** ~~merge dilate+depth-clip into one pass~~ **done** (`reconstruct.ts` — one render-res dispatch, one fewer intermediate round-trip; GPU-verified disocclusion unchanged). Remaining, all deferred as of 2026-07-10 (the rename + node-parity + exposure work above was the safe, high-value batch; these were held back so a core-path change wouldn't ride onto the rename merge): diff --git a/PARITY.md b/PARITY.md new file mode 100644 index 0000000..dc8594e --- /dev/null +++ b/PARITY.md @@ -0,0 +1,103 @@ +# Why @pmndrs/upscaler is not a line-for-line FSR 3.1.5 port + +`@pmndrs/upscaler` derives from AMD FidelityFX FSR — FSR1's EASU/RCAS directly, and an +FSR2/3-style temporal resolver written for WebGPU compute. It is **not** byte-for-byte +FSR 3.1.5, and that is a deliberate, *measured* position, not an unfinished port. This +document explains where we match upstream, where we diverge, and the evidence behind +each choice. + +Reference: FidelityFX SDK commit `60f4ea8` (FSR Upscaler 3.1.5). The full per-pass audit +lives in [`src/shaders/README.md`](src/shaders/README.md); raw benchmark evidence in +`bench/results/`. + +## The short version + +We implemented the source-style pipeline — the full FSR 3.1.5 pass graph, including +Lanczos2 reconstruction, deringed bicubic history, atomic depth scatter, motion +divergence, SPD-style mip chains, luma instability, and the coordinated source resolver — +as three cumulative candidate graphs inside this repo, GPU-validated them, and A/B +benchmarked them against the production path on deterministic scenes. + +**The source-style graphs cost +36% to +76% more GPU compute and produced no visible +quality improvement on our test scenarios.** The differences that exist are sub-4% RMSE +spread across edge detail, with no artifacts, ghosting, or convergence failures on either +side. On a library whose priority order is **performance > quality > realism**, that +result decides the question: the simplified production path ships; the source-parity +graphs remain in-repo as benchmark candidates. + +| Comparison (ratio 2, 1920×1080 display, Apple Metal, ABBA timing) | GPU compute | Δ | +| --- | --- | --- | +| production → source filter/reconstruction graph | 0.63 → 0.85 ms | **+36%** | +| … → + structural inputs/reactivity | 0.85 → 0.91 ms | **+6.5%** | +| production → full source SPD/resolver graph | 0.63 → 1.10 ms | **+76%** | + +Every delta was repeatable across four interleaved A/B blocks with noise floors ≤ 2%. +Reproduce with: +`node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant rcas-fsr315-limiter --comparison `. + +## Where we match upstream (adopted parity) + +- **RCAS numeric math.** The production sharpener uses FSR 3.1.5's lower limiter and the + corrected denoise luma/range math, adopted after A/B measurement showed parity was + free (E01). Denoise is opt-in pending evidence on representative noisy content. +- **Color and exposure domains.** Like upstream, the upscaler applies no tone mapping or + output encoding — input and output are the caller's linear/HDR domain, and internal + conditioning exposure is divided back out before output. An earlier internal ACES/sRGB + transform was removed for source alignment (E03). +- **Core temporal semantics.** Jittered projection (Halton), jitter-free motion vectors, + `prevUV = uv − motion` reprojection, invertible-tonemap accumulation with FSR2's + firefly guard, YCoCg variance clipping, disocclusion-driven history rejection, + luminance-stability locks, auto-exposure conditioning, reactive masks (explicit and + auto-generated from opaque-vs-final diff, FSR2-style) — the algorithmic lineage is + FSR's throughout. +- **EASU (spatial path).** The 12-tap edge analysis, anisotropic Lanczos kernel, tap + placement, and deringing follow `ffx_fsr1.h`; only language-level details differ + (native WGSL division/`inverseSqrt` instead of AMD's approximation helpers). + +## Where we diverge, and why + +**1. Platform constraints (WebGPU is not Vulkan/DX12).** Core WebGPU has no +floating-point storage-texture atomics (source depth scatter), no device-wide atomic +counter for single-pass downsampling (source SPD), no guaranteed f16 arithmetic, and no +swapchain pacing control (which rules out FSR3 frame generation entirely). The candidate +graphs prove these can be *emulated* — storage-buffer atomics, direct mip re-reads — but +the emulations are part of why the source graphs measure slower here. + +**2. Measured cost without measured benefit.** The fused single-pass depth +reconstruction/disocclusion, the 3×3-neighborhood shading detector, and the compact +accumulate pass are simplifications that survived because the source alternatives cost ++36–76% GPU time and the deterministic quality scenarios (static convergence, camera +motion, object-motion disocclusion) could not distinguish them visually. A divergence +is kept only while that remains true — the candidates stay in-repo precisely so this +can be re-tested as scenes, devices, or the library change. + +**3. Scope decisions.** Frame generation is out of scope (browser swapchain limits). +MSAA input is rejected by design — FSR's temporal path *is* the anti-aliaser. The +Transparency & Composition mask is accepted as a dispatch input for API compatibility +but currently maps to the reactive path; upstream's distinct softer T&C channel is +implemented in the structural candidate and will only be promoted with evidence that +the reactive path is insufficient for real content. + +## Honest limits of the evidence + +Current measurements are one adapter family (Apple Metal), one upscale ratio class, and +synthetic torture scenes over short deterministic sequences. The source graphs' +theoretical advantages target harder content — exposure ramps, transparency-heavy +scenes, noisy GI inputs, extreme motion — that the decisive runs did not exercise. The +benchmark harness (`npm run bench`, `scripts/run-benchmark.mjs`) exists so any of these +claims can be re-tested; a repeatable ≥5% result is treated as actionable, <3% as noise. + +## Open items we do intend to converge + +- **Host pre-exposure semantics** (`DispatchInputs.preExposureTexture`): correcting + reprojected history across changing host pre-exposure, as upstream's + `DeltaPreExposure()` does — a correctness fix for HDR apps, measured ~free in the + candidate graph. +- **AMD's viewport/depth-scaled disocclusion constant** in place of the current fixed + threshold guess, kept inside the (faster) fused reconstruction pass. +- **A coarse-mip shading-change detector** (the source design, GPU-proven in the + resolver candidate) to replace the 3×3-neighborhood heuristic, which can + false-positive on high-frequency content under heavy motion. +- **RCAS input-range investigation:** the source resolver's history made RCAS 47% + cheaper in measurement; if production accumulate emits ALU-hostile value ranges, + clamping them is a free performance win. diff --git a/README.md b/README.md index f7d0742..40cc7a0 100644 --- a/README.md +++ b/README.md @@ -26,10 +26,9 @@ The recommended integration is the **TSL node** — drop it in as the output of import * as THREE from 'three/webgpu'; import { upscaleScene, QualityMode } from '@pmndrs/upscaler'; -// The upscaler outputs display-ready sRGB, so make the post output transform -// identity: boot the renderer with NoToneMapping + LinearSRGBColorSpace. -renderer.toneMapping = THREE.NoToneMapping; -renderer.outputColorSpace = THREE.LinearSRGBColorSpace; +// The upscaler remains linear/HDR. Choose presentation independently. +renderer.toneMapping = THREE.ACESFilmicToneMapping; +renderer.outputColorSpace = THREE.SRGBColorSpace; const post = new THREE.PostProcessing(renderer); post.outputNode = upscaleScene(scene, camera, { quality: QualityMode.Quality }); @@ -101,9 +100,8 @@ upscaler.dispatch( { color: rt.textures[0], depth: rt.depthTexture, velocity: rt.textures[1], deltaTime }, camera, ); -// upscaler.outputTexture is a display-resolution three texture — present it on -// a fullscreen quad (already tonemapped + sRGB, so keep renderer tone mapping / -// output encoding off for that draw). +// upscaler.outputTexture is a display-resolution linear/HDR texture. Present it +// through the renderer's normal output transform or continue post-processing it. ``` Runtime knobs live on `upscaler.settings` (`sharpness`, `maxAccumulation`, `exposure`, `debugView`) and take effect next frame. `upscaler.resetHistory()` drops accumulation on camera cuts. diff --git a/bench/PARITY-CANDIDATES.md b/bench/PARITY-CANDIDATES.md new file mode 100644 index 0000000..a9e7c65 --- /dev/null +++ b/bench/PARITY-CANDIDATES.md @@ -0,0 +1,296 @@ +# Parity Candidate Test Guide + +## Current status + +**Update 2026-07-18:** all three bundles are now **GPU-compiled, validated, rendered, +timed, and captured** (after fixing five defects that had prevented any of them from +surviving `init()` — see PARITY-DECISIONS.md). Measured results and the reproduce +command live in `PARITY-DECISIONS.md`; raw evidence in `bench/results/raw/CANDIDATES/`. +The candidate A/B path is simply `run-benchmark.mjs --smoke --variant +--comparison ` — no candidate-specific manifest was needed (the cross-variant +`analyzeAbba` label crash was fixed to compare only shared pass labels). + +The production fallback is unchanged and remains the default. No bundle is adopted: +all three cost more compute than production (+36%, +43%, +76% cumulative) without a +demonstrated visual win on the deterministic scenarios. + +## What is already production behavior + +These changes are adopted; they are not part of the unmeasured candidate decision: + +- **Linear/HDR output:** internal ACES and sRGB presentation transforms were removed from + EASU, RCAS, blit, final output, and debug output. The upscaler returns caller-domain + `rgba16float`; the consuming renderer owns tone mapping and output encoding. +- **RCAS numeric parity:** production RCAS now uses the FSR 3.1.5 lower limiter and the + corrected denoise luma/range math. RCAS denoise remains opt-in because enabling it by + default changed high-contrast detail without a demonstrated noisy-input benefit. + +The production graph, adopted RCAS math, linear/HDR output contract, and opt-in denoise +policy are the fallback for every candidate below. + +## What was authored + +The candidates are cumulative and must be tested in this order: + +1. `source-filter-bundle-v1` +2. `source-structural-bundle-v1` = filter bundle + structural inputs/reactivity +3. `source-spd-resolver-bundle-v1` = structural bundle + SPD signals/coordinated resolver + +This ordering matters. Comparing each bundle only with production would show the total +effect, but adjacent comparisons are needed to attribute the incremental cost and image +change. Later adoption does not have to be all-or-nothing if evidence supports extracting +a compatible stage, but no stage should be transplanted before its dependencies and +state semantics are understood. + +### `source-filter-bundle-v1` + +**Changes** + +- Separates host `preExposure` from internal conditioning exposure, tracks previous and + current values, and moves reprojected history into the current domain before filtering. + Internal conditioning is removed at output while host pre-exposure remains. +- Replaces current-frame reconstruction with radial approximate Lanczos2 and adaptive + kernel bias. +- Replaces history sampling with a deringed 4×4 bicubic Lanczos reconstruction. +- Replaces fused local depth reconstruction with an atomic `u32` nearest-depth scatter, + an explicit pass boundary, and viewport/depth-scaled disocclusion. +- Selects an FSR1-style EASU shader using approximate reciprocal and reciprocal-square-root + helpers while preserving the existing 12-load topology. + +**Likely problems addressed** + +- Exposure adaptation or app pre-exposure changes can otherwise compare current and + history color in different domains, causing pumping, trails, or incorrect clipping. +- Source-style current/history filters may improve sub-pixel reconstruction, moving-edge + stability, and history deringing. +- Atomic previous-depth scatter more closely represents reprojected geometry and may + reduce incorrect disocclusion around moving silhouettes. + +**Why these changes are grouped** + +They form the smallest source-style reconstruction path that can retain the local +accumulation/lock model. Exposure correction must occur before history filtering, and the +depth result supplies the disocclusion signal used by that filter path. + +**Expected tradeoff and fallback** + +The 4×4 history filter, atomic scatter, and extra depth pass can cost more GPU time and +bandwidth. Approximate EASU math may reduce ALU cost, but introduces numeric error that +could soften or destabilize spatial edges. Production exact EASU, local reconstruction, +and local exposure/history behavior remain available by selecting no candidate. + +EASU is exercised only by the spatial path. The registered bundle metadata currently +selects the temporal path, so a normal temporal candidate run will compile the EASU +pipeline but will not measure its dispatch. A focused spatial manifest/runner is required +before making an EASU claim. + +**Evidence needed** + +- WebGPU compilation/validation for all supported ratios and odd render sizes. +- Per-pass and compute-sum timing against production, then visual review of convergence, + camera/object motion, silhouettes, and reset behavior. +- Controlled exposure steps and ramps that distinguish internal conditioning changes + from host pre-exposure changes. +- A separate spatial EASU comparison using edge/detail captures and numeric differences. + +### `source-structural-bundle-v1` + +**Changes** + +- Includes all of `source-filter-bundle-v1`. +- Prepares farthest depth and current luma alongside nearest depth/motion. +- Adds motion divergence to identify unreliable reprojection. +- Adds configurable source-style opaque-versus-final reactive generation. +- Max-dilates application reactivity into the aggressive history-reset/shading channel. +- Keeps Transparency & Composition (T&C) plus motion divergence in a distinct, softer + rectification channel instead of treating them as fully reactive. +- Adds render-resolution accumulation/reset state and atomic transient new-lock + preparation. + +**Likely problems addressed** + +- The current generated reactive mask has fixed policy and one aggressive meaning; + source-style controls should better classify transparent or composition changes. +- Treating T&C exactly like reactive content would discard too much history. The softer + channel should reduce ghosting without forcing every transparent pixel to current color. +- Farthest depth and motion divergence provide context for depth boundaries and + inconsistent motion that simple nearest-depth disocclusion misses. +- Prepared accumulation and new-lock state coordinate resets before display-resolution + accumulation. + +**Why these changes are grouped** + +These signals share the prepare-inputs and prepare-reactivity resource graph. Testing one +without its consumers would pay structural cost without evaluating the behavior it was +added to drive, while mixing them into the local mask channels would make attribution and +fallback semantics ambiguous. + +**Expected tradeoff and fallback** + +Expect extra textures/buffers, an additional prepare-reactivity dispatch, atomic lock +scatter, and more source reads. The likely quality upside is less transparency ghosting, +better silhouette handling, and more selective history distrust; the main quality risks +are over-resetting, noisy masks, flicker, and locks appearing on ordinary edges. The +filter candidate and production graph remain lower-cost fallbacks. + +**Evidence needed** + +- Incremental timing versus `source-filter-bundle-v1`, including `prepareInputs`, + `depthClip`, and `prepareReactivity`. +- Q5 transparency captures with manual/application reactive input, generated reactive + input, and T&C input reviewed separately. +- Reactivity, disocclusion, accumulation-age, locks, and motion-vector debug views to + verify channel meaning and alignment. +- Moving-geometry and camera-motion review for farthest-depth and motion-divergence + behavior, including false positives on stable opaque surfaces. + +### `source-spd-resolver-bundle-v1` + +**Changes** + +- Includes all of `source-structural-bundle-v1`. +- Adds a luma mip chain carrying spatial luma/depth information and frame exposure state. +- Adds a signed current/history luma-difference mip chain and three-mip shading-change + resolve. +- Adds persistent render-resolution four-frame luma history and instability. +- Replaces the local resolver with coordinated source-style current/history + reconstruction, dynamic rectification, accumulation weighting, ridge locks, and state + packing. +- Stores lock lifetime in history alpha. Production stores normalized sample age there, + so the two histories are intentionally incompatible and must reset when switching. + +**Likely problems addressed** + +- Coarse luma context should make shading-change detection less sensitive to local + high-frequency detail than the current 3×3 heuristic. +- Four-frame instability can preserve valid recurring detail that a one-frame comparison + might clip, while still rejecting unstable history. +- Coordinating accumulation, rectification, locks, and prepared state avoids combining + signals designed for different state models. + +**Why these changes are grouped** + +The mip signals, instability history, lock lifetime, accumulation value, and rectification +weights are mutually dependent. Testing them as independent toggles would produce invalid +state combinations. This bundle is the only candidate whose history alpha has source-style +lock semantics. + +**Expected tradeoff and fallback** + +This is the highest-cost and highest-risk candidate: it adds luma/shading dispatches, +history resources, and coarse reductions before a more complex resolver. The WebGPU-safe +mip implementation directly rereads the prepared source at higher levels instead of using +AMD's device-wide atomic SPD counter; that is portable but may duplicate substantial work. +Potential benefits are steadier lighting transitions, better detail retention, and more +coherent locks. Risks include false shading changes, excess history retention, flicker, +ghosting, incorrect reset state, and enough cost to outweigh fidelity gains. The +structural bundle and production graph remain fallbacks. + +**Evidence needed** + +- Incremental timing versus `source-structural-bundle-v1` for `lumaSpd`, `shadingSpd`, + `shadingResolve`, `lumaInstability`, and `accumulate`, plus total compute. +- Exposure-transition, lighting-change, static-convergence, motion, cut, reset, and resize + captures with history reset confirmed at every boundary. +- Shading-change, locks, accumulation-age, exposure, and final debug review that checks + signal meaning rather than only final-image similarity. +- Multi-frame metrics for convergence, temporal variance/flicker, disocclusion trails, + thin-feature retention, and ghost persistence. + +## Static audit corrections + +The post-authoring audit changed candidate-only code to correct likely defects before GPU +work: + +- restored accumulation-state growth in the structural path; +- seeded all four luma-history slots after reset or offscreen reprojection; +- computed reconstruction weights from unclamped tap positions at image borders; +- rounded odd-size SPD allocations so every written mip has sufficient extent; +- compile-time-specialized farthest-depth/current-luma preparation and motion divergence + so `source-filter-bundle-v1` does not silently execute structural work; +- kept benchmark-only constructor hooks out of the public constructor declaration; +- aligned T&C documentation and benchmark metadata with the implemented dispatch inputs. + +GPU-free checks previously passed after those fixes, but static checks cannot prove WGSL +device compilation, binding validity, image quality, timing, or cross-adapter behavior. + +## Later test matrix + +Test in project priority order: **performance > quality > realism**. + +### 1. Performance gate + +Measure both total and incremental cost: + +- production → `source-filter-bundle-v1`; +- `source-filter-bundle-v1` → `source-structural-bundle-v1`; +- `source-structural-bundle-v1` → `source-spd-resolver-bundle-v1`; +- production → each candidate as a total-cost check. + +Use ratios `1`, `1.5`, `2`, and `3`; fixed dimensions, timestep, seeded scene state, +camera path, reset state, and presentation domain; and fresh per-frame timestamp-query +samples. Record every pass median/p95, compute-sum median/p95, missing samples, adapter, +browser/backend, shader keys, and resource graph. + +Interpret repeated measurements using the established practical thresholds: + +- **below 3%:** tied/noise; +- **3–5%:** uncertain; rerun before a decision; +- **at least 5%:** actionable when repeatable. + +A visual regression rejects a candidate regardless of speed. A performance loss of at +least 5% requires a clear, repeatable quality benefit to remain under consideration. + +### 2. Quality gate + +Prioritize the existing deterministic scenarios: + +- **Q0 input/debug validation:** compile/binding sanity and every candidate debug view. +- **Q1 static convergence:** thin-feature retention, lock growth, convergence, flicker. +- **Q2 slow aliasing dolly / Q4 camera-motion hold:** reconstruction stability and trails. +- **Q3 object-motion disocclusion:** atomic depth, motion divergence, silhouette ghosts. +- **Q5 seeded transparency/reactivity:** generated/application reactive masks and softer + T&C behavior. +- **Q9 exposure transition:** conditioning/pre-exposure correction, pumping, and shading + response. +- **Q10 reset/cut/resize:** state packing, reset seeding, and resource recreation. + +Review final output plus motion vectors, disocclusion, accumulation age, locks, exposure, +shading change, and reactivity where each scenario defines them. Preserve blinded captures, +ROI max-absolute error and RMSE, frame-exact comparisons, and reviewer notes. Numerical +similarity is supporting evidence; temporal artifacts and visible regressions decide the +quality gate. + +Add focused odd-width/odd-height cases to validate candidate mip allocation; no existing +Q0–Q10 scenario supplies that coverage. + +Run a separate spatial EASU matrix because the temporal matrix does not dispatch EASU. +Compare edge sharpness, ringing, flat-field stability, small text/grid detail, and GPU cost +between exact production math and the approximate candidate. + +### 3. Realism/coverage gate + +Only after performance and deterministic quality survive, expand to Q6–Q8 screen-space +effects, representative HDR scenes, noisy GI/reflections, transparency-heavy content, +multiple adapters/browsers, and long camera paths. This gate checks whether a source-style +gain generalizes; it must not rescue a candidate that already failed deterministic +performance or visual review. + +## Existing harness entry points and missing work + +Verified existing package entry points: + +- `npm run bench` starts the interactive Vite bench. +- `npm run bench:run` and `npm run bench:capture` execute the immutable E00 harness. +- `npm run bench:compare:rcas` is the focused E01 RCAS comparison. + +The browser configuration recognizes `variant` and `comparison` selectors with these exact +candidate IDs, ratios `1`, `1.5`, `2`, and `3`, and scenarios `Q0` through `Q10`. +Candidate timing labels and resource identities are registered. + +There is **no focused candidate command or immutable candidate manifest yet**. +`bench:run`/`bench:capture` enforce E00's baseline roles for authoritative non-smoke runs, +and `bench:compare:rcas` covers RCAS only. Before collecting decision evidence, author +candidate manifests plus runner/report orchestration for cumulative and adjacent A/B +comparisons, including a separate spatial EASU comparison. Do not treat an interactive +selection or smoke run as adoption evidence. diff --git a/bench/PARITY-DECISIONS.md b/bench/PARITY-DECISIONS.md new file mode 100644 index 0000000..aef2ff2 --- /dev/null +++ b/bench/PARITY-DECISIONS.md @@ -0,0 +1,52 @@ +# FSR 3.1.5 Parity Decisions + +This is the concise decision record for parity experiments. Raw evidence remains +under `bench/results/raw/`; `PARITY-PROGRESS.md` retains detailed program state. + +| Experiment | Candidate | Measured result | Recommendation | User decision | Action | +| --- | --- | --- | --- | --- | --- | +| E01 | FSR 3.1.5 lower limiter | Linear/HDR rerun: total compute median −0.62% (tie); sparse differences remained difficult to see without a heatmap. | Adopt: parity improves without a demonstrated total-cost or quality regression. | Adopt source behavior — 2026-07-17. | Integrated and GPU-verified in the production RCAS shader. | +| E01 | Source denoise luma and center-inclusive range | Linear/HDR rerun found no stable total-compute cost. Corrects green-only luma and the center-excluded range. | Adopt the source math whenever RCAS denoise is enabled. | Adopt source behavior — 2026-07-17. | Integrated and GPU-verified in the production RCAS shader. | +| E01 | Enable temporal RCAS denoise by default | Linear/HDR Q0/Q1 captures changed contrast edges by RMSE 0.55–0.60/255 with maxima 17–33/255; total compute median +2.87% (tie). | Test representative noisy temporal inputs before changing the default; this remains quality policy rather than math-only parity. | Pending explicit default-policy decision. | Keep current opt-in default until the noisy-input review. | +| E03 | Remove internal ACES/sRGB presentation | All output paths now write caller-domain `rgba16float`; temporal, spatial, bilinear, and depth-debug paths compiled and rendered on WebGPU. | Adopt unconditionally: FSR must not own tone mapping or output encoding. | Must remove — 2026-07-17. | Integrated. Bench/examples apply renderer presentation after the upscaler. | +| E05–E07, E15 | `source-filter-bundle-v1` | GPU-validated + measured 2026-07-18 (after 2 compile fixes): **+35–36% total compute** vs production at ratio 2 (accumulate +47%; exposure/RCAS tied; noise floor ≤1.7%). Q0/Q1/Q3 captures artifact-free; steady-state diff vs production grows to RMSE ~10/255. | Do not adopt wholesale — the source reconstruction/history filtering costs ~⅓ more compute with no demonstrated visual win on the bench scenes. Cherry-pick pieces (exposure-domain correction, AMD depth-separation constant) individually. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | +| E04, E08–E10 | `source-structural-bundle-v1` | GPU-validated + measured 2026-07-18: **+6.2–6.9% over the filter bundle** (prepareInputs +30%, depthClip +22%, accumulate/RCAS tied; noise floor ≤2%). Output near-identical to filter bundle (its prepared accumulation/newLocks outputs are currently unconsumed). | Cheap increment, but mostly inert until a consumer exists; evaluate only together with the resolver or after wiring its prepared state into an accumulate path. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | +| E11–E14 | `source-spd-resolver-bundle-v1` | GPU-validated + measured 2026-07-18 (after 3 compile/validation fixes + a `/0.5`→`/20.0` velocity-normalization correction): **+75–78% total compute** vs production (accumulate +38%; RCAS **−47%**, see note). Q0/Q1/Q3 artifact-free, no disocclusion trails; steady-state diff vs production RMSE ~4.6/255 — closer than the filter bundle's ~10/255. | Too expensive to adopt as-is. Two leads worth extracting: (1) why its history makes RCAS 47% cheaper (production history may contain value ranges that are ALU-hostile); (2) its coarse-mip shading detector is the Phase-5 SPD design, now GPU-proven. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | + +## E01 evidence + +- Reproducible comparison: `npm run bench:compare:rcas` +- Verified linear/HDR report: `bench/results/raw/E01/rcas-comparison-linear-hdr/index.html` +- Reopen report: + + ```bash + npm run bench:compare:rcas -- --reuse bench/results/raw/E01/rcas-comparison-linear-hdr + ``` + +## Candidate-bundle evidence (2026-07-18) + +Timing (ratio 2, 1920×1080 display, Apple Metal-3, headless Chrome 150, ABBA blocks, +240 warmup + 300 samples per run, `--smoke`): + +| Comparison | compute-sum A → B | Δ per block | Verdict (≥5% actionable / <3% noise) | +| --- | --- | --- | --- | +| production → filter bundle | 0.627 → 0.852 ms | +35.2…+36.1% | Actionable regression | +| filter → structural bundle | 0.850 → 0.906 ms | +6.2…+6.9% | Actionable regression (small) | +| production → resolver bundle | 0.627 → 1.104 ms | +75.2…+77.6% | Actionable regression | + +Per-pass: filter's cost is entirely `accumulate` (+47%); structural's is `prepareInputs` +(+30%) + `depthClip` (+22%) + the added `prepareReactivity` dispatch; the resolver adds +its SPD/instability passes and a heavier accumulate (+38%) while making `rcas` 47% +cheaper (input-content-dependent ALU — unexplained, worth investigation). + +Quality (Q0/Q1/Q3 captures, 168 blinded pairs per comparison, `review.html` in each +capture directory under `bench/results/raw/CANDIDATES/`): no artifacts, ghost trails, or +convergence failures spotted in any bundle; differences are sub-4% RMSE distributed over +edges/grid detail. Blinded human review remains open before any adoption decision. + +Five defects were fixed before the bundles would compile/run at all (they had never +touched a GPU): WGSL has no `isInf()`; `external` is a reserved keyword; `r8unorm` is +not a core storage-texture format (3 shaders + 4 allocations → `r32float`); the resolver +accumulate declared an unused sampler binding (auto-layout drops it → bind-group failure); +and its history velocity falloff used `/0.5` where every sibling uses `/20.0`. +Reproduce: `node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant --comparison `. diff --git a/bench/PARITY-PROGRESS.md b/bench/PARITY-PROGRESS.md new file mode 100644 index 0000000..7f86701 --- /dev/null +++ b/bench/PARITY-PROGRESS.md @@ -0,0 +1,116 @@ +# FSR 3.1.5 Parity Progress + +## Program Purpose + +This ledger tracks controlled experiments that compare the local WebGPU upscaler with FidelityFX FSR 3.1.5. Source behavior is the design target; a local simplification is retained only when evidence shows that parity is unavailable, materially slower, or worse for this library on the web platform. + +- Pinned FidelityFX SDK source: `60f4ea81909200d8542eca14dccb2628b763a9a3` +- Initial local baseline: `5d6a65e5681e5e95590f3e9a11ce75e43354ca13` (`5d6a65e`) +- Baseline branch: `feat-match-fsr3` +- Baseline unit result: `npm test` passed `54/54` + +## Experiment State Machine + +Only the controller changes experiment state: + +`declared → implementing → static verification → GPU verification → task review → decision → documented` + +An experiment may move backward only to `implementing` for a consolidated fix round. A terminal blocker is recorded without pretending that later gates passed. Dependent experiments begin only from an adopted and documented integration state. + +## Strict Status Contract + +Every implementation or review handoff must use exactly one status: + +- `PASS`: the assigned scope is complete and every required gate available to that task passed. +- `FAIL`: the task ran, but an implementation, verification, or evidence gate failed. +- `BLOCKED`: the assigned scope cannot proceed because of an external, environment, capability, or dependency blocker. +- `SCOPE_BLOCKED`: completion requires a write, redesign, or investigation outside the immutable manifest. +- `USER_DECISION_REQUIRED`: evidence exposes a product, API, quality, performance, or scope trade-off that only the user may decide. + +A report must also include `changed_files`, `commands`, `artifacts`, `gates`, and `concerns`. `PASS` is invalid if required evidence or gate results are absent. Agents must not reinterpret a blocker as permission to broaden scope. + +## Blocker Taxonomy + +- `scope`: a required file or change is outside the exact allowlist; report `SCOPE_BLOCKED`. +- `dependency`: a prerequisite experiment is not adopted and documented; report `BLOCKED`. +- `tooling`: a command, browser, or local tool fails independently of the implementation; retry once, then report `BLOCKED`. +- `capability`: required WebGPU features, especially `timestamp-query`, are unavailable; report `BLOCKED` and preserve environment evidence. +- `validation`: WGSL compilation, WebGPU validation, uncaught runtime errors, device loss, or static verification fails; report `FAIL`. +- `evidence`: required captures, fresh timing samples, adapter metadata, or review records cannot be produced; report `BLOCKED`. +- `product-decision`: valid evidence leaves a public API, default, or quality/performance trade-off unresolved; report `USER_DECISION_REQUIRED`. + +The controller may assign one bounded read-only investigation for a blocker. That investigation cannot modify integration state, expand an allowlist, or redesign the experiment. + +## Fix-Round Limit + +Each experiment allows at most two consolidated fix rounds after its initial implementation. A round addresses one controller-approved batch of critical or important findings. After round two, the controller must redesign the manifest, retain the baseline, record a blocker, or escalate to the user. + +## Documentation Ownership + +- The controller owns this ledger, immutable manifests, state transitions, decisions, and cross-experiment dependencies. +- Concise measured results, recommendations, user votes, and resulting actions are tracked in `bench/PARITY-DECISIONS.md`. +- Implementers own only files explicitly listed in their manifest and may not edit manifests or this ledger. +- Reviewers and research agents are read-only unless a separate exact allowlist says otherwise. +- Implementation agents whose exact allowlists share any path are serialized. The controller must finish or stop the active writer before starting another overlapping writer. +- Parallel work is limited to independent, read-only research, review, or artifact analysis with no shared mutable state. +- The controller updates `src/shaders/README.md` only after an experiment decision and updates public documentation only for shipped behavior. +- Raw captures and timing artifacts follow `bench/results/README.md` once the harness creates it; manifests remain versioned records. +- No agent may commit, branch, create a worktree, or add files to an allowlist without explicit controller authorization. + +## Experiment Index + +| ID | Experiment | State | Fix rounds | Manifest | Decision | +| --- | --- | ---: | ---: | --- | --- | +| E00 | Harness Foundation | `adopted` | 2 of 2 — maximum reached; controller redesign 4 | `bench/results/experiments/e00-harness.json` | Directional baseline established; publication-grade acceptance deferred | +| E01 | RCAS numeric parity | `documented` | 0 of 2 | Directional first pass | Source limiter and denoise math adopted; denoise default remains separate | +| E03 | Linear/HDR output domain | `documented` | 0 of 2 | Directional integration | Internal ACES/sRGB removed by user decision | +| E04–E07, E15 | Source reconstruction/filter bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +35–36% compute vs production; visually clean; not adopted (see PARITY-DECISIONS.md) | +| E08–E10 | Structural inputs/reactivity bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +6.2–6.9% over filter bundle; largely inert outputs; not adopted | +| E11–E14 | SPD temporal resolver/state bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +75–78% compute vs production; visually clean; not adopted; RCAS −47% anomaly worth study | + +E00 covers only the deterministic Phase 1 benchmark foundation and baseline-versus-baseline acceptance machinery. It does not authorize parity shader algorithm changes. Fix rounds 1 and 2 were controller-authorized pre-harness contract corrections; no harness implementation had begun when they were issued. Controller redesign 1 froze the readiness reset contract against installed three `0.185.1`. Controller redesign 2 closed the post-implementation acceptance boundary: immutable A/B roles, exact reviewer-record coverage, noise-first retry classification, fixed timing-pass identities, unique timing sequences, and explicit page teardown. Controller redesign 3 corrected two acceptance-analysis defects exposed by authoritative execution: E00 aggregate timing acceptance uses the compute-sum noise floor while noisy passes are ineligible for individual claims, and E00 visual acceptance uses a representative Q0/Q1/Q10 matrix instead of pre-running the full domain-experiment Cartesian product. Every selected tuple still receives all 45 numerical reload comparisons; human review samples 120 blinded pairs and records all declared ROI bounds. Controller redesign 4 followed the fresh task review: run and review evidence is now bound to one manifest plus complete working-tree digest, review persistence includes the evidence and ordered-record identity, review-only validates the authoritative numerical capture before accepting grades, and user-owned CDP targets are closed and awaited. + +The earlier timing and capture artifacts passed their numerical gates but were produced under superseded manifest digests and cannot close E00. They remain diagnostic evidence only. Revision 4 static verification passed with lint, typecheck, 83 tests, build, runner syntax, and diff whitespace. Q2-Q9 reduced WebGPU smokes passed, as did the user-owned CDP target-cleanup smoke. + +Revision 4 authoritative timing is blocked at ratio 1 after its one permitted cold-browser retry: a concentrated GPU slowdown affected roughly 100 frames in one 600-frame block, producing compute-sum p95 noise near 50 percent. The bound authoritative capture completed all 264 tuples and 11,880 pairs but failed 61 pairs across five localized Q0 tuples; differences cluster on the moving torus-knot highlight and one derived accumulation-age region. Two focused five-reload reruns of the failed tuples passed, confirming intermittence but not authorizing replacement of the complete failed set. + +A clean-boot revision 4 timing rerun reproduced monotonic drift across the long ABBA sequence: endpoint A runs rose from roughly 0.78 ms to 0.96–1.08 ms while adjacent B runs stayed closer. This prevents publication-grade claims near the strict 1.5–2.5 percent noise limits, but does not prevent the harness from identifying substantial directional shader changes. Per user direction, E00 is adopted as a first-pass engineering tool: repeatable changes of at least 5 percent are actionable, changes below 3 percent are treated as tied/noise, and 3–5 percent remains uncertain. Visual regressions still reject candidates regardless of speed. Formal cross-platform and fine-margin acceptance is deferred until a candidate survives the shader-parity program. + +E01's first pass compiled the FSR 3.1.5 lower limiter and source denoise luma/range as an isolated RCAS shader. The user adopted both math changes on 2026-07-17. They are now in the production RCAS pipeline; the prior implementation remains benchmark-only. A fresh linear/HDR rerun kept total compute within the practical noise band and retained only sparse lower-limiter differences. Enabling source denoise by default also remained within noise but changed high-contrast detail across the frame: Q0/Q1 RMSE was about 0.55–0.60/255 with maxima of 17–33/255. Denoise therefore stays opt-in until a targeted noisy-input fixture shows that the broad sharpening reduction is beneficial. + +`npm run bench:compare:rcas` reproduces the focused comparison end to end, generates short directional timing summaries plus blinded visual reviews, serves the report locally, and opens it in the default browser. The current reference is `bench/results/raw/E01/rcas-comparison-linear-hdr/index.html`; reopen it without GPU work using `npm run bench:compare:rcas -- --reuse bench/results/raw/E01/rcas-comparison-linear-hdr`. + +E03 removed the internal Narkowicz ACES approximation and sRGB encoding from EASU, RCAS, and blit. Final and debug output now use `rgba16float`, and the public texture remains in the caller's linear/HDR domain. The benchmark and examples choose three's ACES filmic tone mapping plus sRGB only when presenting to screen. Static verification passed lint, typecheck, 86 tests, build, and diff whitespace. Real WebGPU verification rendered temporal, spatial, bilinear, and depth-debug paths without validation failure. + +## Authored source-style candidates + +Three cumulative internal candidates are now authored for later A/B work. This is +implementation state only: no benchmark, browser GPU validation, visual review, timing +claim, or adoption decision has been made. + +See `bench/PARITY-CANDIDATES.md` for the candidate hypotheses, cumulative dependencies, +fallbacks, and the performance-first test matrix required before any adoption decision. + +- `source-filter-bundle-v1` replaces current/history reconstruction, EASU implementation + math, and fused depth reconstruction with source-style radial approximate Lanczos2, + bicubic Lanczos history, approximate EASU helpers, atomic reconstructed-depth scatter, + and viewport/depth-scaled disocclusion. It also corrects history across changing + conditioning and host pre-exposure domains while retaining the local accumulation state. +- `source-structural-bundle-v1` cumulatively adds configurable source reactive generation, + farthest depth/current luma preparation, motion divergence, max-dilated application + reactivity with reset coupling, a distinct softer T&C channel, render-resolution + accumulation state, and atomic new-lock preparation. +- `source-spd-resolver-bundle-v1` cumulatively adds luma and signed-difference mip chains, + multi-mip shading change, persistent four-frame luma instability, and one coordinated + source-style accumulation/rectification/lock/state model. History alpha stores lock + lifetime in this candidate and is not compatible with the production sample-age alpha. + +The production graph remains the fallback and default. RCAS keeps the adopted source +limiter/math, linear/HDR output remains unchanged, and temporal RCAS denoise remains +opt-in. Candidate timing labels and shader/resource identities are registered, but should +not be interpreted as measured evidence until the later benchmark program runs. + +The bounded GPU-free verification pass completed with 156 tests, typecheck, lint, and the +library/declaration build passing. This advances the authored candidates only to static +verification; WebGPU compilation, validation, captures, timings, review, and adoption +remain pending. diff --git a/bench/THREE-TEMPORAL-COMPARISON.md b/bench/THREE-TEMPORAL-COMPARISON.md index cdd49b6..e06c9f1 100644 --- a/bench/THREE-TEMPORAL-COMPARISON.md +++ b/bench/THREE-TEMPORAL-COMPARISON.md @@ -442,21 +442,20 @@ Do not copy or design around TAAU's current lock behavior until its second-outpu #### Local -The default temporal output runs RCAS. Each RCAS tap is inverse-tonemapped, de-exposed, transformed through fixed ACES plus sRGB, and then sharpened in that display-referred space before the pass writes `rgba8unorm`. See `../src/shaders/rcas.ts:4-30`, `../src/shaders/rcas.ts:36-61`, and `../src/shaders/common.ts:100-120`. - -The public texture is therefore display-ready under the repository's current presentation contract. See `../src/Upscaler.ts:237-247`. +The default temporal output runs RCAS. Each tap is inverse-tonemapped, de-exposed, and +sharpened in the caller's linear/HDR domain before the pass writes `rgba16float`. +Presentation is deliberately outside the upscaler. **Pros** -- Simple direct presentation. -- Bench paths can share one known transform. +- Composable linear/HDR output. +- Caller-controlled tone mapping, output color space, and later post-processing. - Built-in sharpening and optional RCAS denoise. **Cons** -- Not a general linear/HDR graph output. -- Prevents later HDR post-processing, alternate tone mapping, wide-gamut output, or caller-controlled exposure after the resolve. -- Makes direct resolver comparisons unfair unless three's output receives the same transform and sharpening policy. +- Direct presentation requires the integration to configure its renderer/output transform. +- Fair resolver comparisons must apply the same presentation transform after each result. #### three diff --git a/bench/results/.gitignore b/bench/results/.gitignore new file mode 100644 index 0000000..8524d3f --- /dev/null +++ b/bench/results/.gitignore @@ -0,0 +1,4 @@ +raw/ +*.png +*.csv +*.heatmap.png diff --git a/bench/results/README.md b/bench/results/README.md new file mode 100644 index 0000000..22896a7 --- /dev/null +++ b/bench/results/README.md @@ -0,0 +1,66 @@ +# Benchmark Results + +E00 artifacts are produced by `npm run bench:run` and `npm run bench:capture`. +The immutable experiment contract remains in `experiments/e00-harness.json`. + +## Layout + +- `raw/E00//run.json` records arguments, the manifest SHA-256, the + baseline Git SHA, and a digest of every tracked or untracked non-ignored file + in the working tree. +- Performance runs emit one JSON and CSV file per ratio, ABBA repetition, and + block position. JSON retains every fresh frame-tagged sample, missing counts, + per-pass median/p95, and compute-sum median/p95. +- Capture runs emit canvas-only lossless PNGs. Filenames identify blinded + label, variant, reload, scenario, subrun, ratio, frame, and debug view. +- `capture-analysis.json` records all numerical reload pairs. Passing pairs stay + in the aggregate file; failed pairs additionally emit standalone metrics and + difference heatmaps. +- `review.html` presents the sampled A/B pairs without variant names, outlines + every declared ROI, stores progress locally, and exports + `rubric-reviewed.json`. Review state is isolated by manifest, working-tree, + and ordered-record digests. Review-only validation rejects stale contracts, + changed source trees, incomplete capture protocols, and failed numerical + analyses before accepting grades. Validate an export without recapturing: + + ```bash + node scripts/run-benchmark.mjs \ + --review-only \ + --output bench/results/raw/E00/ \ + --rubric /path/to/rubric-reviewed.json + ``` + +- Each run retains the required browser log channels. Any validation error, + uncaught exception, or reported device loss invalidates the run. + +Raw output is ignored by Git. Promote only controller-reviewed summaries or +small diagnostic artifacts under a separately authorized manifest. + +The authored source-style bundles do not have result manifests yet. Their hypotheses, +cumulative ordering, and required later evidence are documented in +`bench/PARITY-CANDIDATES.md`; do not store interactive or smoke output as adoption +evidence. + +Q6-Q8 construct reduced-resolution three.js effect graphs for GTAO, SSR, SSGI, +the spatial denoiser, and the recurrent denoiser. Their readiness, source-owned +sampling state, node-frame maps, velocity history, and effect history are reset +before recorded frame zero as specified by the E00 manifest. + +## Focused RCAS comparison + +Run the practical E01 comparison with: + +```bash +npm run bench:compare:rcas +``` + +It captures legacy RCAS versus the FSR 3.1.5 lower limiter, then the lower +limiter versus the temporal denoise default. It also records short ratio-2 +timing blocks, writes a summary report, starts a local review server, and opens +the report in the default browser. Press Ctrl+C when review is complete. + +To reopen an existing report without rerunning the GPU work: + +```bash +npm run bench:compare:rcas -- --reuse bench/results/raw/E01/ +``` diff --git a/bench/results/experiments/README.md b/bench/results/experiments/README.md new file mode 100644 index 0000000..5f7d59f --- /dev/null +++ b/bench/results/experiments/README.md @@ -0,0 +1,120 @@ +# Parity Experiment Manifests + +## Purpose + +Each JSON file is the immutable execution contract for one parity experiment. It fixes the hypothesis, comparison, write scope, evidence, validation gates, and rollback conditions before implementation starts. + +Once a manifest reaches `implementing`, agents must not edit it. If the contract is wrong or incomplete, the controller closes that run and creates a new manifest revision; an implementer cannot reinterpret or extend the existing contract. + +## Immutable Manifest Schema + +Every manifest is valid JSON and contains all of these fields: + +- `schema_version`: manifest schema identifier, currently `1`. +- `experiment`: object with immutable `id`, `name`, `phase`, `task`, and `status`. +- `purpose`: bounded reason for running the experiment. +- `hypothesis`: falsifiable expected outcome. +- `comparison`: object with `class`, `baseline_variant`, `candidate_variant`, and `single_variable`. +- `source`: pinned reference name, version, commit, and relevant reference behavior. +- `baseline`: full local SHA, short SHA, branch, and recorded verification evidence. +- `dependencies`: experiment IDs that must already be adopted and documented. +- `target_artifact`: exact visual, correctness, or performance behavior the candidate is intended to improve or preserve. +- `known_regression_risks`: explicit quality, correctness, performance, platform, and maintenance risks evaluated by the experiment. +- `per_pass_and_compute_timing_budget`: numerical median/p95 limits for every affected pass and the total compute sum, including the declared noise-floor rule. +- `required_platform_divergence_evidence`: required platform reason, affected scenarios, visual impact, median/p95 impact, and maintenance cost. +- `scope`: exact repository-relative `allowed_write_paths` and explicit `forbidden_changes`. +- `commands`: required static and real-device commands, in execution order where order matters. +- `scenarios`: exact scenario IDs, ratios, dimensions, timestep, MSAA state, and required debug views. +- `capture_protocol`: exact absolute and event-relative frame captures, numerical image-equivalence rules, artifact rubric, and pass/fail aggregation. +- `validation_gates`: static, runtime, visual, timing, and review requirements. +- `timing_protocol`: feature requirements, warm-up, sample count, block order/repetitions, isolation, statistics, metadata, and invalidation rules. +- `expected_behavior`: output domain, presentation transform, jitter owner, velocity convention, reset behavior, resource graph, and binding expectations. +- `required_evidence`: complete evidence record required before a decision. +- `rollback_conditions`: conditions that immediately invalidate or revert the candidate. +- `result_contract`: allowed decision values and the evidence required to set one. +- `report_contract`: required agent status and handoff fields. + +Arrays of file paths are literal allowlists, not examples or directory prefixes. A listed directory never authorizes unlisted descendants. Repository-relative paths are resolved from `/Users/dex/Developer/fsr3`. + +Deterministic effect scenarios must define their renderer/device/pipeline/resource readiness barrier before recorded frame zero. Unrecorded readiness work may compile and allocate, but its temporal history must be cleared or source-defined reseeding must occur on recorded frame zero. A source-owned sampling sequence must cite the exact installed source/version and state its frame-index, rotation, offset, and noise formulas; a pinned private bridge requires a runtime shape guard and cannot be presented as public API. + +Capture thresholds are immutable manifest inputs. Measured repeated-baseline behavior is evidence about the environment and may block a run, but it cannot widen max-absolute, RMSE, dimension, alpha, rubric, or aggregation limits. + +## Result Decisions + +The controller records exactly one decision after the evidence gates: + +- `adopt`: candidate meets its quality, correctness, performance, and review gates. +- `iterate`: evidence supports another explicitly remanifested candidate; the current run is not adopted. +- `retain-local`: the local baseline remains preferable on measured quality, performance, or maintainability. +- `platform-divergence`: source parity is unavailable or materially worse because of a documented web/WebGPU constraint, and the measured local alternative is retained. +- `blocked`: the experiment cannot reach a valid decision because required capability, dependency, environment, or evidence is unavailable. + +An implementation status such as `PASS` is not an adoption decision. Only the controller may set a result decision. + +## Required Evidence Fields + +Every result record must include: + +- manifest ID, contract revision, immutable manifest digest, and working-tree digest; +- baseline and candidate IDs plus source and local SHAs; +- exact changed files and final diff summary; +- commands, exit codes, test counts, and timestamps; +- browser, browser version, operating system, adapter, backend, WebGPU features, three version, dimensions, DPR, ratio, settings, and shader/pipeline keys; +- validation logs covering WGSL compilation, WebGPU errors, uncaught exceptions, and device loss; +- scenario, frame, debug view, blinded A/B label, ROI, and artifact-rubric records for every capture; +- raw fresh GPU samples with frame tags, missing-sample count, per-pass and compute-sum median/p95, ABBA block results, and measured noise floor; +- reset, jitter-owner, active-resolver, output-domain, resource-graph, and binding checks; +- reviewer findings, fix-round count, unresolved concerns, decision, and decision rationale; +- platform reason and unsupported or degraded capability; +- affected scenario IDs, ratios, frames, debug views, and platforms; +- visual impact with blinded captures, ROIs, rubric grades, max-absolute error, and RMSE; +- per-pass and compute-sum median/p95 impact with raw samples and the measured noise floor; +- maintenance cost covering extra code paths, feature detection, tests, validation matrix, and likely upkeep for `platform-divergence`. + +Evidence must describe failures and inconclusive measurements as recorded. Missing evidence cannot be inferred from screenshots, FPS, a successful build, or an agent assertion. + +## Controller Responsibilities + +The controller: + +1. Authors and freezes the manifest before assigning implementation. +2. Confirms prerequisite decisions and the clean baseline. +3. Gives each agent the manifest and its exact file allowlist. +4. Rejects any unlisted write and classifies it as `SCOPE_BLOCKED`. +5. Serializes all implementation agents whose allowlists overlap by one or more exact paths. +6. Verifies changed paths and static commands before GPU work. +7. Runs or assigns real-device validation and evidence collection. +8. Assigns a fresh read-only task review. +9. Consolidates critical and important findings into no more than two fix rounds. +10. Makes the result decision and updates `bench/PARITY-PROGRESS.md`. +11. Starts dependent work only from an adopted, documented integration state. + +The controller may narrow a task without changing its manifest. Any expansion requires a new manifest revision before work resumes. + +Parallel work is permitted only when every participant is read-only, independent, and has no shared mutable artifact or integration state. Implementation agents with overlapping allowlists are never concurrent, even if they intend to edit different paths from those lists. + +## Agent Responsibilities + +Implementers: + +- write only exact paths in `scope.allowed_write_paths`; +- make only the change described by the hypothesis and `single_variable`; +- stop with `SCOPE_BLOCKED` before touching any additional path; +- do not edit manifests, the progress ledger, unrelated algorithms, public APIs, dependencies, CI, or documentation unless each exact path is allowed; +- run required commands that are available in their assigned environment; +- return only the required report fields with complete artifacts, gate results, evidence, and concerns. + +Reviewers and research agents remain read-only. Validation agents may create artifacts only when their exact artifact paths are listed in a separate manifest. No agent may expand scope, silently fix adjacent issues, commit, create a branch/worktree, or run baseline and candidate simultaneously during authoritative timing. + +## Handoff Status + +Every agent handoff uses exactly one of: + +`PASS | FAIL | BLOCKED | SCOPE_BLOCKED | USER_DECISION_REQUIRED` + +It also includes: + +`changed_files | commands | artifacts | gates | concerns` + +`PASS` means the assigned task and its available gates passed; it does not mean the candidate is adopted. diff --git a/bench/results/experiments/e00-harness.json b/bench/results/experiments/e00-harness.json new file mode 100644 index 0000000..0ce143d --- /dev/null +++ b/bench/results/experiments/e00-harness.json @@ -0,0 +1,1764 @@ +{ + "schema_version": 1, + "experiment": { + "id": "E00", + "name": "Harness Foundation", + "phase": 1, + "task": "Tasks 1.1 through 1.6", + "status": "implementing", + "fix_round": 2, + "fix_round_limit_reached": true, + "contract_revision": 4, + "contract_correction": "Controller redesign 4 binds every run and review to one manifest plus working-tree digest, isolates review persistence, closes user-owned CDP targets, and finalizes E00 timing acceptance from measured baseline variance." + }, + "purpose": "Build and accept the deterministic, single-variant benchmark foundation required before any FSR parity algorithm experiment begins.", + "hypothesis": "The local baseline can run through a deterministic one-variant harness with frame-exact captures, fresh timestamp-query samples, and reproducible reset behavior while preserving its shader algorithms and output. Baseline-versus-baseline ABBA runs will establish a usable machine and browser timing noise floor.", + "comparison": { + "class": "infrastructure-equivalence", + "baseline_variant": "local-baseline-5d6a65e", + "candidate_variant": "local-baseline-through-e00-harness", + "single_variable": "Execution, capture, validation, and timing infrastructure; resolver shader algorithms, pass math, resource formats, and public behavior remain unchanged." + }, + "source": { + "name": "FidelityFX SDK FSR Upscaler", + "version": "3.1.5", + "commit": "60f4ea81909200d8542eca14dccb2628b763a9a3", + "reference_behavior": "Source behavior is pinned for later parity experiments. E00 does not port or authorize any source shader algorithm." + }, + "baseline": { + "sha": "5d6a65e5681e5e95590f3e9a11ce75e43354ca13", + "short_sha": "5d6a65e", + "branch": "feat-match-fsr3", + "verification": { + "command": "npm test", + "result": "passed", + "tests_passed": 54, + "tests_failed": 0 + } + }, + "dependencies": [], + "target_artifact": { + "name": "Deterministic baseline-equivalence evidence", + "description": "Frame-exact baseline captures, complete validation logs, and fresh frame-tagged GPU timing distributions produced by one active resolver without changing baseline image behavior.", + "success_signal": "Five-reload capture sets pass the numerical equivalence protocol, every deterministic scenario and debug gate passes, and four ABBA repetitions establish an acceptable timing noise floor at every required ratio." + }, + "known_regression_risks": [ + "Harness orchestration changes frame order, projection jitter ownership, velocity generation, or history reset timing.", + "Wall-clock, OrbitControls, resize handlers, random values, asynchronous readback order, or upstream effects make supposedly identical runs non-deterministic.", + "Capture compares different dimensions, transfer domains, alpha handling, presentation transforms, frames, or debug views.", + "Timing collection returns stale, missing, duplicated, or untagged samples or changes the measured GPU graph.", + "Variant registration instantiates two resolvers, compiles inactive candidates into the measured graph, or leaves stale timing labels.", + "Automated reset fails to clear temporal, exposure-adaptation, scenario-event, PRNG, jitter, and pending-readback state.", + "Q5 through Q8 fixture integration changes production shader algorithms instead of supplying deterministic upstream inputs.", + "A browser or adapter capability difference is mislabeled as algorithm evidence without the required platform-divergence record." + ], + "per_pass_and_compute_timing_budget": { + "scope": "Every existing GPU pass label and the compute sum at ratios 1, 1.5, 2, and 3.", + "median_relative_delta_limit": "max(3%, 2 * N_median)", + "p95_relative_delta_limit": "max(5%, 2 * N_p95)", + "per_pass_rule": "For each pass label, compare the mean of the two A run statistics with the mean of the two B run statistics inside each ABBA repetition. A pass supports an individual performance claim only when its own N_median and N_p95 meet the declared noise limits; otherwise retain its samples in compute sum, mark the pass ineligible, and require a later pass-specific microbenchmark before making a claim.", + "compute_sum_rule": "Apply the same median and p95 limits independently to the sum of all resolver compute-pass samples in at least three of four repetitions.", + "new_gpu_work": "No additional production GPU pass or dispatch is permitted in E00. Instrumentation query resolve and readback work is excluded from compute sum but reported separately.", + "noise_floor_acceptance": { + "N_median_max": 0.015, + "N_p95_max": 0.025, + "unit": "fractional relative delta", + "failure": "E00 is blocked if any required ratio's compute-sum noise floor exceeds either maximum after one clean rerun. A per-pass failure does not block aggregate E00 acceptance, but it forbids an individual claim for that pass until a later microbenchmark establishes a usable floor." + } + }, + "required_platform_divergence_evidence": { + "platform_reason": "Identify the exact browser, WebGPU, backend, adapter, driver, feature, limit, validation rule, or timestamp-query behavior preventing parity and include reproducible logs.", + "affected_scenarios": "List every affected platform, scenario ID, ratio, frame, debug view, and ROI; state which unaffected controls still pass.", + "visual_impact": "Provide blinded baseline/candidate captures and numerical max-absolute, RMSE, alpha, dimension, and artifact-rubric results for each affected ROI.", + "median_p95_impact": "Provide raw fresh samples, missing counts, per-pass and compute-sum median/p95 deltas, ABBA repetition results, and the independently derived noise floor.", + "maintenance_cost": "Record additional code paths, capability checks, tests, artifact fixtures, cross-platform validation obligations, production-bundle cost, and expected future upkeep." + }, + "scope": { + "allowed_write_paths": [ + "bench/src/types/benchmark.d.ts", + "bench/src/benchmark/config.ts", + "bench/src/benchmark/variants.ts", + "bench/src/benchmark/BenchmarkResolver.ts", + "bench/src/BenchPipeline.ts", + "src/internal/ComputePass.ts", + "src/shaders/shaders.test.ts", + "bench/src/benchmark/clock.ts", + "bench/src/benchmark/scenarios.ts", + "bench/src/BenchScene.ts", + "bench/src/main.ts", + "src/internal/GpuTimer.ts", + "bench/src/benchmark/collector.ts", + "bench/src/benchmark/api.ts", + "bench/src/benchmark/environment.ts", + "scripts/benchmark-contract.mjs", + "scripts/benchmark-contract.test.mjs", + "scripts/run-benchmark.mjs", + "bench/results/README.md", + "bench/results/.gitignore", + "package.json" + ], + "forbidden_changes": [ + "Any write to a path not exactly listed in allowed_write_paths.", + "Any parity shader algorithm, filter, sampling kernel, accumulation, rectification, lock, exposure, reactive, depth, RCAS, EASU, color-domain, or presentation-order change.", + "Any public API or public default change.", + "Any production resource format, binding layout, bind-group order, pass graph, output-domain, or dispatch behavior change except passing unchanged baseline behavior through the benchmark resolver adapter.", + "Any dependency addition or package-manager lockfile change.", + "Any GPU-dependent CI test or CI workflow change.", + "Any replacement of raw WGSL or the existing three WebGPU bridge with TSL.", + "Any edit to the parity plan, this manifest, or bench/PARITY-PROGRESS.md by an implementer.", + "Any commit, branch, worktree, stash, or destructive Git operation." + ], + "implementation_serialization": { + "rule": "All implementation agents whose immutable allowed_write_paths overlap by one or more exact paths execute serially.", + "handoff": "The controller verifies the prior writer is complete or stopped and records its changed paths before assigning the next overlapping writer.", + "parallel_exception": "Parallel participants must be independent and read-only, with no shared mutable artifact, integration state, or write permission." + } + }, + "commands": { + "static": [ + "npm run lint", + "npm run typecheck", + "npm test", + "npm run build" + ], + "real_device": [ + "npm run dev -- --host 127.0.0.1", + "npm run bench:capture -- --experiment E00 --variant baseline --comparison baseline --ratios 1,1.5,2,3", + "npm run bench:run -- --experiment E00 --variant baseline --comparison baseline --ratios 1,1.5,2,3 --blocks 4 --warmup 240 --samples 600" + ], + "required_log_channels": [ + "Log.entryAdded", + "Runtime.consoleAPICalled", + "Runtime.exceptionThrown" + ] + }, + "scenarios": { + "output": { + "width": 1920, + "height": 1080, + "device_pixel_ratio": 1, + "timestep_seconds": 0.016666666666666666, + "msaa": false + }, + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "formula_conventions": { + "frame": "n is the zero-based integer frame index.", + "time": "t = n / 60 seconds; no wall-clock value is read.", + "look_at": "Camera orientation is the right-handed three.js lookAt quaternion for the stated position, target, and up vector [0,1,0], recomputed from absolute values each frame.", + "projection": "Perspective fields are vertical field of view in degrees, aspect width/height, near, and far. Projection jitter is applied later by the sole active upscaler and is not part of these unjittered camera formulas.", + "baseline_scene_formula": { + "knots": "For knot i in {0,1,2}: position=(-6+6*i,2.2,-4), rotationEulerXYZ=(0.35*t+i,0.5*t,0).", + "spheres": "For sphere i in {0,1,2,3}, a=0.9*t+i*pi/2 and position=(5.5*cos(a),1.1+0.4*sin(2*t+i),5.5*sin(a)); rotation is fixed identity.", + "floor_fence_bulb": "Floor, 60 pickets, and bulb retain the fixed transforms created by BenchScene: floor rotationX=-pi/2; picket i position=(-12+0.4*i,1.2,3.5); bulb position=(0,5.5,-2).", + "lighting": "Directional light color 0xfff2df, intensity 3.2, position (8,14,6); hemisphere sky 0x9fb4d4, ground 0x2a2620, intensity 0.9.", + "exposure": "settings.exposure=1.0 and autoExposure=true unless the scenario states otherwise." + }, + "baseline_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "room_scene_formula": { + "camera_target": [ + 0, + 3, + -5 + ], + "geometry": "Fixed example-06/09/10 room: floor 60x60 at rotationX=-pi/2; walls at (-8,5,-4), (8,5,-4), and (0,5,-12); box i in {0..4} at (-5+2.5*i,1+0.25*i,-6+3*(i mod 2)); sphere at (2,1.6,-2).", + "lighting": "Studio lighting from examples/shared/props plus ambient color 0x404860 at intensity 0.4; all transforms and intensities remain fixed.", + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "installed_three_effect_sequences": { + "version": "0.185.1", + "frame_mapping": "After the readiness barrier, shape-check renderer._nodes.nodeFrame, then set frameId=0, renderId=0, time=0, deltaTime=0, lastTime=performance.now(), updateMap=new WeakMap(), updateBeforeMap=new WeakMap(), and updateAfterMap=new WeakMap(). Replacing all three maps is required because NodeFrame stores prior frame/render IDs per node; reusing frame IDs without fresh maps can suppress updates. Permit exactly one renderer animation-loop tick per recorded scenario frame and no other NodeFrame.update() call; Animation.update calls NodeFrame.update() before the user callback, so the exact effect frame ID for recorded scenario frame n is f(n)=n+1. The harness must verify these mutable fields against node_modules/three/src/renderers/common/Renderer.js, node_modules/three/src/renderers/common/Animation.js, and node_modules/three/src/nodes/core/NodeFrame.js before using this pinned private bridge.", + "readiness_barrier": [ + "Await renderer.init(), verify backend.device is a live GPUDevice, and record adapter/features before graph construction.", + "Construct the exact selected subrun graph and deterministic DenoiseNode noise texture, then execute unrecorded readiness renders with recording disabled until every expected input/effect/upscaler texture has a backend GPUTexture, NodeManager._buildQueue.length is 0, NodeManager._buildInProgress is false, and all selected shader pipelines have completed compilation.", + "Await device.queue.onSubmittedWorkDone() and require zero Log.entryAdded WGSL/WebGPU errors, Runtime.exceptionThrown events, uncaught console errors, and device-loss events.", + "Retain the compiled graph and allocated resources. Reset upscaler history and all scenario counters. Set every selected SSRNode._noiseIndex.value to 0 after shape-checking that private uniform. Q8 additionally calls setSize(1,1) on TemporalReprojectNode and RecurrentDenoiseNode so recorded n=0 synchronously takes each source-defined needsRestart seed/clear path and cannot read readiness history.", + "Reset NodeFrame counters and replace all three NodeFrame update maps with the pinned bridge described by frame_mapping. No selected effect or upscaler render occurs between that reset and recorded n=0. The readiness frames are never assigned scenario frame numbers and none of their history survives into recorded output.", + "Recorded n=0 is the first post-reset effect/upscaler execution and is captured when required; it is never replaced by an unrecorded accumulation frame." + ], + "gtao": { + "source": "node_modules/three/examples/jsm/tsl/display/GTAONode.js at installed three 0.185.1", + "sequence": "Q6 uses useTemporalFiltering=false, therefore temporalDirection(n)=0 exactly. If enabled in a future manifest, temporalDirection(n)=[60,300,180,240,120,0][f(n) mod 6]/360.", + "noise": "The static 5x5 repeat-wrapped magic-square texture is generated with the Siamese algorithm in GTAONode.generateMagicSquare(5): start i=2,j=4; place integers 1..25 with the source wrap/collision rules; texel k with magic value m uses angle=2*pi*m/25 and RGBA8=(uint8((0.5+0.5*cos(angle))*255),uint8((0.5+0.5*sin(angle))*255),127,255). No PRNG or per-frame noise index is used." + }, + "ssgi": { + "source": "node_modules/three/examples/jsm/tsl/display/SSGINode.js at installed three 0.185.1; rand is node_modules/three/src/nodes/math/MathNode.js", + "frame_sequence": "With useTemporalFiltering=true, temporalDirection(n)=[60,300,180,240,120,0][f(n) mod 6]/360 and temporalOffset(n)=[0,0.5,0.25,0.75][f(n) mod 4].", + "pixel_noise": "For integer screen pixel p=(px,py), spatialOffset(p)=0.25*((py-px) bitAnd 3), interleavedGradientNoise(p)=fract(52.9829189*fract(0.06711056*px+0.00583715*py)), and noiseJitterIdx(n)=0.02*temporalDirection(n). Define rand(q)=fract(sin(mod(12.9898*q.x+78.233*q.y,pi))*43758.5453). initialRayStep(n,p,uv)=fract(spatialOffset(p)+temporalOffset(n))+rand(2*(uv+(noiseJitterIdx(n),noiseJitterIdx(n)))-1).", + "slice_rotation": "For slice i in [0,sliceCount), rotationAngle(n,p,i)=(i+interleavedGradientNoise(p)+temporalDirection(n))*pi/sliceCount. Q6/Q7 use the node defaults unless the scenario states counts; Q8 fixes sliceCount=2 and stepCount=8." + }, + "ssr": { + "source": "node_modules/three/examples/jsm/tsl/display/SSRNode.js and node_modules/three/examples/jsm/tsl/utils/RNoise.js at installed three 0.185.1", + "sequence": "Q6-Q8 construct SSR with stochastic=false. Readiness advances SSRNode._noiseIndex independently of NodeFrame, so the barrier explicitly shape-checks and resets that private uniform to 0. Recorded updateBefore then computes noiseIndex(n)=(n+1) mod 2147483647. The non-stochastic shader does not sample bindAnalyticNoise, so no frame-varying SSR noise affects output. Mirror ray direction, march, and box-blur taps are deterministic functions of current G-buffer values." + }, + "denoise_node": { + "source": "node_modules/three/examples/jsm/tsl/display/DenoiseNode.js and node_modules/three/examples/jsm/math/SimplexNoise.js at installed three 0.185.1", + "default_limitation": "DenoiseNode.generateDefaultNoise() constructs SimplexNoise() with Math.random, so the installed default texture is not reload-deterministic and may not be used for E00 evidence.", + "deterministic_override": "Before compilation, assign the existing DenoiseNode.noiseNode property to a 64x64 RepeatWrapping RGBA8 texture generated by SimplexNoise({random}) with xorshift32 seed 0x0d3e015e. xorshift32 applies x^=x<<13, x^=x>>>17, x^=x<<5 with uint32 masking and returns x/4294967296. SimplexNoise sets p[j]=floor(random()*256) for j=0..255 and perm[j]=p[j bitAnd 255] for j=0..511. For i,j in 0..63, RGBA channels are uint8((noise(i,j),noise(i+64,j),noise(i,j+64),noise(i+64,j+64))*0.5*255+0.5*255) exactly as the source's per-channel (noise*0.5+0.5)*255 conversion.", + "rotation": "DenoiseNode.index remains its source default 0 for Q6-Q8. The installed expression indexes noiseTexel with index.mod(4)*2*pi; at index 0 this selects red without scaling its value. At pixel uv, sample the deterministic repeat-wrapped texture at noiseUv=(uv.x,1-uv.y)*(resolution/64), let r be sampled red, and use rotation vector (sin(r),cos(r)); the 16 source sample vectors use angle_i=2*pi*2*i/16 and radius_i=i/15." + }, + "recurrent_denoise": { + "source": "node_modules/three/examples/jsm/tsl/display/RecurrentDenoiseNode.js and node_modules/three/examples/jsm/tsl/utils/RNoise.js at installed three 0.185.1", + "frame_sequence": "RecurrentDenoiseNode.updateBefore assigns noiseIndex(n)=f(n)=n+1. Let k=n+1+83, tileSize=32, screenPixel=floor(uv*resolution), offset=floor(fract((0.7548776662*k,0.5698402910*k))*32), and coords=(screenPixel+offset) mod 32.", + "analytic_noise": "Let P=1.32471795724474602596 and z=coords.x/P+coords.y/(P^2)+83. The four noise channels are fract(z*P*(1/P)), fract(z*(2*P)*(1/P^2)), fract(z*(3*P)*0.4198754210), and fract(z*(4*P)*(1/P^3)). Rotation angle=2*pi*noise.r and matrix=((cos,-sin),(sin,cos)).", + "kernel": "For tap i in 0..7, theta=(i+0.5)*2.399827721492203 and radius=sqrt((i+0.5)/8); baseOffset=(cos(theta),sin(theta))*radius before the source's deterministic adaptive skew and edge-stopping terms." + }, + "temporal_reproject": { + "source": "node_modules/three/examples/jsm/tsl/display/TemporalReprojectNode.js at installed three 0.185.1", + "sequence": "TemporalReprojectNode has no random sampling index. Its recorded n=0 needsRestart path seeds internal history from current beauty, then subsequent frames are deterministic functions of current inputs, previous history/depth/normal, velocity, and camera matrices." + } + }, + "roi_coordinates": "ROIs are normalized [x,y,width,height] in top-left canvas coordinates and are resolved to integer pixels by floor(x*width), floor(y*height), ceil(widthFraction*width), ceil(heightFraction*height)." + }, + "required": [ + { + "id": "Q0", + "name": "input-debug-validation", + "purpose": "Validate color, depth, motion, disocclusion, accumulation age, locks, exposure, shading change, and reactivity inputs and debug outputs.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 143, + "duration_seconds": 2.4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "jitter-free velocity", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "119" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "floor_grid": [ + 0.05, + 0.55, + 0.9, + 0.45 + ], + "fence_and_spheres": [ + 0.05, + 0.28, + 0.9, + 0.42 + ] + } + } + }, + { + "id": "Q1", + "name": "static-convergence", + "purpose": "Establish deterministic accumulation and settled baseline output.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "zero object and camera velocity except jitter compensation", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "119", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "thin_features": [ + 0.08, + 0.3, + 0.84, + 0.55 + ] + } + } + }, + { + "id": "Q2", + "name": "slow-aliasing-dolly", + "purpose": "Replay an absolute slow dolly across aliasing-torture geometry.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "u=n/239; position=(9-3*u,6,12-4*u), target=(0,1.6,0), up=(0,1,0).", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera velocity", + "auto exposure", + "locks" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "floor_grid": [ + 0, + 0.48, + 1, + 0.52 + ], + "fence": [ + 0.05, + 0.35, + 0.9, + 0.32 + ] + } + } + }, + { + "id": "Q3", + "name": "object-motion-disocclusion", + "purpose": "Replay independent object motion and silhouette disocclusion.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly; sphere and knot transforms are assigned from absolute t each frame.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut, resize, or mid-run reset.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "independent object velocity", + "auto exposure", + "locks" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "moving_spheres": [ + 0.12, + 0.2, + 0.76, + 0.58 + ], + "fence_silhouette": [ + 0.05, + 0.35, + 0.9, + 0.3 + ] + } + } + }, + { + "id": "Q4", + "name": "camera-motion-hold", + "purpose": "Replay slow yaw, fast translation, rotation, and a final hold.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 479, + "duration_seconds": 8 + }, + "camera_formula": "Let T=(0,1.6,0), R=15, h=4.4, theta0=atan2(12,9). For 0<=n<=119, theta=theta0+0.25*n/119 and position=T+(R*cos(theta),h,R*sin(theta)). Let C119 be that n=119 position. For 120<=n<=239, u=(n-120)/119 and position=C119+u*(-8,0,-6). Let C239 be that n=239 position. For 240<=n<=359, theta=atan2(C239.z-T.z,C239.x-T.x)+0.9*(n-240)/119, radius=sqrt((C239.x-T.x)^2+(C239.z-T.z)^2), and position=T+(radius*cos(theta),C239.y-T.y,radius*sin(theta)). For 360<=n<=479 hold the n=359 position. Target=T and up=(0,1,0) for all n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Motion segment boundaries are n=120, n=240, and n=360; no cut, resize, or reset after the initial reset before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera velocity", + "auto exposure", + "locks", + "shading-change detector" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "2*P-1", + "118", + "119", + "120", + "121", + "238", + "239", + "240", + "241", + "358", + "359", + "360", + "361", + "383", + "479" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "thin_geometry": [ + 0.05, + 0.25, + 0.9, + 0.55 + ] + } + } + }, + { + "id": "Q5", + "name": "seeded-transparency-reactivity", + "purpose": "Replay transparent surfaces and seeded particles with aligned reactive inputs.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "Freeze baseline objects at t=0. For particle i in {0..127}, consume four initialization values ux,uy,uz,up from the seeded PRNG and assign phase=2*pi*up, base=(-5+10*ux,0.7+4*uy,-5+10*uz); at frame n assign position=(base.x+0.35*sin(0.7*t+phase),base.y+0.6*((0.35*t+up) mod 1),base.z+0.35*cos(0.7*t+phase)). Particle transform is recomputed absolutely and does not advance the PRNG.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; reset re-seeds and regenerates all particle constants before replay. No cut or resize.", + "randomness": { + "seed_uint32": 1592594996, + "seed_hex": "0x5eed1234", + "prng": "xorshift32: x ^= x<<13; x ^= x>>>17; x ^= x<<5 with uint32 masking after each operation; output=x/4294967296", + "policy": "Exactly four outputs per particle are consumed in ascending particle index during reset. Rendering consumes no random values." + }, + "active_inputs_effects": [ + "bench opaque color", + "transparent particle final color", + "manual particle coverage reactive mask", + "float depth", + "jitter-free velocity", + "auto exposure" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "accumulation-age", + "locks", + "reactivity" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "particle_volume": [ + 0.18, + 0.1, + 0.64, + 0.72 + ], + "opaque_edges": [ + 0.05, + 0.35, + 0.9, + 0.45 + ] + } + } + }, + { + "id": "Q6", + "name": "isolated-screenspace-effects", + "purpose": "Exercise isolated GTAO, SSR, and SSGI inputs.", + "initial_camera": { + "position": [ + 0, + 4, + 10 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(0,4,10), target=(0,3,-5), up=(0,1,0) for every n.", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Run three independent reset-before-n=0 subruns in order GTAO, SSR, SSGI. Each subrun reloads the page and creates one resolver.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier independently to each GTAO, SSR, and SSGI subrun. The exact barrier must complete before n=0; readiness renders are unrecorded, then upscaler history and NodeFrame are reset. GTAO, SSR without multibounce, SSGI, and DenoiseNode own no temporal history, so no readiness accumulation remains.", + "effect_sampling_sequence": "GTAO uses installed_three_effect_sequences.gtao with temporalDirection(n)=0. SSR uses installed_three_effect_sequences.ssr with stochastic=false, so its incremented noise index is dead shader state. SSGI uses installed_three_effect_sequences.ssgi with f(n)=n+1. SSR and SSGI spatial denoisers use installed_three_effect_sequences.denoise_node with the seeded noiseNode override and fixed index=0.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for the deterministic DenoiseNode SimplexNoise texture; GTAO magic-square and SSGI shader noise use their exact source formulas.", + "policy": "Generate the DenoiseNode texture once before compilation, then make no CPU PRNG calls. Record f(n)=n+1, SSGI temporalDirection/temporalOffset, GTAO temporalDirection=0, SSR stochastic=false, and DenoiseNode index=0 for every capture." + }, + "active_inputs_effects": [ + "Subrun gtao: beauty * GTAO(depth, normal)", + "Subrun ssr: beauty + spatially denoised SSR(beauty, depth, normal, packed metalness and roughness)", + "Subrun ssgi: beauty * AO + diffuse * spatially denoised SSGI GI", + "float depth", + "jitter-free velocity", + "normal MRT", + "effect-specific packed material or diffuse MRT", + "RCAS denoise=true" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "floor_reflection": [ + 0.05, + 0.5, + 0.9, + 0.5 + ], + "wall_contact_and_bounce": [ + 0.08, + 0.08, + 0.84, + 0.6 + ] + } + } + }, + { + "id": "Q7", + "name": "in-graph-screenspace-composition", + "purpose": "Exercise in-graph SSGI and SSR composition with one jitter owner.", + "initial_camera": { + "position": [ + 0, + 4, + 10.5 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(7*sin(0.15*t),4,9+1.5*cos(0.15*t)), target=(0,3,-5), up=(0,1,0).", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Reset once before n=0; no cut or resize. The complete graph is rebuilt only before n=0.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier to the complete in-graph SSGI+SSR subrun. The renderer/device, selected pipelines, graph queue, effect textures, G-buffer resources, and upscaler resources must be ready before n=0. Readiness renders are unrecorded; SSGI, non-stochastic SSR, and DenoiseNode are stateless, then upscaler history and NodeFrame are reset so recorded n=0 is the first accumulated frame.", + "effect_sampling_sequence": "SSGI uses installed_three_effect_sequences.ssgi with f(n)=n+1. SSR uses installed_three_effect_sequences.ssr with stochastic=false and no sampled analytic noise. Both spatial DenoiseNode instances use installed_three_effect_sequences.denoise_node with seed 0x0d3e015e and fixed index=0. The upscaler is the only projection-jitter owner.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for deterministic DenoiseNode SimplexNoise textures; SSGI shader noise follows its pinned analytical formulas.", + "policy": "Generate identical deterministic denoise textures before compilation and make no later CPU PRNG calls. Record f(n)=n+1, SSGI direction/offset, SSR stochastic=false, and each DenoiseNode index=0." + }, + "active_inputs_effects": [ + "in-graph beauty", + "SSGI AO and spatially denoised GI", + "spatially denoised SSR", + "float depth", + "jitter-free velocity", + "packed normal plus roughness MRT", + "packed diffuse plus metalness MRT", + "RCAS denoise=true", + "upscaler jitter=true and sole owner" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "179", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "glossy_floor": [ + 0.05, + 0.48, + 0.9, + 0.52 + ], + "colored_walls": [ + 0.05, + 0.05, + 0.9, + 0.62 + ] + } + } + }, + { + "id": "Q8", + "name": "recurrent-denoiser-characterization", + "purpose": "Record the existing recurrent-denoiser conflict without changing it.", + "initial_camera": { + "position": [ + 0, + 4, + 10.5 + ], + "target": [ + 0, + 3, + -5 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 55, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(7*sin(0.15*t),4,9+1.5*cos(0.15*t)), target=(0,3,-5), up=(0,1,0).", + "object_formula": "Use formula_conventions.room_scene_formula fixed geometry; no object transforms change.", + "light_exposure_formula": "Use room_scene_formula lighting; exposure=1.0 and autoExposure=true for every n.", + "events": "Run three independent reset-before-n=0 subruns in order builtin, spatial, recurrent. Each subrun reloads, rebuilds the graph, and resets all effect and upscaler history.", + "readiness_barrier": "Apply formula_conventions.installed_three_effect_sequences.readiness_barrier independently to builtin, spatial, and recurrent subruns; it must complete before n=0. After readiness and before NodeFrame reset, builtin only resets upscaler history; spatial resets upscaler history and forces RecurrentDenoiseNode.setSize(1,1); recurrent resets upscaler history and forces both TemporalReprojectNode.setSize(1,1) and RecurrentDenoiseNode.setSize(1,1). Thus recorded n=0 takes the installed source needsRestart seed/clear paths and no unrecorded temporal or denoise history survives.", + "effect_sampling_sequence": "All subruns use installed_three_effect_sequences.ssgi at sliceCount=2, stepCount=8, and f(n)=n+1. SSR is non-stochastic per installed_three_effect_sequences.ssr and its DenoiseNode uses the fixed seeded texture/index contract. Spatial and recurrent denoisers use installed_three_effect_sequences.recurrent_denoise with noiseIndex(n)=n+1; recurrent additionally uses installed_three_effect_sequences.temporal_reproject, which has no random index and seeds from current beauty on n=0.", + "randomness": { + "seed_uint32": 222167390, + "seed_hex": "0x0d3e015e", + "prng": "xorshift32 only for the deterministic SSR DenoiseNode SimplexNoise texture; SSGI and RecurrentDenoiseNode use their pinned shader formulas.", + "policy": "After deterministic texture creation, make no CPU PRNG calls. Record f(n)=n+1, SSGI direction/offset, RecurrentDenoiseNode noiseIndex and RNoise offset, SSR stochastic=false, and DenoiseNode index=0 for each frame." + }, + "active_inputs_effects": [ + "SSGI slices=2 and steps=8", + "Subrun builtin: raw SSGI GI with FSR owning temporal accumulation", + "Subrun spatial: recurrentDenoise mode=diffuse and accumulate=false", + "Subrun recurrent: temporalReproject plus recurrentDenoise mode=diffuse and accumulate=true", + "SSR=true", + "RCAS denoise=true", + "upscaler jitter=true" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "59", + "119", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "accumulation-age" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "flat_walls": [ + 0.08, + 0.08, + 0.84, + 0.52 + ], + "occlusion_edges": [ + 0.18, + 0.24, + 0.64, + 0.54 + ] + } + } + }, + { + "id": "Q9", + "name": "exposure-transition", + "purpose": "Replay a light step at frame 60, a linear ramp over frames 120 through 179, and a reverse transition at frame 180.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Directional color and position remain baseline. Directional intensity I(n)=3.2 for 0<=n<60; I(n)=8.0 for 60<=n<120; I(n)=8.0-6.0*(n-120)/59 for 120<=n<180; I(n)=3.2 for 180<=n<=239. Hemisphere intensity remains 0.9. settings.exposure=1.0 and autoExposure=true for every n; no external exposure texture is bound.", + "events": "Light step 3.2 to 8.0 at n=60; inclusive linear ramp 8.0 to 2.0 over n=120..179; reverse step 2.0 to 3.2 at n=180. Reset only before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "zero scene velocity except jitter compensation", + "auto exposure", + "shading-change detector", + "locks" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "2*P-1", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "149", + "178", + "179", + "180", + "181", + "182", + "184", + "188", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "lit_knots": [ + 0.18, + 0.16, + 0.64, + 0.38 + ], + "hdr_bulb": [ + 0.43, + 0.08, + 0.14, + 0.2 + ] + } + } + }, + { + "id": "Q10", + "name": "reset-cut-resize", + "purpose": "Replay a camera cut and history reset at frame 60 and deterministic resize resets at frames 120 and 180.", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "For 0<=n<60 position=(9,6,12); for 60<=n<=239 position=(-7,4,9). Target=(0,1.6,0) and up=(0,1,0) for every n. Projection is 50-degree perspective with near=0.1 and far=200; aspect equals the active output width/height after each resize.", + "object_formula": "Use formula_conventions.baseline_scene_formula exactly with absolute t=n/60; object animation does not restart at the mid-run history resets.", + "light_exposure_formula": "Use baseline lighting; exposure=1.0 and autoExposure=true for every n. Exposure adaptation history is cleared at each declared reset.", + "events": "Before n=0 configure 1920x1080 and reset. At the start of n=60 set camera position=(-7,4,9), recompute lookAt, then reset all upscaler histories and jitter to phase zero while the scenario frame remains 60. At the start of n=120 resize physical output to 1280x720 at DPR 1, rebuild size-dependent resources, and reset history/jitter. At the start of n=180 restore 1920x1080 at DPR 1, rebuild size-dependent resources, and reset history/jitter.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color", + "float depth", + "camera and object velocity", + "auto exposure", + "locks", + "shading-change detector", + "size-dependent resource rebuild" + ], + "captures": { + "frames": [ + "0", + "1", + "2", + "23", + "P-1", + "P", + "2*P-1", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "122", + "124", + "128", + "136", + "143", + "179", + "180", + "181", + "182", + "184", + "188", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "moving_silhouettes": [ + 0.08, + 0.18, + 0.84, + 0.58 + ] + } + } + } + ], + "required_debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "capture_protocol": { + "jitter_period_symbol": "P", + "jitter_period_definition": "The configured jitter phase count reported by the active baseline resolver for the current render and display dimensions.", + "jitter_period_by_ratio": { + "1": 8, + "1.5": 18, + "2": 32, + "3": 72 + }, + "harness_acceptance_matrix": { + "purpose": "E00 proves deterministic input/debug capture, static convergence, and cut/reset/resize handling without pre-running every later experiment's visual matrix. Q2-Q9 remain required implemented scenarios and receive reduced GPU smoke coverage; their authoritative captures belong to the domain experiments that consume them.", + "review_sampling": "Run all 45 numerical reload comparisons for every selected tuple. Human review samples the blinded A-B reload-1 pair at the frames and views below. Each review record receives one grade after the reviewer inspects the full frame and every declared ROI listed in that record.", + "reviewer_count": 1, + "human_review_scenarios": { + "Q0": { + "frames": [ + "P", + "2*P-1" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "Q1": { + "frames": [ + "119" + ], + "debug_views": [ + "final", + "accumulation-age" + ] + }, + "Q10": { + "frames": [ + "60", + "120", + "180" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ] + } + }, + "scenarios": { + "Q0": { + "frames": [ + "0", + "P", + "2*P-1" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age", + "locks", + "exposure", + "shading-change", + "reactivity" + ] + }, + "Q1": { + "frames": [ + "0", + "23", + "119" + ], + "debug_views": [ + "final", + "accumulation-age" + ] + }, + "Q10": { + "frames": [ + "59", + "60", + "61", + "119", + "120", + "121", + "179", + "180", + "181" + ], + "debug_views": [ + "final", + "motion-vectors", + "disocclusion", + "accumulation-age" + ] + } + } + }, + "absolute_frame_expressions": [ + "0", + "1", + "2", + "4", + "8", + "16", + "23", + "P-1", + "P", + "2*P-1", + "119" + ], + "event_relative_frames": { + "Q9_frame_60_light_step": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q9_frame_120_ramp_start": [ + -1, + 0, + 1, + 29, + 58, + 59 + ], + "Q9_frame_180_reverse_transition": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_60_camera_cut_and_reset": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_120_resize": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ], + "Q10_frame_180_restore_resize": [ + -1, + 0, + 1, + 2, + 4, + 8, + 16, + 23 + ] + }, + "rules": [ + "Resolve every frame expression to an integer and record P and the resolved frame list in the artifact metadata.", + "Reset and reload baseline and comparison runs to the same initial state before replay.", + "Capture only the canvas at the scenario's active physical dimensions; hide GUI, stats, and browser chrome.", + "Use the identical baseline presentation path and browser capture settings for both blinded labels.", + "Record scenario, ratio, frame, debug view, blinded label, and ROI with every image." + ], + "numerical_equivalence": { + "independent_reload_capture_count_per_variant": 5, + "comparison_pairs": "Compare all 10 unordered pairs among the five baseline reloads to derive repeatability, all 10 unordered pairs among the five candidate reloads, and the 25 baseline-to-candidate pairs for each identical scenario, subrun, ratio, resolved frame, debug view, and ROI tuple.", + "pixel_domain": "Decode lossless PNG to straight RGBA8. Compare R, G, and B as normalized display-referred sRGB code values byte/255 before any linearization, color management, resizing, filtering, or alpha premultiplication.", + "max_absolute_rgb_tolerance": 0.00392156862745098, + "rmse_rgb_tolerance": 0.000980392156862745, + "dimension_rule": "Both images must exactly equal the scenario's active physical canvas width and height. Q10 expects 1920x1080 before frame 120, 1280x720 for frames 120 through 179, and 1920x1080 from frame 180.", + "alpha_rule": "Every decoded pixel in both images must have alpha byte 255. Alpha is an exact gate and is excluded from RGB max-absolute and RMSE only after this gate passes.", + "metric_formula": "For N pixels and three RGB channels, maxAbsolute=max(abs(a_c-b_c))/255 and RMSE=sqrt(sum((a_c-b_c)/255)^2/(3*N)). Calculate full-frame and each declared ROI independently.", + "failure_artifacts": "For every failed pair, retain both PNGs, an absolute-difference heatmap with a fixed 0..1/255 legend, metric JSON, scenario metadata, and the failed ROI." + }, + "artifact_rubric": { + "0": "No visible difference at 1x or 4x nearest-neighbor inspection; numerical gates pass.", + "1": "Difference is detectable only at 4x inspection or in a difference heatmap; normal 1x viewing is unaffected.", + "2": "Difference is visible at 1x when the reviewer knows the ROI but is not distracting during motion.", + "3": "Difference is distracting at 1x or causes clear shimmer, ghosting, ringing, blur, exposure lag, mask shift, or reset residue.", + "4": "Invalid or severe output: black/NaN pixels, dimension/domain mismatch, full-screen flashing, device/validation failure, or unusable temporal instability." + }, + "pass_fail_aggregation": { + "pair": "PASS only when dimensions and alpha pass and full-frame plus every ROI max-absolute and RMSE are within tolerance.", + "capture_tuple": "PASS only when all 45 required repeatability and cross-variant pairs pass; every tuple selected by harness_acceptance_matrix.human_review_scenarios must receive a reviewer grade of 0 or 1, and the median grade across the experiment must be 0.", + "scenario": "PASS only when every required subrun, ratio, resolved frame, debug view, and ROI capture tuple passes.", + "experiment": "The E00 visual equivalence gate passes only when every tuple in capture_protocol.harness_acceptance_matrix passes. Q2-Q9 must pass reduced GPU smoke coverage before E00 closes and receive authoritative visual coverage in their consuming domain experiments. One failed pair fails its tuple and scenario; reruns may replace the complete five-capture set once, never only the failed image." + } + }, + "validation_gates": { + "static": [ + "git diff --check reports no errors.", + "The changed-path set is a subset of scope.allowed_write_paths.", + "Lint, typecheck, all unit tests, and build pass.", + "Structural tests prove the registry cannot instantiate more than one active resolver.", + "Baseline shader strings remain byte-for-byte identical or are demonstrated behaviorally equivalent." + ], + "runtime": [ + "Unknown or unsupported variants fail before rendering.", + "Exactly one resolver, one pass graph, and one projection-jitter owner are active per page load.", + "Candidate and baseline pipeline metadata identify shader key, settings, resource graph, supported ratios, WGSL overrides, and assembled chunks.", + "There are zero WGSL compilation errors, WebGPU validation errors, uncaught runtime exceptions, and device-loss events.", + "Every authoritative timing run has timestamp-query support and fresh frame-tagged samples.", + "No stale timing label survives a graph change and pending readbacks drain before finalization." + ], + "visual": [ + "Every required capture pair passes capture_protocol.numerical_equivalence: exact dimensions and alpha=255, max absolute RGB <= 1/255, and RGB RMSE <= 0.25/255 in display-referred sRGB code-value space. Measured repeatability is recorded but cannot widen either fixed threshold.", + "Motion vectors remain smooth and jitter-free, disocclusion does not flash full screen, accumulation age converges, locks and exposure reset correctly, shading-change is stable on a still scene, and reactivity remains aligned.", + "No black output, NaN output, incorrect dimensions, double tone mapping, output-domain shift, stale history, or shifted mask is present." + ], + "timing": [ + "Each run produces exactly 600 fresh measured samples after 240 warm-up frames.", + "All four ABBA repetitions complete at ratios 1, 1.5, 2, and 3.", + "At least three of four repetitions pass the equivalence threshold.", + "Baseline-versus-baseline median compute-sum delta is no greater than the larger of 3 percent or twice the measured baseline noise floor.", + "Baseline-versus-baseline p95 compute-sum delta is no greater than the larger of 5 percent or twice the measured baseline noise floor.", + "At every ratio, compute-sum N_median must be <=1.5 percent and N_p95 <=2.5 percent. Baseline equivalence then uses median max(3 percent,2*N_median) and p95 max(5 percent,2*N_p95), each passing at least three of four repetitions. One full cold-browser rerun is allowed; a second compute-sum noise-floor failure blocks E00. Per-pass noise eligibility is recorded separately and an ineligible pass cannot support an individual performance claim." + ], + "review": [ + "A fresh read-only reviewer verifies manifest compliance, scope, one-variant isolation, reset behavior, sample freshness, and evidence completeness.", + "Critical and important findings are consolidated into no more than two fix rounds.", + "No parity experiment starts until E00 is adopted and documented." + ] + }, + "timing_protocol": { + "required_feature": "timestamp-query", + "performance_claim_without_feature": "invalid", + "warmup_frames_per_run": 240, + "fresh_samples_per_run": 600, + "block_sequence": [ + "A", + "B", + "B", + "A" + ], + "block_repetitions": 4, + "variant_mapping": { + "A": "local-baseline-5d6a65e", + "B": "local-baseline-through-e00-harness" + }, + "noise_floor_derivation": { + "inputs": "For each ratio, pass label, compute sum, and statistic kind k in {median,p95}, each ABBA repetition r yields A1_r, B1_r, B2_r, A2_r from 600 fresh samples per run.", + "same_variant_relative_deltas": "dA_r=abs(A1_r-A2_r)/((A1_r+A2_r)/2) and dB_r=abs(B1_r-B2_r)/((B1_r+B2_r)/2).", + "quantile": "Sort the four dA values and independently the four dB values. q95 uses linear interpolation at zero-based index 0.95*(count-1). N_k=max(q95(dA),q95(dB)).", + "acceptable_noise_floor": "For compute sum at every ratio, N_median<=0.015 and N_p95<=0.025. Apply the same limits to per-pass claim eligibility when baseline median is at least 0.020 milliseconds; smaller passes are timer-resolution-limited. Any pass that exceeds either limit remains in compute sum but cannot support an individual performance claim until a pass-specific microbenchmark establishes a usable floor.", + "retry_policy": "If either compute-sum noise limit fails, discard the complete ratio run, cold-restart the browser once, and repeat all four ABBA repetitions. A second failure blocks E00 for that environment. Per-pass eligibility failures are recorded without triggering the aggregate E00 retry.", + "comparison_delta": "Within repetition r, meanA=(A1_r+A2_r)/2, meanB=(B1_r+B2_r)/2, and D_r=abs(meanA-meanB)/((meanA+meanB)/2). Median passes when D_r<=max(0.03,2*N_median); p95 passes when D_r<=max(0.05,2*N_p95). Each statistic must pass in at least three of four repetitions.", + "recording": "Record all A1/B1/B2/A2 statistics, dA, dB, sorted delta arrays, interpolated q95 values, N_median, N_p95, comparison limits, D values, pass counts, and retry state." + }, + "isolation": "Navigate or reload between runs. Instantiate and execute one resolver only; split-screen and concurrent A/B execution are forbidden for timing.", + "statistics": [ + "raw frame-tagged samples", + "missing-sample count", + "per-pass median", + "per-pass p95", + "compute-sum median", + "compute-sum p95", + "per-repetition A/B delta", + "baseline noise floor" + ], + "required_metadata": [ + "experiment ID", + "manifest digest", + "local SHA", + "working-tree digest including tracked and untracked non-ignored files", + "browser and version", + "operating system", + "GPU adapter", + "backend", + "WebGPU features", + "three version", + "physical dimensions", + "device pixel ratio", + "render ratio", + "fixed timestep", + "variant ID", + "shader key", + "pipeline key", + "settings", + "resource graph" + ], + "invalid_run_conditions": [ + "Missing timestamp-query support.", + "Any stale, duplicated, untagged, or missing measured sample.", + "Any validation, runtime, device-loss, output-domain, reset, resolver-isolation, or jitter-owner failure.", + "Any interactive input, wall-clock animation, resize handler, OrbitControls damping, or non-deterministic random input active in automated mode.", + "Baseline and comparison use different inputs, dimensions, ratios, presentation, capture settings, or browser process configuration." + ] + }, + "expected_behavior": { + "output_domain": "Display-referred sRGB produced by the existing internal ACES plus sRGB transform and stored in the existing output texture.", + "presentation_transform": "Identity sampling of the already transformed output; renderer remains NoToneMapping with LinearSRGBColorSpace and must not apply another tone or transfer transform.", + "jitter_owner": "The active upscaler resolver is the sole projection-jitter owner in temporal mode; automated inputs are rendered after its begin-frame jitter is applied.", + "velocity_convention": "Jitter-free NDC velocity converted by motionScale (0.5, -0.5), with reprojection prevUV = uv - motion.", + "reset_behavior": "Reset returns the deterministic frame clock and jitter sequence to frame zero, clears every temporal history, accumulation-age, lock, exposure-adaptation, pending timing, and scenario-event state, then reproduces frame-zero output without stale data.", + "resource_graph": "The existing local render, reconstruct, accumulate, luminance/exposure, RCAS or blit, debug, output, and history resources remain unchanged. The harness adds orchestration and metadata only and instantiates one graph.", + "binding_expectations": [ + "Every existing compute pass keeps the shared FsrConstants uniform buffer at group 0 binding 0.", + "Existing bind-group entry order, storage formats, sample types, depth-only views, and output mip-level view remain unchanged.", + "WGSL pipeline constants may specialize declared override values, but E00 introduces no alternate shader algorithm or additional production binding.", + "Disabled or unselected variants leave no alternate algorithm body, unused binding, pass, texture, or active resolver in the measured graph." + ] + }, + "required_evidence": [ + "Manifest ID and cryptographic digest.", + "Baseline and candidate IDs, source commit, local SHA, branch, and exact changed paths.", + "Command lines, exit codes, test totals, and execution timestamps.", + "Environment, adapter, WebGPU feature, dimensions, ratio, settings, shader key, pipeline key, and resource-graph metadata.", + "WGSL compilation, WebGPU validation, console, exception, and device-loss logs.", + "Resolved capture frames including P, scenario IDs, debug views, blinded labels, ROIs, and artifact-rubric observations.", + "Five-reload baseline and candidate capture sets, all fixed-threshold max-absolute/RMSE results, dimensions, alpha checks, rubric grades, and measured repeatability; repeatability cannot alter the manifest thresholds.", + "Fresh frame-tagged raw timestamp samples, missing count, per-pass and compute-sum median/p95, all ABBA repetition results, and measured noise floor.", + "Assertions for one active resolver, one jitter owner, output domain, reset state, resource graph, and binding stability.", + "Fresh reviewer findings, fix-round count, unresolved concerns, and controller decision rationale.", + "For platform-divergence: exact platform reason, affected platforms/scenarios/ratios/frames/debug views/ROIs, blinded visual and numerical impact, raw per-pass and compute-sum median/p95 impact, measured noise floor, and maintenance cost." + ], + "rollback_conditions": [ + "Any write outside scope.allowed_write_paths or any parity shader algorithm change.", + "Any WGSL compilation error, WebGPU validation error, uncaught runtime exception, or device loss.", + "Black or NaN output, incorrect dimensions, non-255 alpha, double tone mapping, output-domain mismatch, max absolute RGB above 1/255, or RGB RMSE above 0.25/255.", + "More than one active resolver or projection-jitter owner, or velocity generated from a jittered projection.", + "Full-screen disocclusion flashing, accumulation age that never converges, stale locks or history after reset, shifted reactive masks, or reset that retains prior timing or temporal state.", + "Missing, stale, duplicated, or untagged timing samples; undrained readbacks; or timing without timestamp-query.", + "Failure to complete exact sample counts and four ABBA repetitions at every required ratio.", + "More than two consolidated fix rounds without a new controller-authored manifest." + ], + "result_contract": { + "allowed_decisions": [ + "adopt", + "iterate", + "retain-local", + "platform-divergence", + "blocked" + ], + "decision_owner": "controller", + "decision_requirements": { + "adopt": "All static, runtime, visual, timing, evidence, and review gates pass.", + "iterate": "Evidence supports a bounded redesigned harness candidate that requires a new immutable manifest revision.", + "retain-local": "The harness candidate is validly measured but cannot preserve baseline correctness, determinism, quality, or acceptable overhead.", + "platform-divergence": "A documented browser or WebGPU platform constraint prevents the source-aligned harness behavior and measured evidence supports a bounded local alternative.", + "blocked": "A required dependency, capability, environment, or evidence gate remains unavailable after the allowed retry and fix policy." + } + }, + "report_contract": { + "allowed_statuses": [ + "PASS", + "FAIL", + "BLOCKED", + "SCOPE_BLOCKED", + "USER_DECISION_REQUIRED" + ], + "required_fields": [ + "status", + "changed_files", + "commands", + "artifacts", + "gates", + "concerns" + ], + "scope_rule": "If any additional write path or forbidden change appears necessary, stop before modifying it and return SCOPE_BLOCKED.", + "pass_rule": "PASS means only that the assigned implementation scope and all available assigned gates passed; it does not authorize adoption." + } +} diff --git a/bench/src/BenchPipeline.ts b/bench/src/BenchPipeline.ts index 40599fc..bd87e26 100644 --- a/bench/src/BenchPipeline.ts +++ b/bench/src/BenchPipeline.ts @@ -1,16 +1,126 @@ import * as THREE from 'three/webgpu'; -import { mrt, output, texture, velocity } from 'three/tsl'; - import { - Upscaler, - QualityMode, - getQualityModeRatio, - type DebugView, -} from '@pmndrs/upscaler'; + diffuseColor, + metalness, + mrt, + normalView, + output, + pass, + roughness, + texture, + vec4, + velocity, +} from 'three/tsl'; +import { ao } from 'three/addons/tsl/display/GTAONode.js'; +import { denoise } from 'three/addons/tsl/display/DenoiseNode.js'; +import { recurrentDenoise } from 'three/addons/tsl/display/RecurrentDenoiseNode.js'; +import { ssr } from 'three/addons/tsl/display/SSRNode.js'; +import { ssgi } from 'three/addons/tsl/display/SSGINode.js'; +import { temporalReproject } from 'three/addons/tsl/display/TemporalReprojectNode.js'; +import { SimplexNoise } from 'three/addons/math/SimplexNoise.js'; + +import { DebugView, QualityMode, getQualityModeRatio } from '@pmndrs/upscaler'; /** Bench render modes — what fills the screen each frame. */ export type BenchMode = 'native' | 'bilinear' | 'fsr1-spatial' | 'upscale-temporal'; +type EffectScenario = { + id: 'Q6' | 'Q7' | 'Q8'; + subrun: string | null; + scene: THREE.Scene; + camera: THREE.PerspectiveCamera; +}; + +type SizedEffectNode = { + setSize(width: number, height: number): void; +}; + +type EffectUpdateNode = SizedEffectNode & { + updateBefore(frame: unknown): void; +}; + +type SsrPrivateNode = { + _noiseIndex: { value: number }; +}; + +type TextureExpectation = { + name: string; + texture: THREE.Texture | (() => THREE.Texture); + width: number; + height: number; + format?: GPUTextureFormat; +}; + +type NodeFrameBridge = { + frameId: number; + renderId: number; + time: number; + deltaTime: number; + lastTime: number; + updateMap: WeakMap; + updateBeforeMap: WeakMap; + updateAfterMap: WeakMap; + update(): void; +}; + +function deterministicDenoiseNoise(): THREE.DataTexture { + let seed = 0x0d3e015e; + const simplex = new SimplexNoise({ + random(): number { + seed = (seed ^ ((seed << 13) >>> 0)) >>> 0; + seed = (seed ^ (seed >>> 17)) >>> 0; + seed = (seed ^ ((seed << 5) >>> 0)) >>> 0; + return seed / 4294967296; + }, + }); + const size = 64; + const data = new Uint8Array(size * size * 4); + for (let i = 0; i < size; i++) { + for (let j = 0; j < size; j++) { + const offset = (i * size + j) * 4; + data[offset] = (simplex.noise(i, j) * 0.5 + 0.5) * 255; + data[offset + 1] = (simplex.noise(i + size, j) * 0.5 + 0.5) * 255; + data[offset + 2] = (simplex.noise(i, j + size) * 0.5 + 0.5) * 255; + data[offset + 3] = (simplex.noise(i + size, j + size) * 0.5 + 0.5) * 255; + } + } + const noise = new THREE.DataTexture(data, size, size); + noise.wrapS = THREE.RepeatWrapping; + noise.wrapT = THREE.RepeatWrapping; + noise.needsUpdate = true; + return noise; +} + +function seededDenoise( + node: unknown, + depth: unknown, + normal: unknown, + camera: THREE.PerspectiveCamera, +): ReturnType { + const originalRandom = Math.random; + let constructorSeed = 0x0d3e015e; + Math.random = (): number => { + constructorSeed = (constructorSeed ^ ((constructorSeed << 13) >>> 0)) >>> 0; + constructorSeed = (constructorSeed ^ (constructorSeed >>> 17)) >>> 0; + constructorSeed = (constructorSeed ^ ((constructorSeed << 5) >>> 0)) >>> 0; + return constructorSeed / 4294967296; + }; + let effect: ReturnType; + try { + effect = denoise(node as never, depth as never, normal as never, camera); + } finally { + Math.random = originalRandom; + } + const mutable = effect as unknown as { + noiseNode: { value?: THREE.Texture }; + index: { value: number }; + }; + mutable.noiseNode.value?.dispose(); + mutable.noiseNode = texture(deterministicDenoiseNoise()) as never; + mutable.index.value = 0; + return effect; +} + /** * Owns everything between "a scene + camera" and "pixels on the canvas": * the low-resolution scene render target (color + velocity MRT + depth), @@ -21,33 +131,55 @@ export type BenchMode = 'native' | 'bilinear' | 'fsr1-spatial' | 'upscale-tempor * comparisons are apples-to-apples. */ export class BenchPipeline { - readonly upscaler: Upscaler; + readonly resolver: BenchmarkResolver; //* Presentation private readonly _renderer: THREE.WebGPURenderer; private readonly _quad: THREE.QuadMesh; private readonly _quadMaterial: THREE.NodeMaterial; + private readonly _depthOnlyMaterial: THREE.MeshBasicMaterial; + private readonly _effectMaterial: THREE.NodeMaterial; + private readonly _effectQuad: THREE.QuadMesh; //* Scene Target private _renderTarget: THREE.RenderTarget | null = null; + private _reactiveTarget: THREE.RenderTarget | null = null; private readonly _mrtNode = mrt({ output, velocity }); // Single-output MRT for non-temporal modes — its output count matches the // count:1 render target so color attachment 0 is actually written. private readonly _mrtOutputOnly = mrt({ output }); + private _effectScenario: EffectScenario | null = null; + private _effectPass: ReturnType | null = null; + private _effectSsrNodes: SsrPrivateNode[] = []; + private _effectSizedNodes: SizedEffectNode[] = []; + private _effectTemporalNodes: unknown[] = []; + private _effectTextures: TextureExpectation[] = []; + private _velocitySeedFrame = -2; private _mode: BenchMode = 'upscale-temporal'; private _quality: QualityMode = QualityMode.Quality; private _displayWidth = 0; private _displayHeight = 0; - constructor(renderer: THREE.WebGPURenderer) { + /** + * Creates a pipeline around exactly one resolver factory. + * @param renderer - Initialized WebGPU renderer + * @param resolverFactory - Registry-owned factory for the active variant + * @param metadata - Selected variant metadata + */ + constructor( + renderer: THREE.WebGPURenderer, + resolverFactory: BenchmarkResolverFactory, + metadata: BenchmarkVariantMetadata, + ) { this._renderer = renderer; - this.upscaler = new Upscaler({ renderer }); - this.upscaler.init(); + this.resolver = resolverFactory(renderer, metadata); // Motion vectors must be jitter-free — hand the velocity node the // upscaler's unjittered projection (contents refresh every frame). - velocity.setProjectionMatrix(this.upscaler.unjitteredProjectionMatrix); + velocity.setProjectionMatrix( + this.resolver.unjitteredProjectionMatrix as THREE.Matrix4, + ); this._quadMaterial = new THREE.NodeMaterial(); // Plain fullscreen present — no depth interaction, no fog. @@ -55,6 +187,15 @@ export class BenchPipeline { this._quadMaterial.depthWrite = false; this._quadMaterial.fog = false; this._quad = new THREE.QuadMesh(this._quadMaterial); + this._effectMaterial = new THREE.NodeMaterial(); + this._effectMaterial.depthTest = false; + this._effectMaterial.depthWrite = false; + this._effectMaterial.fog = false; + this._effectQuad = new THREE.QuadMesh(this._effectMaterial); + this._depthOnlyMaterial = new THREE.MeshBasicMaterial({ + colorWrite: false, + depthWrite: true, + }); } get mode(): BenchMode { @@ -65,6 +206,30 @@ export class BenchPipeline { return this._renderTarget; } + get metadata(): BenchmarkVariantMetadata { + return this.resolver.metadata; + } + + get usesEffectGraph(): boolean { + return this._effectScenario !== null; + } + + /** + * Selects the pinned three.js effect graph built during configuration. + * @param id - Q6, Q7, or Q8 + * @param subrun - Manifest-selected effect subrun + * @param scene - Fixed room fixture + * @param camera - Scenario camera + */ + configureEffectScenario( + id: 'Q6' | 'Q7' | 'Q8', + subrun: string | null, + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + ): void { + this._effectScenario = { id, subrun, scene, camera }; + } + /** * (Re)builds the pipeline for a display size, mode, and quality preset. * @param displayWidth - Canvas width in physical pixels @@ -77,6 +242,41 @@ export class BenchPipeline { displayHeight: number, mode: BenchMode, quality: QualityMode, + ): void { + const ratio = mode === 'native' ? 1 : getQualityModeRatio(quality); + this._configure(displayWidth, displayHeight, mode, quality, ratio, false); + } + + /** + * Configures an exact manifest ratio for automated execution. + * @param displayWidth - Physical output width + * @param displayHeight - Physical output height + * @param ratio - Manifest display/render ratio + * @param reactive - Allocate the Q5 reactive coverage target + */ + configureBenchmark( + displayWidth: number, + displayHeight: number, + ratio: number, + reactive: boolean, + ): void { + this._configure( + displayWidth, + displayHeight, + 'upscale-temporal', + QualityMode.Quality, + ratio, + reactive, + ); + } + + private _configure( + displayWidth: number, + displayHeight: number, + mode: BenchMode, + quality: QualityMode, + ratio: number, + reactive: boolean, ): void { this._mode = mode; this._quality = quality; @@ -84,11 +284,10 @@ export class BenchPipeline { this._displayHeight = displayHeight; //* Upscaler - const ratio = mode === 'native' ? 1 : getQualityModeRatio(quality); - this.upscaler.configure({ + this.resolver.configure({ displayWidth, displayHeight, - customUpscaleRatio: ratio, + ratio, path: mode === 'upscale-temporal' ? 'temporal' @@ -102,39 +301,395 @@ export class BenchPipeline { // match the MRT output count used in render() (a count:2 target // rendered without a velocity output leaves color attachment 0 black). this._renderTarget?.dispose(); - const rw = this.upscaler.renderWidth; - const rh = this.upscaler.renderHeight; + this._reactiveTarget?.dispose(); + this._reactiveTarget = null; + const rw = this.resolver.renderWidth; + const rh = this.resolver.renderHeight; const temporal = mode === 'upscale-temporal'; - const depthTexture = new THREE.DepthTexture(rw, rh); - depthTexture.type = THREE.FloatType; - this._renderTarget = new THREE.RenderTarget(rw, rh, { - count: temporal ? 2 : 1, - type: THREE.HalfFloatType, - depthTexture, - }); + if (this._effectScenario) { + this._renderTarget = new THREE.RenderTarget(rw, rh, { + count: 1, + type: THREE.HalfFloatType, + depthBuffer: false, + }); + } else { + const depthTexture = new THREE.DepthTexture(rw, rh); + depthTexture.type = THREE.FloatType; + this._renderTarget = new THREE.RenderTarget(rw, rh, { + count: temporal ? 2 : 1, + type: THREE.HalfFloatType, + depthTexture, + }); + } // MRT routes node outputs to attachments BY TEXTURE NAME (see // three's getTextureIndex) — these must match the mrt({...}) keys. this._renderTarget.textures[0].name = 'output'; - if (temporal) this._renderTarget.textures[1].name = 'velocity'; + if (temporal && !this._effectScenario) this._renderTarget.textures[1].name = 'velocity'; + + if (reactive) { + this._reactiveTarget = new THREE.RenderTarget(rw, rh, { + count: 1, + type: THREE.HalfFloatType, + }); + this._reactiveTarget.textures[0].name = 'output'; + } // Present the (re)created output texture on the quad. - this._quadMaterial.colorNode = texture(this.upscaler.outputTexture); + this._quadMaterial.colorNode = texture(this.resolver.outputTexture as THREE.Texture); this._quadMaterial.needsUpdate = true; + if (this._effectScenario) this._buildEffectGraph(ratio); + this.resolver.resetTiming(); + } + + private _buildEffectGraph(ratio: number): void { + const effect = this._effectScenario!; + const scenePass = pass(effect.scene, effect.camera); + this._effectPass = scenePass; + this._effectSsrNodes = []; + this._effectSizedNodes = []; + this._effectTemporalNodes = []; + this._effectTextures = []; + + const combined = effect.id === 'Q7' || effect.id === 'Q8'; + const isolated = effect.subrun; + if (combined) { + scenePass.setMRT( + mrt({ + output, + velocity, + normal: vec4(normalView, roughness), + diffuse: vec4(diffuseColor.rgb, metalness), + }), + ); + } else if (isolated === 'ssgi') { + scenePass.setMRT(mrt({ output, normal: normalView, velocity, diffuse: diffuseColor })); + } else if (isolated === 'ssr') { + scenePass.setMRT( + mrt({ + output, + normal: normalView, + velocity, + material: vec4(metalness, roughness, 0, 0), + }), + ); + } else { + scenePass.setMRT(mrt({ output, normal: normalView, velocity })); + } + scenePass.setResolutionScale(1 / ratio); + if (!scenePass.renderTarget.depthTexture) + throw new Error('Effect PassNode did not allocate its required depth texture.'); + scenePass.renderTarget.depthTexture.type = THREE.FloatType; + + const beauty = scenePass.getTextureNode('output'); + const depth = scenePass.getTextureNode('depth'); + const normal = scenePass.getTextureNode('normal'); + const vel = scenePass.getTextureNode('velocity'); + let rgb = beauty.rgb; + + if (isolated === 'gtao') { + const gtao = ao(depth, normal, effect.camera); + gtao.useTemporalFiltering = false; + const aoTexture = (gtao as unknown as { getTextureNode(): ReturnType }) + .getTextureNode(); + rgb = beauty.rgb.mul(aoTexture.r); + const target = gtao as unknown as { _aoRenderTarget: THREE.RenderTarget }; + if (!target._aoRenderTarget) + throw new Error('Pinned GTAONode._aoRenderTarget shape changed.'); + this._pinEffectResolution(gtao as unknown as EffectUpdateNode); + this._trackEffectTexture('gtao.output', target._aoRenderTarget.texture); + } + + if (combined || isolated === 'ssgi') { + const diffuse = scenePass.getTextureNode('diffuse'); + const giPass = ssgi(beauty, depth, normal, effect.camera); + if (effect.id === 'Q8') { + giPass.sliceCount.value = 2; + giPass.stepCount.value = 8; + } + const aoTexture = giPass.getAONode() as unknown as ReturnType; + const giRaw = giPass.getGINode(); + let gi = giRaw as unknown as ReturnType; + + if (effect.id === 'Q8' && effect.subrun === 'spatial') { + const spatial = recurrentDenoise(giRaw as never, effect.camera, { + depth: depth as never, + normal: normal as never, + raw: giRaw as never, + mode: 'diffuse', + accumulate: false, + }); + gi = spatial as unknown as ReturnType; + this._effectSizedNodes.push(spatial as unknown as SizedEffectNode); + this._pinEffectResolution(spatial as unknown as EffectUpdateNode); + const spatialTarget = ( + spatial as unknown as { getRenderTarget(): THREE.RenderTarget } + ).getRenderTarget(); + this._trackEffectTexture('recurrent.spatial-output', spatialTarget.texture); + } else if (effect.id === 'Q8' && effect.subrun === 'recurrent') { + const reproject = temporalReproject( + giRaw as never, + depth as never, + normal as never, + vel as never, + effect.camera, + { mode: 'diffuse' }, + ); + const recurrent = recurrentDenoise(reproject as never, effect.camera, { + depth: depth as never, + normal: normal as never, + raw: giRaw as never, + mode: 'diffuse', + accumulate: true, + }); + reproject.setHistoryTexture(recurrent as never); + gi = recurrent as unknown as ReturnType; + this._effectSizedNodes.push( + reproject as unknown as SizedEffectNode, + recurrent as unknown as SizedEffectNode, + ); + this._effectTemporalNodes.push(reproject, recurrent); + this._pinEffectResolution(reproject as unknown as EffectUpdateNode); + this._pinEffectResolution(recurrent as unknown as EffectUpdateNode); + const temporalPrivate = reproject as unknown as { + _historyRenderTarget: THREE.RenderTarget; + _resolveRenderTarget: THREE.RenderTarget; + _previousNormalTexture: THREE.Texture; + }; + const recurrentTarget = ( + recurrent as unknown as { getRenderTarget(): THREE.RenderTarget } + ).getRenderTarget(); + if ( + !temporalPrivate._historyRenderTarget || + !temporalPrivate._resolveRenderTarget || + !temporalPrivate._previousNormalTexture + ) + throw new Error('Pinned TemporalReprojectNode render-target shape changed.'); + if (!temporalPrivate._historyRenderTarget.depthTexture) + throw new Error('Temporal history depth texture is unavailable.'); + temporalPrivate._historyRenderTarget.depthTexture.type = THREE.FloatType; + this._trackEffectTexture( + 'temporal.history-color', + temporalPrivate._historyRenderTarget.texture, + ); + if (temporalPrivate._historyRenderTarget.depthTexture) + this._trackEffectTexture( + 'temporal.history-depth', + temporalPrivate._historyRenderTarget.depthTexture, + undefined, + undefined, + 'depth32float', + ); + this._trackEffectTexture( + 'temporal.resolve', + temporalPrivate._resolveRenderTarget.texture, + ); + this._trackEffectTexture( + 'temporal.previous-normal', + () => temporalPrivate._previousNormalTexture, + ); + this._trackEffectTexture('recurrent.output', recurrentTarget.texture); + } else if (effect.id !== 'Q8') { + const denoised = seededDenoise(giRaw, depth, normal, effect.camera); + gi = denoised as unknown as ReturnType< + typeof vec4 + >; + const noise = (denoised as unknown as { noiseNode: { value: THREE.Texture } }) + .noiseNode.value; + this._trackEffectTexture( + 'denoise.ssgi-noise', + noise, + 64, + 64, + 'rgba8unorm', + ); + } + rgb = beauty.rgb.mul(aoTexture.r).add(diffuse.rgb.mul(gi.rgb)); + const privatePass = giPass as unknown as { + _ssgiRenderTarget: THREE.RenderTarget; + }; + if (!privatePass._ssgiRenderTarget) + throw new Error('Pinned SSGINode render-target shape changed.'); + this._pinEffectResolution(giPass as unknown as EffectUpdateNode); + privatePass._ssgiRenderTarget.textures.forEach((effectTexture, index) => + this._trackEffectTexture(`ssgi.attachment-${index}`, effectTexture), + ); + } + + if (combined || isolated === 'ssr') { + const material = + isolated === 'ssr' + ? scenePass.getTextureNode('material') + : scenePass.getTextureNode('diffuse'); + const rough = isolated === 'ssr' ? material.g : normal.a; + const metal = isolated === 'ssr' ? material.r : material.a; + const reflection = ssr(beauty, depth, normal as never, { + stochastic: false, + metalnessNode: metal, + roughnessNode: rough, + camera: effect.camera, + }); + const privateSsr = reflection as unknown as SsrPrivateNode & { + _ssrRenderTarget: THREE.RenderTarget; + }; + if (!privateSsr._noiseIndex || !privateSsr._ssrRenderTarget) + throw new Error('Pinned SSRNode private shape changed.'); + this._effectSsrNodes.push(privateSsr); + this._pinEffectResolution(reflection as unknown as EffectUpdateNode); + this._trackEffectTexture('ssr.base', privateSsr._ssrRenderTarget.texture); + const blurTarget = ( + reflection as unknown as { _blurRenderTarget?: THREE.RenderTarget } + )._blurRenderTarget; + if (!blurTarget) throw new Error('Pinned SSRNode._blurRenderTarget shape changed.'); + this._trackEffectTexture('ssr.blur-mips', blurTarget.texture); + const reflectionTexture = ( + reflection as unknown as { getTextureNode(): unknown } + ).getTextureNode(); + const filteredNode = seededDenoise( + reflectionTexture, + depth, + normal, + effect.camera, + ); + const filtered = filteredNode as unknown as ReturnType; + const noise = (filteredNode as unknown as { noiseNode: { value: THREE.Texture } }) + .noiseNode.value; + this._trackEffectTexture('denoise.ssr-noise', noise, 64, 64, 'rgba8unorm'); + rgb = rgb.add(filtered.rgb); + } + + this._effectMaterial.colorNode = vec4(rgb, beauty.a); + this._effectMaterial.needsUpdate = true; + const depthTexture = scenePass.renderTarget.depthTexture; + const velocityTexture = scenePass.getTexture('velocity'); + if (!depthTexture || !velocityTexture) + throw new Error('Effect graph did not expose depth and velocity textures.'); + scenePass.renderTarget.textures.forEach((effectTexture, index) => + this._trackEffectTexture(`scene-mrt.attachment-${index}`, effectTexture), + ); + this._trackEffectTexture('scene-mrt.depth', depthTexture, undefined, undefined, 'depth32float'); + this._trackEffectTexture('scene-mrt.velocity', velocityTexture); + this._trackEffectTexture('effect.intermediate', this._renderTarget!.texture); + this._trackEffectTexture( + 'resolver.output', + this.resolver.outputTexture as THREE.Texture, + this.resolver.displayWidth, + this.resolver.displayHeight, + ); + } + + private _pinEffectResolution(node: EffectUpdateNode): void { + if (typeof node.setSize !== 'function' || typeof node.updateBefore !== 'function') + throw new Error('Pinned effect sizing shape changed.'); + const width = this.resolver.renderWidth; + const height = this.resolver.renderHeight; + const originalUpdate = node.updateBefore; + node.setSize(width, height); + node.updateBefore = function updateAtResolverResolution(frame: unknown): void { + const renderer = (frame as { renderer?: THREE.WebGPURenderer }).renderer; + if (!renderer || typeof renderer.getDrawingBufferSize !== 'function') + throw new Error('Pinned effect NodeFrame renderer shape changed.'); + const mutableRenderer = renderer as unknown as { + getDrawingBufferSize(target: THREE.Vector2): THREE.Vector2; + }; + const originalGetDrawingBufferSize = mutableRenderer.getDrawingBufferSize; + mutableRenderer.getDrawingBufferSize = (target: THREE.Vector2): THREE.Vector2 => + target.set(width, height); + try { + originalUpdate.call(node, frame); + } finally { + mutableRenderer.getDrawingBufferSize = originalGetDrawingBufferSize; + } + }; + } + + private _trackEffectTexture( + name: string, + textureValue: THREE.Texture | (() => THREE.Texture), + width = this.resolver.renderWidth, + height = this.resolver.renderHeight, + format?: GPUTextureFormat, + ): void { + this._effectTextures.push({ name, texture: textureValue, width, height, format }); + } + + private _rawTexture(textureValue: THREE.Texture): GPUTexture | null { + const backend = this._renderer.backend as unknown as { + get(texture: THREE.Texture): { texture?: GPUTexture } | undefined; + }; + return backend.get(textureValue)?.texture ?? null; + } + + private _assertTexture(expectation: TextureExpectation): void { + const textureValue = + typeof expectation.texture === 'function' + ? expectation.texture() + : expectation.texture; + const raw = this._rawTexture(textureValue); + if (!raw) throw new Error(`GPU texture ${expectation.name} is not backed.`); + if (raw.width !== expectation.width || raw.height !== expectation.height) + throw new Error( + `GPU texture ${expectation.name} is ${raw.width}x${raw.height}; ` + + `expected ${expectation.width}x${expectation.height}.`, + ); + if (expectation.format && raw.format !== expectation.format) + throw new Error( + `GPU texture ${expectation.name} uses ${raw.format}; expected ${expectation.format}.`, + ); + } + + private _assertDispatchTextures( + color: THREE.Texture, + depth: THREE.Texture | undefined, + velocityTexture: THREE.Texture | undefined, + reactive: THREE.Texture | undefined, + ): void { + const width = this.resolver.renderWidth; + const height = this.resolver.renderHeight; + this._assertTexture({ name: 'dispatch.color', texture: color, width, height }); + if (depth) + this._assertTexture({ name: 'dispatch.depth', texture: depth, width, height }); + if (velocityTexture) + this._assertTexture({ + name: 'dispatch.velocity', + texture: velocityTexture, + width, + height, + }); + if (reactive) + this._assertTexture({ name: 'dispatch.reactive', texture: reactive, width, height }); + this._assertTexture({ + name: 'resolver.output', + texture: this.resolver.outputTexture as THREE.Texture, + width: this.resolver.displayWidth, + height: this.resolver.displayHeight, + }); } /** - * Renders one frame: scene → render target → FSR passes → canvas quad. + * Renders deterministic scene inputs without dispatching or presenting. * @param scene - Scene to render * @param camera - Scene camera - * @param deltaTime - Seconds since last frame + * @param reactiveScene - Optional Q5 particle-coverage scene */ - render(scene: THREE.Scene, camera: THREE.PerspectiveCamera, deltaTime: number): void { + renderInput( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + reactiveScene?: THREE.Scene, + ): void { const rt = this._renderTarget; if (!rt) return; const temporal = this._mode === 'upscale-temporal'; //* Scene Pass (jittered when temporal) - this.upscaler.beginFrame(camera); + this.resolver.beginFrame(camera); + if (this._effectPass) { + this._renderer.setMRT(null); + this._renderer.setRenderTarget(rt); + this._effectQuad.render(this._renderer); + this._renderer.setRenderTarget(null); + this.resolver.endFrame(camera); + return; + } // The MRT output count MUST match the render target's attachment count: // rendering into a count:2 target without the velocity output leaves // color attachment 0 unwritten (black). Non-temporal modes therefore @@ -144,23 +699,295 @@ export class BenchPipeline { this._renderer.render(scene, camera); this._renderer.setRenderTarget(null); this._renderer.setMRT(null); - this.upscaler.endFrame(camera); - //* FSR Passes - this.upscaler.dispatch( + //* Optional Q5 Reactive Coverage + if (reactiveScene && this._reactiveTarget) { + const autoClear = this._renderer.autoClear; + this._renderer.autoClear = false; + this._renderer.setMRT(this._mrtOutputOnly); + this._renderer.setRenderTarget(this._reactiveTarget); + this._renderer.clear(true, true, false); + + // Populate an opaque-only depth attachment first. Particles live on + // layer 1, so the manual coverage pass cannot flag occluded volume. + const overrideMaterial = scene.overrideMaterial; + const background = scene.background; + const cameraLayerMask = camera.layers.mask; + scene.overrideMaterial = this._depthOnlyMaterial; + scene.background = null; + camera.layers.disable(1); + this._renderer.render(scene, camera); + scene.overrideMaterial = overrideMaterial; + scene.background = background; + camera.layers.mask = cameraLayerMask; + this._renderer.render(reactiveScene, camera); + this._renderer.setRenderTarget(null); + this._renderer.setMRT(null); + this._renderer.autoClear = autoClear; + } + this.resolver.endFrame(camera); + } + + /** + * Dispatches the active resolver against the most recent input render. + * @param camera - Camera used for input rendering + * @param deltaTime - Fixed or interactive timestep + * @param frameTag - Deterministic frame identity for fresh timing + */ + dispatchResolver( + camera: THREE.PerspectiveCamera, + deltaTime: number, + frameTag: number, + ): void { + const rt = this._renderTarget; + if (!rt) return; + const temporal = this._mode === 'upscale-temporal'; + const effectDepth = this._effectPass?.renderTarget.depthTexture ?? undefined; + const effectVelocity = this._effectPass?.getTexture('velocity'); + const color = rt.textures[0]; + const depth = effectDepth ?? rt.depthTexture ?? undefined; + const velocityTexture = this._effectPass + ? effectVelocity + : temporal + ? rt.textures[1] + : undefined; + const reactive = this._reactiveTarget?.textures[0]; + this._assertDispatchTextures(color, depth, velocityTexture, reactive); + this.resolver.dispatch( { - color: rt.textures[0], - depth: rt.depthTexture ?? undefined, - velocity: temporal ? rt.textures[1] : undefined, + color, + depth, + velocity: velocityTexture, + reactive, deltaTime, + frameTag, }, camera, ); + } - //* Present — output is already display-referred sRGB + /** Presents the resolver output without another transfer transform. */ + present(): void { this._quad.render(this._renderer); } + /** Advances the pinned NodeFrame exactly once for one automated frame. */ + advanceAutomatedFrame(frame: number): void { + const nodeFrame = this._nodeFrame(); + if (nodeFrame.frameId !== frame) + throw new Error( + `Pinned NodeFrame expected frameId ${frame} before update; got ${nodeFrame.frameId}.`, + ); + nodeFrame.update(); + nodeFrame.time = (frame + 1) / 60; + nodeFrame.deltaTime = 1 / 60; + nodeFrame.lastTime = performance.now(); + } + + /** + * Compiles and allocates the selected effect graph before recorded frame zero. + * @param camera - Effect scenario camera + */ + async prepareEffectReadiness(camera: THREE.PerspectiveCamera): Promise { + if (!this._effectScenario) return; + const backend = this._renderer.backend as unknown as { + device?: GPUDevice; + get(texture: THREE.Texture): { texture?: GPUTexture } | undefined; + }; + if (!backend.device || typeof backend.device.queue?.onSubmittedWorkDone !== 'function') + throw new Error('Pinned WebGPU backend device shape changed.'); + const nodes = this._nodeManager(); + this._resetEffectState(camera); + this._resetNodeFrame(0); + + let ready = false; + for (let readinessFrame = 0; readinessFrame < 180; readinessFrame++) { + this.advanceAutomatedFrame(readinessFrame); + this.renderInput(this._effectScenario.scene, camera); + this.dispatchResolver(camera, 1 / 60, -1 - readinessFrame); + this.present(); + await backend.device.queue.onSubmittedWorkDone(); + try { + this._effectTextures.forEach((expectation) => this._assertTexture(expectation)); + ready = nodes._buildQueue.length === 0 && nodes._buildInProgress === false; + } catch { + ready = false; + } + if (ready) break; + await new Promise((resolve) => setTimeout(resolve, 0)); + } + if (!ready) throw new Error('Effect graph readiness barrier timed out.'); + + await this.drainTiming(); + this.reset(this._effectScenario.scene, camera); + } + + private _nodeManager(): { + nodeFrame: NodeFrameBridge; + _buildQueue: unknown[]; + _buildInProgress: boolean; + } { + const nodes = (this._renderer as unknown as { _nodes?: unknown })._nodes as { + nodeFrame?: NodeFrameBridge; + _buildQueue?: unknown[]; + _buildInProgress?: boolean; + }; + if ( + !nodes?.nodeFrame || + !Array.isArray(nodes._buildQueue) || + typeof nodes._buildInProgress !== 'boolean' + ) + throw new Error('Pinned renderer._nodes readiness shape changed.'); + return nodes as { + nodeFrame: NodeFrameBridge; + _buildQueue: unknown[]; + _buildInProgress: boolean; + }; + } + + private _nodeFrame(): NodeFrameBridge { + const nodeFrame = this._nodeManager().nodeFrame; + if ( + !(nodeFrame.updateMap instanceof WeakMap) || + !(nodeFrame.updateBeforeMap instanceof WeakMap) || + !(nodeFrame.updateAfterMap instanceof WeakMap) || + typeof nodeFrame.update !== 'function' + ) + throw new Error('Pinned NodeFrame mutable shape changed.'); + return nodeFrame; + } + + private _resetEffectState(camera: THREE.PerspectiveCamera): void { + if (!this._effectScenario) return; + for (const ssrNode of this._effectSsrNodes) { + if (!ssrNode._noiseIndex || typeof ssrNode._noiseIndex.value !== 'number') + throw new Error('Pinned SSRNode._noiseIndex shape changed.'); + ssrNode._noiseIndex.value = 0; + } + if (this._effectScenario.id === 'Q8') + for (const sizedNode of this._effectSizedNodes) { + if (typeof sizedNode.setSize !== 'function') + throw new Error('Pinned recurrent effect setSize shape changed.'); + sizedNode.setSize(1, 1); + } + + for (const temporalNode of this._effectTemporalNodes) { + const candidate = temporalNode as { + _cameraUniforms?: { updateFromCamera(cameraValue: THREE.Camera): void }; + _noiseIndex?: { value: number }; + }; + if (candidate._cameraUniforms) { + if (typeof candidate._cameraUniforms.updateFromCamera !== 'function') + throw new Error('Pinned temporal camera-uniform shape changed.'); + candidate._cameraUniforms.updateFromCamera(camera); + candidate._cameraUniforms.updateFromCamera(camera); + } + if (candidate._noiseIndex) candidate._noiseIndex.value = 0; + } + } + + private _resetNodeFrame(frame: number): void { + const nodeFrame = this._nodeFrame(); + nodeFrame.frameId = frame; + nodeFrame.renderId = 0; + nodeFrame.time = frame / 60; + nodeFrame.deltaTime = 0; + nodeFrame.lastTime = performance.now(); + nodeFrame.updateMap = new WeakMap(); + nodeFrame.updateBeforeMap = new WeakMap(); + nodeFrame.updateAfterMap = new WeakMap(); + } + + private _seedVelocityHistory( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + ): void { + const velocityNode = velocity as unknown as { + update(frame: { + frameId: number; + camera: THREE.Camera; + object: THREE.Object3D; + }): void; + updateAfter(frame: { object: THREE.Object3D }): void; + }; + if ( + typeof velocityNode.update !== 'function' || + typeof velocityNode.updateAfter !== 'function' + ) + throw new Error('Pinned VelocityNode reset bridge shape changed.'); + + scene.updateMatrixWorld(true); + camera.updateMatrixWorld(true); + const objects: THREE.Object3D[] = []; + scene.traverseVisible((object) => { + if ( + (object as THREE.Mesh).isMesh || + (object as THREE.Line).isLine || + (object as THREE.Points).isPoints + ) + objects.push(object); + }); + for (let passIndex = 0; passIndex < 2; passIndex++) { + const frameId = this._velocitySeedFrame--; + for (const object of objects) { + velocityNode.update({ frameId, camera, object }); + velocityNode.updateAfter({ object }); + } + } + } + + /** + * Interactive convenience preserving the original one-call lifecycle. + * @param scene - Scene to render + * @param camera - Scene camera + * @param deltaTime - Seconds since the previous frame + * @param frameTag - Optional interactive frame tag + */ + render( + scene: THREE.Scene, + camera: THREE.PerspectiveCamera, + deltaTime: number, + frameTag = 0, + ): void { + this.renderInput(scene, camera); + this.dispatchResolver(camera, deltaTime, frameTag); + this.present(); + } + + /** Waits until a fresh timing slot can accept another frame. */ + prepareTiming(): Promise { + return this.resolver.waitForTimingCapacity(); + } + + /** Drains all pending timestamp-query readbacks. */ + drainTiming(): Promise { + return this.resolver.drainTiming(); + } + + /** Returns fresh timing samples collected since the last take/reset. */ + takeTimingSamples(): BenchmarkGpuFrameSample[] { + return this.resolver.takeTimingSamples(); + } + + /** + * Clears all temporal state and reseeds three's velocity history. + * @param scene - Scene at the reset event's absolute transforms + * @param camera - Camera at the reset event's absolute transform + * @param frame - Declared scenario frame retained by event resets + */ + reset(scene?: THREE.Scene, camera?: THREE.PerspectiveCamera, frame = 0): void { + this.resolver.reset(); + this._resetNodeFrame(frame); + if (camera) this._resetEffectState(camera); + const activeScene = this._effectScenario?.scene ?? scene; + if (activeScene && camera) { + ( + this.resolver.unjitteredProjectionMatrix as THREE.Matrix4 + ).copy(camera.projectionMatrix); + this._seedVelocityHistory(activeScene, camera); + } + } + /** Applies runtime settings from the UI (no reconfigure needed). */ applySettings(settings: { sharpness: number; @@ -172,11 +999,19 @@ export class BenchPipeline { detectShadingChanges: boolean; debugView: DebugView; }): void { - Object.assign(this.upscaler.settings, settings); + Object.assign(this.resolver.settings, settings); + // Debug buffers are already normalized visualization colors; only the + // final linear/HDR result should pass through presentation tone mapping. + this._quadMaterial.toneMapped = settings.debugView === DebugView.None; } dispose(): void { + this._effectPass?.dispose(); this._renderTarget?.dispose(); - this.upscaler.dispose(); + this._reactiveTarget?.dispose(); + this._quadMaterial.dispose(); + this._effectMaterial.dispose(); + this._depthOnlyMaterial.dispose(); + this.resolver.dispose(); } } diff --git a/bench/src/BenchScene.ts b/bench/src/BenchScene.ts index ce05f38..ba5949d 100644 --- a/bench/src/BenchScene.ts +++ b/bench/src/BenchScene.ts @@ -9,8 +9,14 @@ import * as THREE from 'three/webgpu'; */ export interface BenchScene { scene: THREE.Scene; + roomScene: THREE.Scene; + reactiveScene: THREE.Scene; /** Advances animations. @param time - Elapsed seconds @param animate - Freeze toggle */ update(time: number, animate: boolean): void; + /** Applies a deterministic absolute scenario frame. */ + applyFrame(frame: BenchmarkFrameState): void; + /** Recreates all seeded Q5 particle constants. */ + resetDeterministicState(): void; } /** Builds the checkerboard+grid floor texture on a canvas (no asset deps). */ @@ -57,6 +63,8 @@ function createGridTexture(): THREE.CanvasTexture { */ export function createBenchScene(): BenchScene { const scene = new THREE.Scene(); + const roomScene = new THREE.Scene(); + const reactiveScene = new THREE.Scene(); scene.background = new THREE.Color(0x10141a); scene.fog = new THREE.Fog(0x10141a, 40, 90); @@ -66,6 +74,66 @@ export function createBenchScene(): BenchScene { scene.add(sun); scene.add(new THREE.HemisphereLight(0x9fb4d4, 0x2a2620, 0.9)); + //* Screen-Space Effect Room ============================================== + roomScene.background = new THREE.Color(0x0a0c10); + const roomSun = new THREE.DirectionalLight(0xfff2df, 3.2); + roomSun.position.set(8, 14, 6); + roomScene.add(roomSun); + roomScene.add(new THREE.HemisphereLight(0x9fb4d4, 0x2a2620, 0.9)); + roomScene.add(new THREE.AmbientLight(0x404860, 0.4)); + + const roomFloor = new THREE.Mesh( + new THREE.PlaneGeometry(60, 60), + new THREE.MeshStandardMaterial({ + color: 0x20242c, + metalness: 0.9, + roughness: 0.12, + }), + ); + roomFloor.rotation.x = -Math.PI / 2; + roomScene.add(roomFloor); + + const wallGeometry = new THREE.BoxGeometry(20, 10, 0.4); + const leftWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0xc0392b, roughness: 0.9 }), + ); + leftWall.position.set(-8, 5, -4); + leftWall.rotation.y = Math.PI / 2; + roomScene.add(leftWall); + const rightWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0x2ecc71, roughness: 0.9 }), + ); + rightWall.position.set(8, 5, -4); + rightWall.rotation.y = -Math.PI / 2; + roomScene.add(rightWall); + const backWall = new THREE.Mesh( + wallGeometry, + new THREE.MeshStandardMaterial({ color: 0x8a8f98, roughness: 0.9 }), + ); + backWall.position.set(0, 5, -12); + roomScene.add(backWall); + + for (let i = 0; i < 5; i++) { + const box = new THREE.Mesh( + new THREE.BoxGeometry(1.6, 2 + i * 0.5, 1.6), + new THREE.MeshStandardMaterial({ color: 0xd8d2c4, roughness: 0.6 }), + ); + box.position.set(-5 + i * 2.5, 1 + i * 0.25, -6 + (i % 2) * 3); + roomScene.add(box); + } + const roomBall = new THREE.Mesh( + new THREE.SphereGeometry(1.4, 48, 32), + new THREE.MeshStandardMaterial({ + color: 0xdfe6f0, + metalness: 0.5, + roughness: 0.15, + }), + ); + roomBall.position.set(2, 1.6, -2); + roomScene.add(roomBall); + //* Floor const floor = new THREE.Mesh( new THREE.PlaneGeometry(120, 120), @@ -123,8 +191,63 @@ export function createBenchScene(): BenchScene { bulb.position.set(0, 5.5, -2); scene.add(bulb); - function update(time: number, animate: boolean): void { - if (!animate) return; + //* Seeded Transparency Fixture ============================================ + const particleCount = 128; + const particleGeometry = new THREE.SphereGeometry(0.06, 8, 6); + const particles = new THREE.InstancedMesh( + particleGeometry, + new THREE.MeshBasicMaterial({ + color: 0x7fdfff, + transparent: true, + opacity: 0.7, + blending: THREE.AdditiveBlending, + depthWrite: false, + }), + particleCount, + ); + const reactiveParticles = new THREE.InstancedMesh( + particleGeometry, + new THREE.MeshBasicMaterial({ + color: 0xffffff, + depthTest: true, + depthWrite: false, + }), + particleCount, + ); + particles.layers.set(1); + reactiveParticles.layers.set(1); + particles.visible = false; + reactiveParticles.visible = false; + scene.add(particles); + reactiveScene.add(reactiveParticles); + + const particleBase = new Float32Array(particleCount * 3); + const particlePhase = new Float32Array(particleCount); + const particleUp = new Float32Array(particleCount); + + function resetDeterministicState(): void { + let seed = 0x5eed1234; + const random = (): number => { + seed = (seed ^ ((seed << 13) >>> 0)) >>> 0; + seed = (seed ^ (seed >>> 17)) >>> 0; + seed = (seed ^ ((seed << 5) >>> 0)) >>> 0; + return seed / 4294967296; + }; + + for (let i = 0; i < particleCount; i++) { + const ux = random(); + const uy = random(); + const uz = random(); + const up = random(); + particleBase[i * 3] = -5 + 10 * ux; + particleBase[i * 3 + 1] = 0.7 + 4 * uy; + particleBase[i * 3 + 2] = -5 + 10 * uz; + particlePhase[i] = 2 * Math.PI * up; + particleUp[i] = up; + } + } + + function updateObjects(time: number): void { knots.forEach((knot, i) => { knot.rotation.x = time * 0.35 + i; knot.rotation.y = time * 0.5; @@ -139,5 +262,40 @@ export function createBenchScene(): BenchScene { }); } - return { scene, update }; + function updateParticles(time: number): void { + const matrix = new THREE.Matrix4(); + for (let i = 0; i < particleCount; i++) { + const phase = particlePhase[i]; + const up = particleUp[i]; + const x = particleBase[i * 3] + 0.35 * Math.sin(0.7 * time + phase); + const y = + particleBase[i * 3 + 1] + 0.6 * ((0.35 * time + up) % 1); + const z = + particleBase[i * 3 + 2] + 0.35 * Math.cos(0.7 * time + phase); + matrix.makeTranslation(x, y, z); + particles.setMatrixAt(i, matrix); + reactiveParticles.setMatrixAt(i, matrix); + } + particles.instanceMatrix.needsUpdate = true; + reactiveParticles.instanceMatrix.needsUpdate = true; + } + + resetDeterministicState(); + updateObjects(0); + updateParticles(0); + + function update(time: number, animate: boolean): void { + if (!animate) return; + updateObjects(time); + } + + function applyFrame(frame: BenchmarkFrameState): void { + updateObjects(frame.animateScene ? frame.sceneTime : 0); + updateParticles(frame.time); + particles.visible = frame.particlesVisible; + reactiveParticles.visible = frame.particlesVisible; + sun.intensity = frame.directionalIntensity; + } + + return { scene, roomScene, reactiveScene, update, applyFrame, resetDeterministicState }; } diff --git a/bench/src/benchmark/BenchmarkResolver.ts b/bench/src/benchmark/BenchmarkResolver.ts new file mode 100644 index 0000000..19a6de0 --- /dev/null +++ b/bench/src/benchmark/BenchmarkResolver.ts @@ -0,0 +1,211 @@ +import type * as THREE from 'three/webgpu'; + +import { Upscaler } from '@pmndrs/upscaler'; +import { RCAS_LEGACY_SHADER, RCAS_SHADER } from '../../../src/shaders/rcas'; + +interface BenchmarkTimerBridge { + readonly enabled: boolean; + setNextFrameTag(frameTag: number): void; + setAuthoritative(authoritative: boolean): void; + waitForAvailableSlot(): Promise; + drain(): Promise; + reset(): void; + takeSamples(): Array<{ + frameTag: number; + sequence: number; + passes: Array<{ label: string; milliseconds: number }>; + }>; +} + +/** + * Adapts the unchanged production upscaler to the benchmark lifecycle. + */ +export class BaselineBenchmarkResolver implements BenchmarkResolver { + readonly metadata: BenchmarkVariantMetadata; + + private readonly _upscaler: Upscaler; + + constructor( + renderer: THREE.WebGPURenderer, + metadata: BenchmarkVariantMetadata, + rcasShader?: string, + candidateBundle?: string, + ) { + this.metadata = metadata; + const options = { + renderer, + _rcasShader: rcasShader, + _candidateBundle: candidateBundle, + }; + this._upscaler = new Upscaler(options); + this._upscaler.init(); + if (typeof metadata.settings.rcasDenoise === 'boolean') + this._upscaler.settings.rcasDenoise = metadata.settings.rcasDenoise; + } + + get outputTexture(): THREE.Texture { + return this._upscaler.outputTexture; + } + + get renderWidth(): number { + return this._upscaler.renderWidth; + } + + get renderHeight(): number { + return this._upscaler.renderHeight; + } + + get displayWidth(): number { + return this._upscaler.displayWidth; + } + + get displayHeight(): number { + return this._upscaler.displayHeight; + } + + get upscaleRatio(): number { + return this._upscaler.upscaleRatio; + } + + get jitterPhaseCount(): number { + return this._upscaler.jitterPhaseCount; + } + + get timestampQuerySupported(): boolean { + return this._timer.enabled; + } + + get unjitteredProjectionMatrix(): THREE.Matrix4 { + return this._upscaler.unjitteredProjectionMatrix; + } + + get settings(): Record { + return this._upscaler.settings as unknown as Record; + } + + get timings(): ReadonlyMap { + return this._upscaler.gpuTimings; + } + + private get _timer(): BenchmarkTimerBridge { + // The benchmark bridge deliberately stays private to the bench. Normal + // library users retain the existing no-op/latest-map timing behavior. + return (this._upscaler as unknown as { _timer: BenchmarkTimerBridge })._timer; + } + + configure(config: BenchmarkResolverConfigure): void { + this._upscaler.configure({ + displayWidth: config.displayWidth, + displayHeight: config.displayHeight, + customUpscaleRatio: config.ratio, + path: config.path, + }); + } + + beginFrame(camera: unknown): void { + this._upscaler.beginFrame(camera as THREE.PerspectiveCamera); + } + + endFrame(camera: unknown): void { + this._upscaler.endFrame(camera as THREE.PerspectiveCamera); + } + + dispatch(inputs: BenchmarkResolverDispatch, camera: unknown): void { + this._timer.setNextFrameTag(inputs.frameTag); + this._upscaler.dispatch( + { + color: inputs.color as THREE.Texture, + depth: inputs.depth as THREE.Texture | undefined, + velocity: inputs.velocity as THREE.Texture | undefined, + reactive: inputs.reactive as THREE.Texture | undefined, + transparencyAndComposition: inputs.transparencyAndComposition as + | THREE.Texture + | undefined, + preExposureTexture: inputs.preExposureTexture as THREE.Texture | undefined, + deltaTime: inputs.deltaTime, + }, + camera as THREE.PerspectiveCamera, + ); + } + + reset(): void { + this._upscaler.resetHistory(); + this._timer.reset(); + } + + resetTiming(): void { + this._timer.reset(); + } + + setAuthoritativeTiming(authoritative: boolean): void { + this._timer.setAuthoritative(authoritative); + } + + waitForTimingCapacity(): Promise { + return this._timer.waitForAvailableSlot(); + } + + drainTiming(): Promise { + return this._timer.drain(); + } + + takeTimingSamples(): BenchmarkGpuFrameSample[] { + return this._timer.takeSamples(); + } + + dispose(): void { + this._upscaler.dispose(); + } +} + +/** + * Creates the unchanged local baseline resolver. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata for the selected identity + * @returns One baseline resolver instance + */ +export function createBaselineResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + RCAS_LEGACY_SHADER, + ); +} + +/** + * Creates the isolated FSR 3.1.5 RCAS numeric candidate. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata for the candidate identity + * @returns One candidate resolver instance + */ +export function createRcasNumericParityResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + RCAS_SHADER, + ); +} + +/** + * Creates one of the cumulative source-style benchmark candidates. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata carrying the candidate bundle ID + * @returns One candidate resolver instance + */ +export function createSourceBundleResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + RCAS_SHADER, + metadata.id, + ); +} diff --git a/bench/src/benchmark/api.ts b/bench/src/benchmark/api.ts new file mode 100644 index 0000000..6e8abcc --- /dev/null +++ b/bench/src/benchmark/api.ts @@ -0,0 +1,235 @@ +import type * as THREE from 'three/webgpu'; + +import { DebugView } from '@pmndrs/upscaler'; + +import type { BenchPipeline } from '../BenchPipeline'; +import type { BenchScene } from '../BenchScene'; +import { BenchmarkClock } from './clock'; +import { BenchmarkCollector } from './collector'; + +interface BenchmarkApiContext { + renderer: THREE.WebGPURenderer; + camera: THREE.PerspectiveCamera; + bench: BenchScene; + pipeline: BenchPipeline | null; + config: BenchmarkRunConfig; + metadata: BenchmarkVariantMetadata; + scenario: BenchmarkScenarioDefinition; + environment: BenchmarkEnvironment; + manifestDigest: string; + validation: BenchmarkValidationRecord[]; + resize(dimensions: BenchmarkDimensions): void; +} + +const DEBUG_VIEWS: Record = { + final: DebugView.None, + 'motion-vectors': DebugView.MotionVectors, + disocclusion: DebugView.Disocclusion, + 'accumulation-age': DebugView.AccumulationAge, + locks: DebugView.Locks, + exposure: DebugView.Exposure, + 'shading-change': DebugView.ShadingChange, + reactivity: DebugView.Reactivity, +}; + +/** + * Exact deterministic browser API consumed by the CDP runner. + */ +class BrowserBenchmarkApi implements UpscalerBenchmarkApi { + readonly ready = true; + readonly config: BenchmarkRunConfig; + readonly metadata: BenchmarkVariantMetadata; + + private readonly _context: BenchmarkApiContext; + private readonly _clock = new BenchmarkClock(); + private readonly _captures: BenchmarkCaptureResult[] = []; + private _result: BenchmarkResult; + + constructor(context: BenchmarkApiContext) { + this._context = context; + this.config = context.config; + this.metadata = context.metadata; + this._result = { + status: context.scenario.unsupported ? 'unsupported' : 'ready', + experiment: 'E00', + manifestDigest: context.manifestDigest, + config: context.config, + variant: context.metadata, + scenario: context.scenario.id, + unsupported: context.scenario.unsupported, + environment: context.environment, + timing: null, + captures: this._captures, + validation: context.validation, + }; + } + + get result(): BenchmarkResult { + return this._result; + } + + async step(frame?: number): Promise { + this._requireSupported(); + const target = frame ?? this._clock.frame; + if (!Number.isInteger(target) || target < 0) + throw new Error(`Benchmark frame must be a non-negative integer: ${target}`); + if (target < this._clock.frame) await this.reset(); + while (this._clock.frame <= target) await this._renderOne(this._clock.frame); + return target; + } + + async reset(): Promise { + const { pipeline, bench, config, scenario, camera } = this._context; + if (pipeline) await pipeline.drainTiming(); + const canvas = this._context.renderer.domElement; + const resized = + canvas.width !== config.dimensions.width || canvas.height !== config.dimensions.height; + if (resized) this._context.resize(config.dimensions); + if (resized && pipeline?.usesEffectGraph) + await pipeline.prepareEffectReadiness(this._context.camera); + bench.resetDeterministicState(); + const frameZero = scenario.frame(0); + this._applyFrameState(frameZero); + pipeline?.reset(bench.scene, camera, 0); + this._clock.reset(); + } + + async capture(request: BenchmarkCaptureRequest): Promise { + this._requireSupported(); + const pipeline = this._context.pipeline!; + if (!this._context.scenario.debugViews.includes(request.debugView)) + throw new Error( + `Debug view ${request.debugView} is not declared for ${this._context.scenario.id}.`, + ); + pipeline.applySettings({ + sharpness: 0.8, + rcasDenoise: + pipeline.resolver.metadata.settings.rcasDenoise === true || + ['Q6', 'Q7', 'Q8'].includes(this._context.scenario.id), + maxAccumulation: 24, + exposure: 1, + autoExposure: true, + lockThinFeatures: true, + detectShadingChanges: true, + debugView: DEBUG_VIEWS[request.debugView], + }); + await this.reset(); + await this.step(request.frame); + const device = ( + this._context.renderer.backend as unknown as { device?: GPUDevice } + ).device; + if (!device) throw new Error('WebGPU device unavailable before capture.'); + await device.queue.onSubmittedWorkDone(); + + const capture = { + scenario: this._context.scenario.id, + subrun: this.config.subrun, + ratio: this.config.ratio, + frame: request.frame, + debugView: request.debugView, + width: this._context.renderer.domElement.width, + height: this._context.renderer.domElement.height, + jitterPeriod: pipeline.resolver.jitterPhaseCount, + }; + this._captures.push(capture); + return capture; + } + + async run(options: BenchmarkRunOptions = {}): Promise { + if (this._context.scenario.unsupported) return this._result; + const pipeline = this._context.pipeline!; + const warmupFrames = options.warmupFrames ?? this.config.warmupFrames; + const sampleFrames = options.sampleFrames ?? this.config.sampleFrames; + if (!Number.isInteger(warmupFrames) || warmupFrames < 0) + throw new Error(`Invalid warmup frame count: ${warmupFrames}`); + if (!Number.isInteger(sampleFrames) || sampleFrames < 1) + throw new Error(`Invalid sample frame count: ${sampleFrames}`); + + pipeline.resolver.setAuthoritativeTiming(this.config.authoritativeTiming); + if (this.config.authoritativeTiming && !pipeline.resolver.timestampQuerySupported) + throw new Error('Performance mode requires timestamp-query support.'); + + await this.reset(); + for (let index = 0; index < warmupFrames; index++) await this.step(); + await pipeline.drainTiming(); + pipeline.takeTimingSamples(); + pipeline.resolver.resetTiming(); + + const firstMeasuredFrame = this._clock.frame; + const expectedFrames = Array.from( + { length: sampleFrames }, + (_, index) => firstMeasuredFrame + index, + ); + const collector = new BenchmarkCollector( + expectedFrames, + this.metadata.pipeline.timingPassLabels, + ); + for (let index = 0; index < sampleFrames; index++) await this.step(); + await pipeline.drainTiming(); + collector.add(pipeline.takeTimingSamples()); + const timing = collector.summarize(); + this._result = { ...this._result, status: 'complete', timing }; + + if ( + this.config.authoritativeTiming && + (timing.invalidityCount !== 0 || + timing.receivedFrames !== sampleFrames) + ) { + this._result = { ...this._result, status: 'failed' }; + throw new Error( + `Invalid fresh timing set: expected ${sampleFrames}, received ${timing.receivedFrames}, ` + + `missing ${timing.missingFrameCount}, invalidities ${timing.invalidityCount}.`, + ); + } + return this._result; + } + + private _requireSupported(): void { + const unsupported = this._context.scenario.unsupported; + if (unsupported) + throw new Error(`${unsupported.code}: ${unsupported.capability}: ${unsupported.reason}`); + if (!this._context.pipeline) throw new Error('Benchmark resolver is unavailable.'); + } + + private async _renderOne(frame: number): Promise { + const { pipeline, scenario, camera, bench, config } = this._context; + if (!pipeline) return; + const scenarioFrame = scenario.frame(Math.min(frame, scenario.endFrame)); + if (scenarioFrame.resize) this._context.resize(scenarioFrame.resize); + this._applyFrameState(scenarioFrame); + if (scenarioFrame.resetHistory) pipeline.reset(bench.scene, camera, frame); + + await pipeline.prepareTiming(); + pipeline.advanceAutomatedFrame(frame); + pipeline.renderInput( + bench.scene, + camera, + scenarioFrame.particlesVisible ? bench.reactiveScene : undefined, + ); + pipeline.dispatchResolver(camera, config.timestepSeconds, frame); + pipeline.present(); + this._clock.seek(frame + 1); + } + + private _applyFrameState(frame: BenchmarkFrameState): void { + const { camera, bench } = this._context; + camera.position.fromArray(frame.cameraPosition); + camera.up.set(0, 1, 0); + camera.lookAt(...frame.cameraTarget); + camera.updateProjectionMatrix(); + camera.updateMatrixWorld(true); + bench.applyFrame(frame); + bench.scene.updateMatrixWorld(true); + bench.roomScene.updateMatrixWorld(true); + bench.reactiveScene.updateMatrixWorld(true); + } +} + +/** + * Creates the globally exposed deterministic benchmark API. + * @param context - Initialized browser benchmark dependencies + * @returns Exact CDP-facing API + */ +export function createBenchmarkApi(context: BenchmarkApiContext): UpscalerBenchmarkApi { + return new BrowserBenchmarkApi(context); +} diff --git a/bench/src/benchmark/clock.ts b/bench/src/benchmark/clock.ts new file mode 100644 index 0000000..4f0d1b8 --- /dev/null +++ b/bench/src/benchmark/clock.ts @@ -0,0 +1,34 @@ +/** + * Integer-frame clock used by every automated E00 scenario. + */ +export class BenchmarkClock { + private _frame = 0; + + get frame(): number { + return this._frame; + } + + get time(): number { + return this._frame / 60; + } + + /** Returns the current frame and advances exactly once. */ + step(): number { + return this._frame++; + } + + /** Restores recorded frame zero without consulting wall time. */ + reset(): void { + this._frame = 0; + } + + /** + * Positions the clock at an exact integer frame. + * @param frame - Zero-based frame index + */ + seek(frame: number): void { + if (!Number.isInteger(frame) || frame < 0) + throw new Error(`Benchmark frame must be a non-negative integer: ${frame}`); + this._frame = frame; + } +} diff --git a/bench/src/benchmark/collector.ts b/bench/src/benchmark/collector.ts new file mode 100644 index 0000000..c42e9ff --- /dev/null +++ b/bench/src/benchmark/collector.ts @@ -0,0 +1,236 @@ +function percentile(values: readonly number[], quantile: number): number | null { + if (values.length === 0) return null; + const sorted = [...values].sort((a, b) => a - b); + const position = quantile * (sorted.length - 1); + const lower = Math.floor(position); + const upper = Math.ceil(position); + if (lower === upper) return sorted[lower]; + return sorted[lower] + (sorted[upper] - sorted[lower]) * (position - lower); +} + +function summarizePass( + label: string, + samples: number[], + expectedFrames: number, +): BenchmarkPassSummary { + return { + label, + samples, + median: percentile(samples, 0.5), + p95: percentile(samples, 0.95), + missingCount: expectedFrames - samples.length, + }; +} + +/** + * Collects complete fresh frame samples and derives manifest statistics. + */ +export class BenchmarkCollector { + private readonly _expectedFrameTags: readonly number[]; + private readonly _expectedPassLabels: readonly string[] | null; + private readonly _samples: BenchmarkGpuFrameSample[] = []; + + constructor(expectedFrameTags: readonly number[], expectedPassLabels?: readonly string[]) { + this._expectedFrameTags = [...expectedFrameTags]; + this._expectedPassLabels = expectedPassLabels + ? [...expectedPassLabels].sort() + : null; + } + + /** + * Adds fresh samples returned by the resolver timer. + * @param samples - Complete frame-tagged samples + */ + add(samples: readonly BenchmarkGpuFrameSample[]): void { + this._samples.push( + ...samples.map((sample) => ({ + ...sample, + passes: sample.passes.map((pass) => ({ ...pass })), + })), + ); + } + + /** Builds raw, missing, per-pass, and compute-sum statistics. */ + summarize(): BenchmarkTimingSummary { + const expected = new Set(this._expectedFrameTags); + const candidates = new Map(); + const invalidSamples: BenchmarkInvalidTimingSample[] = []; + const sequences = new Set(); + let duplicateFrameCount = 0; + let duplicateSequenceCount = 0; + let unexpectedFrameCount = 0; + let duplicatePassLabelCount = 0; + let invalidValueCount = 0; + + for (const sample of this._samples) { + if (sequences.has(sample.sequence)) { + duplicateSequenceCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-sequence', + detail: `Sequence ${sample.sequence} was received more than once.`, + }); + continue; + } + sequences.add(sample.sequence); + if (!expected.has(sample.frameTag)) { + unexpectedFrameCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'unexpected-frame', + detail: `Frame ${sample.frameTag} was not requested.`, + }); + continue; + } + if (candidates.has(sample.frameTag)) { + duplicateFrameCount++; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-frame', + detail: `Frame ${sample.frameTag} was received more than once.`, + }); + continue; + } + + const labels = new Set(); + let structurallyValid = true; + for (const pass of sample.passes) { + if (labels.has(pass.label)) { + duplicatePassLabelCount++; + structurallyValid = false; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'duplicate-pass-label', + detail: `Pass label ${pass.label} occurs more than once.`, + }); + } + labels.add(pass.label); + if (!Number.isFinite(pass.milliseconds) || pass.milliseconds < 0) { + invalidValueCount++; + structurallyValid = false; + invalidSamples.push({ + frameTag: sample.frameTag, + sequence: sample.sequence, + reason: 'invalid-pass-value', + detail: `${pass.label} has invalid duration ${pass.milliseconds}.`, + }); + } + } + if (structurallyValid) candidates.set(sample.frameTag, sample); + } + + // The modal sorted label signature is the complete measured graph. This + // avoids allowing one malformed first/last frame to define completeness. + const signatureCounts = new Map(); + for (const sample of candidates.values()) { + const signature = sample.passes.map((pass) => pass.label).sort().join('\u0000'); + signatureCounts.set(signature, (signatureCounts.get(signature) ?? 0) + 1); + } + const expectedSignature = this._expectedPassLabels + ? this._expectedPassLabels.join('\u0000') + : ([...signatureCounts].sort( + (a, b) => b[1] - a[1] || a[0].localeCompare(b[0]), + )[0]?.[0] ?? ''); + const expectedPassLabels = this._expectedPassLabels + ? [...this._expectedPassLabels] + : expectedSignature + ? expectedSignature.split('\u0000') + : []; + const byFrame = new Map(); + let inconsistentPassSetCount = 0; + for (const [frameTag, sample] of candidates) { + const signature = sample.passes.map((pass) => pass.label).sort().join('\u0000'); + if (signature !== expectedSignature || expectedPassLabels.length === 0) { + inconsistentPassSetCount++; + invalidSamples.push({ + frameTag, + sequence: sample.sequence, + reason: 'inconsistent-pass-set', + detail: `Expected [${expectedPassLabels.join(', ')}], received [${sample.passes + .map((pass) => pass.label) + .sort() + .join(', ')}].`, + }); + continue; + } + byFrame.set(frameTag, sample); + } + + const passes = expectedPassLabels.map((label) => { + const values: number[] = []; + for (const frameTag of this._expectedFrameTags) { + const value = byFrame + .get(frameTag) + ?.passes.find((pass) => pass.label === label)?.milliseconds; + if (value !== undefined) values.push(value); + } + return summarizePass(label, values, this._expectedFrameTags.length); + }); + + const computeSums: number[] = []; + for (const frameTag of this._expectedFrameTags) { + const sample = byFrame.get(frameTag); + if (sample) computeSums.push( + sample.passes.reduce((sum, pass) => sum + pass.milliseconds, 0), + ); + } + + return { + expectedFrames: this._expectedFrameTags.length, + receivedFrames: byFrame.size, + missingFrameCount: this._expectedFrameTags.length - byFrame.size, + duplicateFrameCount, + duplicateSequenceCount, + unexpectedFrameCount, + duplicatePassLabelCount, + invalidValueCount, + inconsistentPassSetCount, + invalidityCount: + duplicateFrameCount + + duplicateSequenceCount + + unexpectedFrameCount + + duplicatePassLabelCount + + invalidValueCount + + inconsistentPassSetCount + + (this._expectedFrameTags.length - byFrame.size), + expectedPassLabels, + invalidSamples, + passes, + computeSum: summarizePass( + 'compute-sum', + computeSums, + this._expectedFrameTags.length, + ), + raw: [...byFrame.values()].sort((a, b) => a.frameTag - b.frameTag), + }; + } +} + +/** + * Serializes a timing result without dropping raw evidence. + * @param summary - Collector summary + * @returns Pretty JSON + */ +export function timingSummaryToJson(summary: BenchmarkTimingSummary): string { + return JSON.stringify(summary, null, 2); +} + +/** + * Serializes one row per fresh frame/pass plus compute sum. + * @param summary - Collector summary + * @returns RFC-4180-compatible CSV text + */ +export function timingSummaryToCsv(summary: BenchmarkTimingSummary): string { + const rows = ['frame,sequence,label,milliseconds']; + for (const sample of summary.raw) { + for (const pass of sample.passes) + rows.push(`${sample.frameTag},${sample.sequence},${pass.label},${pass.milliseconds}`); + const sum = sample.passes.reduce((total, pass) => total + pass.milliseconds, 0); + rows.push(`${sample.frameTag},${sample.sequence},compute-sum,${sum}`); + } + return `${rows.join('\n')}\n`; +} diff --git a/bench/src/benchmark/config.ts b/bench/src/benchmark/config.ts new file mode 100644 index 0000000..1978e2c --- /dev/null +++ b/bench/src/benchmark/config.ts @@ -0,0 +1,89 @@ +const RATIOS = [1, 1.5, 2, 3] as const; +const VARIANTS = [ + 'baseline', + 'local-baseline-5d6a65e', + 'local-baseline-through-e00-harness', + 'rcas-fsr315-limiter', + 'rcas-fsr315-numeric', + 'source-filter-bundle-v1', + 'source-structural-bundle-v1', + 'source-spd-resolver-bundle-v1', +] as const; +const SCENARIOS = ['Q0', 'Q1', 'Q2', 'Q3', 'Q4', 'Q5', 'Q6', 'Q7', 'Q8', 'Q9', 'Q10'] as const; + +function numberParam(params: URLSearchParams, name: string, fallback: number): number { + const raw = params.get(name); + if (raw === null) return fallback; + const value = Number(raw); + if (!Number.isFinite(value)) throw new Error(`Invalid benchmark ${name}: ${raw}`); + return value; +} + +function enumParam( + params: URLSearchParams, + name: string, + values: readonly T[], + fallback: T, +): T { + const raw = params.get(name); + if (raw === null) return fallback; + if (!values.includes(raw as T)) throw new Error(`Invalid benchmark ${name}: ${raw}`); + return raw as T; +} + +/** + * Parses the deterministic E00 browser configuration. + * @param search - URL query string, including or excluding the leading `?` + * @returns A validated benchmark configuration + */ +export function parseBenchmarkConfig(search = window.location.search): BenchmarkRunConfig { + const params = new URLSearchParams(search); + const mode = enumParam( + params, + 'benchMode', + ['interactive', 'performance', 'capture'] as const, + 'interactive', + ); + const ratio = numberParam(params, 'ratio', 2); + if (!RATIOS.includes(ratio as (typeof RATIOS)[number])) + throw new Error(`Unsupported benchmark ratio: ${ratio}`); + + const width = Math.floor(numberParam(params, 'width', mode === 'interactive' ? window.innerWidth : 1920)); + const height = Math.floor( + numberParam(params, 'height', mode === 'interactive' ? window.innerHeight : 1080), + ); + if (width < 1 || height < 1) throw new Error('Benchmark dimensions must be positive integers.'); + + const variant = enumParam(params, 'variant', VARIANTS, 'baseline'); + const comparison = enumParam(params, 'comparison', VARIANTS, 'baseline'); + const scenario = enumParam(params, 'scenario', SCENARIOS, 'Q1'); + const experiment = params.get('experiment') ?? 'E00'; + if (experiment !== 'E00') throw new Error(`Unsupported benchmark experiment: ${experiment}`); + + const warmupFrames = Math.floor(numberParam(params, 'warmup', 240)); + const sampleFrames = Math.floor(numberParam(params, 'samples', 600)); + if (warmupFrames < 0 || sampleFrames < 1) + throw new Error('Benchmark warmup must be non-negative and samples must be positive.'); + + return { + experiment: 'E00', + mode, + variant, + comparison, + scenario, + subrun: params.get('subrun'), + ratio, + dimensions: { + width, + height, + devicePixelRatio: mode === 'interactive' ? Math.min(window.devicePixelRatio, 2) : 1, + }, + timestepSeconds: 1 / 60, + warmupFrames, + sampleFrames, + authoritativeTiming: mode === 'performance', + }; +} + +/** Ratios accepted by the immutable E00 manifest. */ +export const BENCHMARK_RATIOS: readonly number[] = RATIOS; diff --git a/bench/src/benchmark/environment.ts b/bench/src/benchmark/environment.ts new file mode 100644 index 0000000..07e3979 --- /dev/null +++ b/bench/src/benchmark/environment.ts @@ -0,0 +1,78 @@ +import * as THREE from 'three/webgpu'; + +interface RendererBackendDetails { + isWebGPUBackend?: boolean; + device?: GPUDevice; + adapter?: GPUAdapter; +} + +/** Computes the immutable E00 manifest SHA-256 digest in the browser. */ +export async function getManifestDigest(): Promise { + const response = await fetch('/results/experiments/e00-harness.json', { cache: 'no-store' }); + if (!response.ok) throw new Error(`Unable to read E00 manifest: ${response.status}`); + const bytes = await response.arrayBuffer(); + const digest = await crypto.subtle.digest('SHA-256', bytes); + return [...new Uint8Array(digest)].map((byte) => byte.toString(16).padStart(2, '0')).join(''); +} + +/** + * Captures browser, adapter, WebGPU, and fixed-run metadata. + * @param renderer - Initialized renderer + * @param config - Validated E00 configuration + * @returns Environment metadata required by benchmark artifacts + */ +export function collectBenchmarkEnvironment( + renderer: THREE.WebGPURenderer, + config: BenchmarkRunConfig, +): Promise { + const backend = renderer.backend as RendererBackendDetails; + const device = backend.device; + return navigator.gpu + .requestAdapter() + .then((adapter) => { + const adapterInfo = backend.adapter?.info ?? adapter?.info; + const adapterName = adapterInfo + ? [ + adapterInfo.vendor, + adapterInfo.architecture, + adapterInfo.device, + adapterInfo.description, + ] + .filter(Boolean) + .join(' ') + : 'adapter-info-unavailable'; + return { + browser: navigator.userAgent, + operatingSystem: navigator.platform, + adapter: adapterName, + backend: backend.isWebGPUBackend === true ? 'WebGPU' : 'unknown', + webgpuFeatures: device ? [...device.features].sort() : [], + threeVersion: THREE.REVISION, + dimensions: { ...config.dimensions }, + ratio: config.ratio, + fixedTimestep: config.timestepSeconds, + }; + }); +} + +/** + * Reports asynchronous device loss through the validation log channel. + * @param renderer - Initialized WebGPU renderer + * @param records - Mutable benchmark validation record list + */ +export function monitorDeviceLoss( + renderer: THREE.WebGPURenderer, + records: BenchmarkValidationRecord[], +): void { + const backend = renderer.backend as RendererBackendDetails; + void backend.device?.lost.then((info) => { + const record = { + channel: 'GPUDevice.lost', + level: 'error', + text: `${info.reason}: ${info.message}`, + timestamp: performance.now(), + }; + records.push(record); + console.error('WebGPU device lost.', record); + }); +} diff --git a/bench/src/benchmark/scenarios.ts b/bench/src/benchmark/scenarios.ts new file mode 100644 index 0000000..7ef47f3 --- /dev/null +++ b/bench/src/benchmark/scenarios.ts @@ -0,0 +1,320 @@ +const BASE_POSITION = [9, 6, 12] as const; +const BASE_TARGET = [0, 1.6, 0] as const; +const ROOM_TARGET = [0, 3, -5] as const; + +function state( + frame: number, + cameraPosition: readonly [number, number, number] = BASE_POSITION, + cameraTarget: readonly [number, number, number] = BASE_TARGET, +): BenchmarkFrameState { + return { + frame, + time: frame / 60, + cameraPosition, + cameraTarget, + sceneTime: frame / 60, + animateScene: false, + directionalIntensity: 3.2, + resetHistory: false, + resize: null, + particlesVisible: false, + }; +} + +function baselineAnimated(frame: number): BenchmarkFrameState { + return { ...state(frame), animateScene: true }; +} + +function q2(frame: number): BenchmarkFrameState { + const u = frame / 239; + return state(frame, [9 - 3 * u, 6, 12 - 4 * u]); +} + +function q4(frame: number): BenchmarkFrameState { + const target = BASE_TARGET; + const radius = 15; + const height = 4.4; + const theta0 = Math.atan2(12, 9); + const theta119 = theta0 + 0.25; + const c119 = [ + target[0] + radius * Math.cos(theta119), + target[1] + height, + target[2] + radius * Math.sin(theta119), + ] as const; + + if (frame <= 119) { + const theta = theta0 + (0.25 * frame) / 119; + return state(frame, [ + target[0] + radius * Math.cos(theta), + target[1] + height, + target[2] + radius * Math.sin(theta), + ]); + } + + const c239 = [c119[0] - 8, c119[1], c119[2] - 6] as const; + if (frame <= 239) { + const u = (frame - 120) / 119; + return state(frame, [c119[0] - 8 * u, c119[1], c119[2] - 6 * u]); + } + + const startTheta = Math.atan2(c239[2] - target[2], c239[0] - target[0]); + const orbitRadius = Math.hypot(c239[0] - target[0], c239[2] - target[2]); + const orbitFrame = Math.min(frame, 359); + const theta = startTheta + (0.9 * (orbitFrame - 240)) / 119; + return state(frame, [ + target[0] + orbitRadius * Math.cos(theta), + c239[1], + target[2] + orbitRadius * Math.sin(theta), + ]); +} + +function roomMotion(frame: number): BenchmarkFrameState { + const time = frame / 60; + return state(frame, [7 * Math.sin(0.15 * time), 4, 9 + 1.5 * Math.cos(0.15 * time)], ROOM_TARGET); +} + +function q9(frame: number): BenchmarkFrameState { + let directionalIntensity = 3.2; + if (frame >= 60 && frame < 120) directionalIntensity = 8; + else if (frame >= 120 && frame < 180) + directionalIntensity = 8 - (6 * (frame - 120)) / 59; + return { ...state(frame), directionalIntensity }; +} + +function q10(frame: number): BenchmarkFrameState { + const afterCut = frame >= 60; + return { + ...baselineAnimated(frame), + cameraPosition: afterCut ? [-7, 4, 9] : BASE_POSITION, + resetHistory: frame === 60 || frame === 120 || frame === 180, + resize: + frame === 120 + ? { width: 1280, height: 720, devicePixelRatio: 1 } + : frame === 180 + ? { width: 1920, height: 1080, devicePixelRatio: 1 } + : null, + particlesVisible: false, + }; +} + +const SCENARIOS: Record = { + Q0: { + id: 'Q0', + name: 'input-debug-validation', + endFrame: 143, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '119'], + debugViews: [ + 'final', + 'motion-vectors', + 'disocclusion', + 'accumulation-age', + 'locks', + 'exposure', + 'shading-change', + 'reactivity', + ], + rois: { + full: [0, 0, 1, 1], + floor_grid: [0.05, 0.55, 0.9, 0.45], + fence_and_spheres: [0.05, 0.28, 0.9, 0.42], + }, + subruns: [], + unsupported: null, + frame: baselineAnimated, + }, + Q1: { + id: 'Q1', + name: 'static-convergence', + endFrame: 239, + captures: ['0', '1', '2', '4', '8', '16', '23', 'P-1', 'P', '2*P-1', '119', '239'], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { full: [0, 0, 1, 1], thin_features: [0.08, 0.3, 0.84, 0.55] }, + subruns: [], + unsupported: null, + frame: state, + }, + Q2: { + id: 'Q2', + name: 'slow-aliasing-dolly', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age', 'locks'], + rois: { floor_grid: [0, 0.48, 1, 0.52], fence: [0.05, 0.35, 0.9, 0.32] }, + subruns: [], + unsupported: null, + frame: q2, + }, + Q3: { + id: 'Q3', + name: 'object-motion-disocclusion', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + moving_spheres: [0.12, 0.2, 0.76, 0.58], + fence_silhouette: [0.05, 0.35, 0.9, 0.3], + }, + subruns: [], + unsupported: null, + frame: baselineAnimated, + }, + Q4: { + id: 'Q4', + name: 'camera-motion-hold', + endFrame: 479, + captures: [ + '0', '23', 'P-1', 'P', '2*P-1', '118', '119', '120', '121', '238', '239', + '240', '241', '358', '359', '360', '361', '383', '479', + ], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age', 'shading-change'], + rois: { full: [0, 0, 1, 1], thin_geometry: [0.05, 0.25, 0.9, 0.55] }, + subruns: [], + unsupported: null, + frame: q4, + }, + Q5: { + id: 'Q5', + name: 'seeded-transparency-reactivity', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'accumulation-age', 'locks', 'reactivity'], + rois: { + full: [0, 0, 1, 1], + particle_volume: [0.18, 0.1, 0.64, 0.72], + opaque_edges: [0.05, 0.35, 0.9, 0.45], + }, + subruns: [], + unsupported: null, + frame: (frame) => ({ ...state(frame), particlesVisible: true }), + }, + Q6: { + id: 'Q6', + name: 'isolated-screenspace-effects', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + floor_reflection: [0.05, 0.5, 0.9, 0.5], + wall_contact_and_bounce: [0.08, 0.08, 0.84, 0.6], + }, + subruns: ['gtao', 'ssr', 'ssgi'], + unsupported: null, + frame: (frame) => state(frame, [0, 4, 10], ROOM_TARGET), + }, + Q7: { + id: 'Q7', + name: 'in-graph-screenspace-composition', + endFrame: 239, + captures: ['0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '119', '179', '239'], + debugViews: ['final', 'motion-vectors', 'disocclusion', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + glossy_floor: [0.05, 0.48, 0.9, 0.52], + colored_walls: [0.05, 0.05, 0.9, 0.62], + }, + subruns: [], + unsupported: null, + frame: roomMotion, + }, + Q8: { + id: 'Q8', + name: 'recurrent-denoiser-characterization', + endFrame: 239, + captures: ['0', '1', '2', '4', '8', '16', '23', 'P-1', 'P', '2*P-1', '59', '119', '239'], + debugViews: ['final', 'motion-vectors', 'accumulation-age'], + rois: { + full: [0, 0, 1, 1], + flat_walls: [0.08, 0.08, 0.84, 0.52], + occlusion_edges: [0.18, 0.24, 0.64, 0.54], + }, + subruns: ['builtin', 'spatial', 'recurrent'], + unsupported: null, + frame: roomMotion, + }, + Q9: { + id: 'Q9', + name: 'exposure-transition', + endFrame: 239, + captures: [ + '0', '23', 'P-1', 'P', '2*P-1', '59', '60', '61', '62', '64', '68', '76', + '83', '119', '120', '121', '149', '178', '179', '180', '181', '182', '184', + '188', '196', '203', '239', + ], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { + full: [0, 0, 1, 1], + lit_knots: [0.18, 0.16, 0.64, 0.38], + hdr_bulb: [0.43, 0.08, 0.14, 0.2], + }, + subruns: [], + unsupported: null, + frame: q9, + }, + Q10: { + id: 'Q10', + name: 'reset-cut-resize', + endFrame: 239, + captures: [ + '0', '1', '2', '23', 'P-1', 'P', '2*P-1', '59', '60', '61', '62', '64', + '68', '76', '83', '119', '120', '121', '122', '124', '128', '136', '143', + '179', '180', '181', '182', '184', '188', '196', '203', '239', + ], + debugViews: [ + 'final', + 'motion-vectors', + 'disocclusion', + 'accumulation-age', + 'locks', + 'exposure', + 'shading-change', + ], + rois: { full: [0, 0, 1, 1], moving_silhouettes: [0.08, 0.18, 0.84, 0.58] }, + subruns: [], + unsupported: null, + frame: q10, + }, +}; + +/** + * Returns a manifest-defined scenario and validates its subrun. + * @param id - Scenario ID + * @param subrun - Optional effect subrun + * @returns Immutable scenario contract + */ +export function getBenchmarkScenario( + id: BenchmarkScenarioId, + subrun: string | null = null, +): BenchmarkScenarioDefinition { + const scenario = SCENARIOS[id]; + if (scenario.subruns.length > 0 && (!subrun || !scenario.subruns.includes(subrun))) + throw new Error(`Scenario ${id} requires subrun: ${scenario.subruns.join(', ')}`); + if (scenario.subruns.length === 0 && subrun) + throw new Error(`Scenario ${id} does not define subruns.`); + return scenario; +} + +/** + * Resolves manifest frame expressions against a jitter period. + * @param expressions - Integer or `P` expressions from the scenario contract + * @param jitterPeriod - Active resolver jitter phase count + * @returns Ordered, unique, integer capture frames + */ +export function resolveCaptureFrames( + expressions: readonly string[], + jitterPeriod: number, +): number[] { + const values = expressions.map((expression) => { + if (/^\d+$/.test(expression)) return Number(expression); + if (expression === 'P') return jitterPeriod; + if (expression === 'P-1') return jitterPeriod - 1; + if (expression === '2*P-1') return 2 * jitterPeriod - 1; + throw new Error(`Unsupported capture frame expression: ${expression}`); + }); + return [...new Set(values)]; +} + +/** All immutable E00 scenario contracts. */ +export const BENCHMARK_SCENARIOS: Readonly> = + SCENARIOS; diff --git a/bench/src/benchmark/variants.ts b/bench/src/benchmark/variants.ts new file mode 100644 index 0000000..e76cee2 --- /dev/null +++ b/bench/src/benchmark/variants.ts @@ -0,0 +1,257 @@ +import { + createBaselineResolver, + createRcasNumericParityResolver, + createSourceBundleResolver, +} from './BenchmarkResolver'; + +const SUPPORTED_RATIOS = [1, 1.5, 2, 3] as const; +const RESOURCE_GRAPH = [ + 'scene-color-depth-velocity', + 'exposure', + 'reconstruct', + 'accumulate-history-locks', + 'rcas-or-blit', + 'debug-or-output', +] as const; +const ASSEMBLED_CHUNKS = [ + 'constants', + 'color', + 'depth', + 'tonemap', + 'pass-body', +] as const; + +let activeResolverCount = 0; + +function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { + const sourceBundle = + id === 'source-filter-bundle-v1' || + id === 'source-structural-bundle-v1' || + id === 'source-spd-resolver-bundle-v1'; + const structural = + id === 'source-structural-bundle-v1' || id === 'source-spd-resolver-bundle-v1'; + const spdResolver = id === 'source-spd-resolver-bundle-v1'; + const rcasLimiterParity = id === 'rcas-fsr315-limiter'; + const rcasNumericParity = rcasLimiterParity || id === 'rcas-fsr315-numeric'; + const rcasDenoise = id === 'rcas-fsr315-numeric'; + const sourceResourceGraph = spdResolver + ? [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth-farthest-luma', + 'luma-spd-frame-info', + 'depth-clip-motion-divergence', + 'signed-difference-shading-spd', + 'prepare-reactivity-accumulation-new-locks', + 'four-frame-luma-instability', + 'source-resolver-history-lock-alpha', + 'rcas-or-blit', + 'debug-or-output', + ] + : structural + ? [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth-farthest-luma', + 'exposure-history-frame-info', + 'depth-clip-motion-divergence', + 'prepare-reactivity-accumulation-new-locks', + 'source-filter-local-state', + 'rcas-or-blit', + 'debug-or-output', + ] + : [ + 'scene-color-depth-velocity', + 'prepare-inputs-atomic-depth', + 'exposure-history-frame-info', + 'depth-clip', + 'source-current-history-filters', + 'rcas-or-blit', + 'debug-or-output', + ]; + const sourceTimingLabels = spdResolver + ? [ + 'prepareInputs', + 'lumaSpd', + 'depthClip', + 'shadingSpd', + 'shadingResolve', + 'prepareReactivity', + 'lumaInstability', + 'accumulate', + 'rcas', + ] + : structural + ? [ + 'prepareInputs', + 'exposure', + 'depthClip', + 'prepareReactivity', + 'accumulate', + 'rcas', + ] + : ['prepareInputs', 'exposure', 'depthClip', 'accumulate', 'rcas']; + return { + id, + name: spdResolver + ? 'Source SPD temporal resolver bundle v1' + : structural + ? 'Source structural inputs/reactivity bundle v1' + : id === 'source-filter-bundle-v1' + ? 'Source reconstruction/filter bundle v1' + : rcasDenoise + ? 'FSR 3.1.5 RCAS limiter + denoise' + : rcasLimiterParity + ? 'FSR 3.1.5 RCAS lower limiter' + : id === 'local-baseline-through-e00-harness' + ? 'Local baseline through E00 harness' + : 'Local baseline 5d6a65e', + supportedRatios: SUPPORTED_RATIOS, + settings: { + path: 'temporal', + sharpness: 0.8, + rcasDenoise, + maxAccumulation: 24, + exposure: 1, + autoExposure: true, + lockThinFeatures: true, + detectShadingChanges: true, + }, + resourceGraph: sourceBundle ? sourceResourceGraph : RESOURCE_GRAPH, + pipeline: { + shaderKey: sourceBundle + ? id + : rcasNumericParity + ? rcasDenoise + ? 'rcas-fsr315-numeric' + : 'rcas-fsr315-limiter' + : 'local-baseline-5d6a65e', + pipelineKey: sourceBundle ? id : 'temporal-baseline', + assembledChunks: sourceBundle + ? [ + 'constants', + 'source-inputs', + 'source-filter', + ...(structural ? ['source-reactivity'] : []), + ...(spdResolver ? ['source-spd', 'source-resolver-state'] : []), + 'rcas-source-math', + ] + : ASSEMBLED_CHUNKS, + wgslOverrides: sourceBundle + ? { + motionInputAtDisplayResolution: false, + motionCancelJitter: false, + prepareStructuralSignals: structural, + depthClipMotionDivergence: structural, + reactiveUseComponentMax: true, + reactiveApplyThreshold: true, + reactiveBinary: false, + reactiveThreshold: 0.04, + reactiveScale: 2, + reactiveBinaryValue: 1, + } + : {}, + timingPassLabels: sourceBundle + ? sourceTimingLabels + : ['exposure', 'reconstruct', 'accumulate', 'rcas'], + }, + }; +} + +const DEFAULT_DEFINITIONS: BenchmarkVariantDefinition[] = [ + 'baseline', + 'local-baseline-5d6a65e', + 'local-baseline-through-e00-harness', +].map((id) => ({ + metadata: metadata(id as BenchmarkVariantId), + create: createBaselineResolver, +})); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-fsr315-limiter'), + create: createRcasNumericParityResolver, +}); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-fsr315-numeric'), + create: createRcasNumericParityResolver, +}); +for (const id of [ + 'source-filter-bundle-v1', + 'source-structural-bundle-v1', + 'source-spd-resolver-bundle-v1', +] as const) { + DEFAULT_DEFINITIONS.push({ + metadata: metadata(id), + create: createSourceBundleResolver, + }); +} + +/** + * Registry enforcing one active resolver across the page. + */ +export class SingleVariantRegistry { + private readonly _definitions = new Map(); + private _active: BenchmarkResolver | null = null; + + constructor(definitions: readonly BenchmarkVariantDefinition[] = DEFAULT_DEFINITIONS) { + for (const definition of definitions) { + if (this._definitions.has(definition.metadata.id)) + throw new Error(`Duplicate benchmark variant: ${definition.metadata.id}`); + this._definitions.set(definition.metadata.id, definition); + } + } + + /** + * Validates a variant without constructing its GPU graph. + * @param id - Requested variant identity + * @param ratio - Requested display/render ratio + * @returns Immutable variant metadata + */ + resolve(id: string, ratio: number): BenchmarkVariantMetadata { + const definition = this._definitions.get(id); + if (!definition) throw new Error(`Unknown benchmark variant: ${id}`); + if (!definition.metadata.supportedRatios.includes(ratio)) + throw new Error(`Variant ${id} does not support ratio ${ratio}.`); + return definition.metadata; + } + + /** + * Creates the page's sole active resolver. + * @param id - Requested variant identity + * @param ratio - Requested display/render ratio + * @param renderer - Initialized renderer passed to the resolver factory + * @returns The active resolver + */ + create(id: string, ratio: number, renderer: unknown): BenchmarkResolver { + const variantMetadata = this.resolve(id, ratio); + if (this._active || activeResolverCount !== 0) + throw new Error('A benchmark resolver is already active.'); + + const definition = this._definitions.get(id)!; + const resolver = definition.create(renderer, variantMetadata); + this._active = resolver; + activeResolverCount++; + return resolver; + } + + /** Disposes and releases the sole resolver lease. */ + disposeActive(): void { + if (!this._active) return; + this._active.dispose(); + this._active = null; + activeResolverCount--; + } + + /** + * Releases a resolver already disposed by its owning pipeline. + * @param resolver - Resolver whose GPU resources were explicitly disposed + */ + releaseDisposed(resolver: BenchmarkResolver): void { + if (this._active !== resolver) + throw new Error('Cannot release a resolver that is not the active registry instance.'); + this._active = null; + activeResolverCount--; + } +} + +/** Returns the number of active resolvers without requiring a GPU. */ +export function getActiveResolverCount(): number { + return activeResolverCount; +} diff --git a/bench/src/main.ts b/bench/src/main.ts index a507fa5..519ce96 100644 --- a/bench/src/main.ts +++ b/bench/src/main.ts @@ -1,11 +1,20 @@ -import * as THREE from 'three/webgpu'; import { OrbitControls } from 'three/addons/controls/OrbitControls.js'; +import * as THREE from 'three/webgpu'; import { DebugView, QualityMode } from '@pmndrs/upscaler'; import { BenchPipeline } from './BenchPipeline'; import { createBenchScene } from './BenchScene'; import { createBenchUI, type BenchState } from './BenchUI'; +import { createBenchmarkApi } from './benchmark/api'; +import { parseBenchmarkConfig } from './benchmark/config'; +import { + collectBenchmarkEnvironment, + getManifestDigest, + monitorDeviceLoss, +} from './benchmark/environment'; +import { getBenchmarkScenario } from './benchmark/scenarios'; +import { SingleVariantRegistry } from './benchmark/variants'; //* WebGPU Guard const fatal = document.getElementById('fatal')!; @@ -16,7 +25,14 @@ if (!navigator.gpu) { throw new Error('WebGPU unavailable'); } -//* State +//* Deterministic Configuration ============================================== +const config = parseBenchmarkConfig(); +const automated = config.mode !== 'interactive'; +const scenario = getBenchmarkScenario(config.scenario, config.subrun); +const registry = new SingleVariantRegistry(); +// Validation occurs before renderer construction, so bad identities never render. +const variantMetadata = registry.resolve(config.variant, config.ratio); + const state: BenchState = { mode: 'upscale-temporal', quality: QualityMode.Performance, @@ -32,65 +48,126 @@ const state: BenchState = { autoOrbit: true, }; -// Cap DPR — the point of an upscaler bench is control over pixel counts. -const dpr = Math.min(window.devicePixelRatio, 2); +const dpr = config.dimensions.devicePixelRatio; const displaySize = () => ({ - width: Math.floor(window.innerWidth * dpr), - height: Math.floor(window.innerHeight * dpr), + width: automated ? config.dimensions.width : Math.floor(window.innerWidth * dpr), + height: automated ? config.dimensions.height : Math.floor(window.innerHeight * dpr), }); //* Renderer const renderer = new THREE.WebGPURenderer({ antialias: false }); renderer.setPixelRatio(dpr); -renderer.setSize(window.innerWidth, window.innerHeight); -// The FSR output pass already applies tonemapping + sRGB encoding in WGSL, so -// the presentation quad must go to the canvas untouched. -renderer.toneMapping = THREE.NoToneMapping; -renderer.outputColorSpace = THREE.LinearSRGBColorSpace; +renderer.setSize( + automated ? config.dimensions.width / dpr : window.innerWidth, + automated ? config.dimensions.height / dpr : window.innerHeight, + true, +); +// The upscaler stays linear/HDR; presentation belongs to the renderer. +renderer.toneMapping = THREE.ACESFilmicToneMapping; +renderer.outputColorSpace = THREE.SRGBColorSpace; document.body.appendChild(renderer.domElement); await renderer.init(); -const backend = renderer.backend as { isWebGPUBackend?: boolean }; -if (backend.isWebGPUBackend !== true) { +const backend = renderer.backend as { isWebGPUBackend?: boolean; device?: GPUDevice }; +if (backend.isWebGPUBackend !== true || !backend.device) { fatal.style.display = 'grid'; - fatal.textContent = 'three fell back to the WebGL backend — the FSR3 bench needs real WebGPU.'; - throw new Error('WebGL fallback active'); + fatal.textContent = 'three fell back to the WebGL backend — the bench needs real WebGPU.'; + throw new Error('WebGPU backend unavailable'); } +if (automated) { + const animation = (renderer as unknown as { _animation?: { stop(): void } })._animation; + if (!animation || typeof animation.stop !== 'function') + throw new Error('Pinned renderer._animation bridge shape changed.'); + // Automated frames advance the pinned NodeFrame bridge explicitly; leaving + // three's ambient RAF loop active would inject uncontrolled wall-clock ticks. + animation.stop(); +} +const validation: BenchmarkValidationRecord[] = []; +monitorDeviceLoss(renderer, validation); //* Scene & Camera const bench = createBenchScene(); -const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 200); +const camera = new THREE.PerspectiveCamera( + 50, + config.dimensions.width / config.dimensions.height, + 0.1, + 200, +); camera.position.set(9, 6, 12); -const controls = new OrbitControls(camera, renderer.domElement); -controls.target.set(0, 1.6, 0); -controls.enableDamping = true; +camera.lookAt(0, 1.6, 0); +camera.layers.enable(1); +if (scenario.id === 'Q6' || scenario.id === 'Q7' || scenario.id === 'Q8') { + camera.fov = 55; + camera.position.set(0, 4, scenario.id === 'Q6' ? 10 : 10.5); + camera.lookAt(0, 3, -5); + camera.updateProjectionMatrix(); +} +const controls = automated ? null : new OrbitControls(camera, renderer.domElement); +if (controls) { + controls.target.set(0, 1.6, 0); + controls.enableDamping = true; +} //* Pipeline & UI -const pipeline = new BenchPipeline(renderer); +const pipeline = scenario.unsupported + ? null + : new BenchPipeline( + renderer, + (activeRenderer) => registry.create(config.variant, config.ratio, activeRenderer), + variantMetadata, + ); +if (pipeline && (scenario.id === 'Q6' || scenario.id === 'Q7' || scenario.id === 'Q8')) + pipeline.configureEffectScenario(scenario.id, config.subrun, bench.roomScene, camera); function reconfigure(): void { + if (!pipeline) return; const { width, height } = displaySize(); pipeline.configure(width, height, state.mode, state.quality); } -createBenchUI(state, reconfigure, () => pipeline.upscaler.resetHistory()); -reconfigure(); - -window.addEventListener('resize', () => { - renderer.setSize(window.innerWidth, window.innerHeight); - camera.aspect = window.innerWidth / window.innerHeight; +function resizeBenchmark(dimensions: BenchmarkDimensions): void { + renderer.setPixelRatio(dimensions.devicePixelRatio); + renderer.setSize( + dimensions.width / dimensions.devicePixelRatio, + dimensions.height / dimensions.devicePixelRatio, + true, + ); + camera.aspect = dimensions.width / dimensions.height; camera.updateProjectionMatrix(); + pipeline?.configureBenchmark( + dimensions.width, + dimensions.height, + config.ratio, + scenario.id === 'Q5', + ); +} + +if (automated) { + resizeBenchmark(config.dimensions); + await pipeline?.prepareEffectReadiness(camera); + pipeline?.reset(bench.scene, camera, 0); +} +else { + createBenchUI(state, reconfigure, () => pipeline?.reset(bench.scene, camera)); reconfigure(); -}); + window.addEventListener('resize', () => { + renderer.setSize(window.innerWidth, window.innerHeight); + camera.aspect = window.innerWidth / window.innerHeight; + camera.updateProjectionMatrix(); + reconfigure(); + }); +} //* Stats Readout const statsEl = document.getElementById('stats')!; +if (automated) statsEl.style.display = 'none'; let frameCount = 0; let fpsAccum = 0; let statsClock = 0; let fps = 0; function updateStats(dt: number): void { + if (!pipeline) return; frameCount++; fpsAccum += dt; statsClock += dt; @@ -100,53 +177,87 @@ function updateStats(dt: number): void { fpsAccum = 0; statsClock = 0; - const u = pipeline.upscaler; + const resolver = pipeline.resolver; const lines = [ `mode ${state.mode}`, - `render ${u.renderWidth}×${u.renderHeight}`, - `display ${u.displayWidth}×${u.displayHeight} (${u.upscaleRatio.toFixed(2)}x)`, - `jitter ${u.jitterPhaseCount} phases`, + `render ${resolver.renderWidth}×${resolver.renderHeight}`, + `display ${resolver.displayWidth}×${resolver.displayHeight} (${resolver.upscaleRatio.toFixed(2)}x)`, + `jitter ${resolver.jitterPhaseCount} phases`, `fps ${fps.toFixed(0)}`, ]; - if (u.gpuTimings.size > 0) { + if (resolver.timings.size > 0) { lines.push('--- gpu (ms) ---'); let total = 0; - for (const [label, ms] of u.gpuTimings) { - lines.push(`${label.padEnd(10)}${ms.toFixed(3)}`); - total += ms; + for (const [label, milliseconds] of resolver.timings) { + lines.push(`${label.padEnd(10)}${milliseconds.toFixed(3)}`); + total += milliseconds; } lines.push(`${'total'.padEnd(10)}${total.toFixed(3)}`); } - statsEl.innerHTML = lines.map((l) => l.replace(/^(\S+)/, '$1')).join('\n'); + statsEl.innerHTML = lines.map((line) => line.replace(/^(\S+)/, '$1')).join('\n'); } +//* Automated API ============================================================= +const manifestDigest = await getManifestDigest(); +const environment = await collectBenchmarkEnvironment(renderer, config); +window.__UPSCALER_BENCH__ = createBenchmarkApi({ + renderer, + camera, + bench, + pipeline, + config, + metadata: variantMetadata, + scenario, + environment, + manifestDigest, + validation, + resize: resizeBenchmark, +}); + +let pageDisposed = false; +window.addEventListener( + 'pagehide', + () => { + if (pageDisposed) return; + pageDisposed = true; + renderer.setAnimationLoop(null); + controls?.dispose(); + if (pipeline) { + pipeline.dispose(); + registry.releaseDisposed(pipeline.resolver); + } + renderer.dispose(); + window.__UPSCALER_BENCH__ = undefined; + }, + { once: true }, +); + //* Main Loop const timer = new THREE.Timer(); +let interactiveFrame = 0; -renderer.setAnimationLoop(() => { - timer.update(); - const dt = Math.min(timer.getDelta(), 0.1); - const time = timer.getElapsed(); +if (!automated && pipeline && controls) { + renderer.setAnimationLoop(() => { + timer.update(); + const dt = Math.min(timer.getDelta(), 0.1); + const time = timer.getElapsed(); - if (state.autoOrbit) { - controls.autoRotate = true; + controls.autoRotate = state.autoOrbit; controls.autoRotateSpeed = 0.6; - } else { - controls.autoRotate = false; - } - controls.update(); - - bench.update(time, state.animate); - pipeline.applySettings({ - sharpness: state.sharpness, - rcasDenoise: state.rcasDenoise, - maxAccumulation: state.maxAccumulation, - exposure: state.exposure, - autoExposure: state.autoExposure, - lockThinFeatures: state.lockThinFeatures, - detectShadingChanges: state.detectShadingChanges, - debugView: state.debugView, + controls.update(); + + bench.update(time, state.animate); + pipeline.applySettings({ + sharpness: state.sharpness, + rcasDenoise: state.rcasDenoise, + maxAccumulation: state.maxAccumulation, + exposure: state.exposure, + autoExposure: state.autoExposure, + lockThinFeatures: state.lockThinFeatures, + detectShadingChanges: state.detectShadingChanges, + debugView: state.debugView, + }); + pipeline.render(bench.scene, camera, dt, interactiveFrame++); + updateStats(dt); }); - pipeline.render(bench.scene, camera, dt); - updateStats(dt); -}); +} diff --git a/bench/src/types/benchmark.d.ts b/bench/src/types/benchmark.d.ts new file mode 100644 index 0000000..c2de2a6 --- /dev/null +++ b/bench/src/types/benchmark.d.ts @@ -0,0 +1,273 @@ +declare type BenchmarkMode = 'interactive' | 'performance' | 'capture'; +declare type BenchmarkVariantId = + | 'baseline' + | 'local-baseline-5d6a65e' + | 'local-baseline-through-e00-harness' + | 'rcas-fsr315-limiter' + | 'rcas-fsr315-numeric' + | 'source-filter-bundle-v1' + | 'source-structural-bundle-v1' + | 'source-spd-resolver-bundle-v1'; +declare type BenchmarkScenarioId = + | 'Q0' + | 'Q1' + | 'Q2' + | 'Q3' + | 'Q4' + | 'Q5' + | 'Q6' + | 'Q7' + | 'Q8' + | 'Q9' + | 'Q10'; +declare type BenchmarkDebugView = + | 'final' + | 'motion-vectors' + | 'disocclusion' + | 'accumulation-age' + | 'locks' + | 'exposure' + | 'shading-change' + | 'reactivity'; + +declare interface BenchmarkDimensions { + width: number; + height: number; + devicePixelRatio: number; +} + +declare interface BenchmarkRunConfig { + experiment: 'E00'; + mode: BenchmarkMode; + variant: BenchmarkVariantId; + comparison: BenchmarkVariantId; + scenario: BenchmarkScenarioId; + subrun: string | null; + ratio: number; + dimensions: BenchmarkDimensions; + timestepSeconds: number; + warmupFrames: number; + sampleFrames: number; + authoritativeTiming: boolean; +} + +declare interface BenchmarkPipelineMetadata { + shaderKey: string; + pipelineKey: string; + assembledChunks: readonly string[]; + wgslOverrides: Readonly>; + timingPassLabels: readonly string[]; +} + +declare interface BenchmarkVariantMetadata { + id: BenchmarkVariantId; + name: string; + supportedRatios: readonly number[]; + settings: Readonly>; + resourceGraph: readonly string[]; + pipeline: BenchmarkPipelineMetadata; +} + +declare interface BenchmarkResolverConfigure { + displayWidth: number; + displayHeight: number; + ratio: number; + path: 'bilinear' | 'spatial' | 'temporal'; +} + +declare interface BenchmarkResolverDispatch { + color: unknown; + depth?: unknown; + velocity?: unknown; + reactive?: unknown; + transparencyAndComposition?: unknown; + preExposureTexture?: unknown; + deltaTime: number; + frameTag: number; +} + +declare interface BenchmarkResolver { + readonly metadata: BenchmarkVariantMetadata; + readonly outputTexture: unknown; + readonly renderWidth: number; + readonly renderHeight: number; + readonly displayWidth: number; + readonly displayHeight: number; + readonly upscaleRatio: number; + readonly jitterPhaseCount: number; + readonly timestampQuerySupported: boolean; + readonly unjitteredProjectionMatrix: unknown; + readonly settings: Record; + readonly timings: ReadonlyMap; + configure(config: BenchmarkResolverConfigure): void; + beginFrame(camera: unknown): void; + endFrame(camera: unknown): void; + dispatch(inputs: BenchmarkResolverDispatch, camera: unknown): void; + reset(): void; + resetTiming(): void; + setAuthoritativeTiming(authoritative: boolean): void; + waitForTimingCapacity(): Promise; + drainTiming(): Promise; + takeTimingSamples(): BenchmarkGpuFrameSample[]; + dispose(): void; +} + +declare type BenchmarkResolverFactory = ( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +) => BenchmarkResolver; + +declare interface BenchmarkVariantDefinition { + metadata: BenchmarkVariantMetadata; + create: BenchmarkResolverFactory; +} + +declare interface BenchmarkGpuPassSample { + label: string; + milliseconds: number; +} + +declare interface BenchmarkGpuFrameSample { + frameTag: number; + sequence: number; + passes: BenchmarkGpuPassSample[]; +} + +declare interface BenchmarkPassSummary { + label: string; + samples: number[]; + median: number | null; + p95: number | null; + missingCount: number; +} + +declare interface BenchmarkTimingSummary { + expectedFrames: number; + receivedFrames: number; + missingFrameCount: number; + duplicateFrameCount: number; + duplicateSequenceCount: number; + unexpectedFrameCount: number; + duplicatePassLabelCount: number; + invalidValueCount: number; + inconsistentPassSetCount: number; + invalidityCount: number; + expectedPassLabels: string[]; + invalidSamples: BenchmarkInvalidTimingSample[]; + passes: BenchmarkPassSummary[]; + computeSum: BenchmarkPassSummary; + raw: BenchmarkGpuFrameSample[]; +} + +declare interface BenchmarkInvalidTimingSample { + frameTag: number; + sequence: number; + reason: + | 'unexpected-frame' + | 'duplicate-frame' + | 'duplicate-sequence' + | 'duplicate-pass-label' + | 'invalid-pass-value' + | 'inconsistent-pass-set'; + detail: string; +} + +declare interface BenchmarkUnsupportedCapability { + code: string; + capability: string; + reason: string; +} + +declare interface BenchmarkScenarioDefinition { + id: BenchmarkScenarioId; + name: string; + endFrame: number; + captures: readonly string[]; + debugViews: readonly BenchmarkDebugView[]; + rois: Readonly>; + subruns: readonly string[]; + unsupported: BenchmarkUnsupportedCapability | null; + frame(frame: number): BenchmarkFrameState; +} + +declare interface BenchmarkFrameState { + frame: number; + time: number; + cameraPosition: readonly [number, number, number]; + cameraTarget: readonly [number, number, number]; + sceneTime: number; + animateScene: boolean; + directionalIntensity: number; + resetHistory: boolean; + resize: BenchmarkDimensions | null; + particlesVisible: boolean; +} + +declare interface BenchmarkCaptureRequest { + frame: number; + debugView: BenchmarkDebugView; +} + +declare interface BenchmarkCaptureResult { + scenario: BenchmarkScenarioId; + subrun: string | null; + ratio: number; + frame: number; + debugView: BenchmarkDebugView; + width: number; + height: number; + jitterPeriod: number; +} + +declare interface BenchmarkRunOptions { + warmupFrames?: number; + sampleFrames?: number; +} + +declare interface BenchmarkValidationRecord { + channel: string; + level: string; + text: string; + timestamp: number; +} + +declare interface BenchmarkEnvironment { + browser: string; + operatingSystem: string; + adapter: string; + backend: string; + webgpuFeatures: string[]; + threeVersion: string; + dimensions: BenchmarkDimensions; + ratio: number; + fixedTimestep: number; +} + +declare interface BenchmarkResult { + status: 'ready' | 'complete' | 'unsupported' | 'failed'; + experiment: 'E00'; + manifestDigest: string; + config: BenchmarkRunConfig; + variant: BenchmarkVariantMetadata; + scenario: BenchmarkScenarioId; + unsupported: BenchmarkUnsupportedCapability | null; + environment: BenchmarkEnvironment; + timing: BenchmarkTimingSummary | null; + captures: BenchmarkCaptureResult[]; + validation: BenchmarkValidationRecord[]; +} + +declare interface UpscalerBenchmarkApi { + readonly ready: boolean; + readonly config: BenchmarkRunConfig; + readonly metadata: BenchmarkVariantMetadata; + readonly result: BenchmarkResult; + step(frame?: number): Promise; + reset(): Promise; + capture(request: BenchmarkCaptureRequest): Promise; + run(options?: BenchmarkRunOptions): Promise; +} + +interface Window { + __UPSCALER_BENCH__?: UpscalerBenchmarkApi; +} diff --git a/examples/README.md b/examples/README.md index f70c1c3..d4976fd 100644 --- a/examples/README.md +++ b/examples/README.md @@ -61,7 +61,7 @@ inspector can't give you per-GPU-pass times. Notes for the DPR demo: Every demo drives the library through [`shared/UpscalePresenter.ts`](shared/UpscalePresenter.ts), which encapsulates the whole integration recipe (jitter-free velocity, MRT output count matched to the render-target attachment count, float depth, the -`NoToneMapping` present). New demos should reuse it rather than re-deriving the +linear/HDR output, and renderer-owned presentation). New demos should reuse it rather than re-deriving the wiring — the one exception is `06`, which drives the raw `Upscaler` directly so it can feed FSR3 the output of a TSL pass graph. diff --git a/examples/shared/boot.ts b/examples/shared/boot.ts index ea4621a..5f81770 100644 --- a/examples/shared/boot.ts +++ b/examples/shared/boot.ts @@ -2,8 +2,7 @@ import * as THREE from 'three/webgpu'; /** * Shared WebGPU bootstrap for the examples. Guards for WebGPU support, creates - * a `WebGPURenderer` configured for FSR3 presentation (the FSR output is - * already display-referred sRGB, so tone mapping / output encoding are off), + * a `WebGPURenderer` configured to present the upscaler's linear/HDR output, * and awaits `init()` — throwing loudly if three falls back to WebGL. * * @param options - Optional canvas parent (defaults to `document.body`) @@ -28,9 +27,9 @@ export async function bootRenderer(options: { parent?: HTMLElement } = {}): Prom const renderer = new THREE.WebGPURenderer({ antialias: false }); renderer.setPixelRatio(dpr); renderer.setSize(window.innerWidth, window.innerHeight); - // FSR output is already tonemapped + sRGB-encoded in WGSL — present untouched. - renderer.toneMapping = THREE.NoToneMapping; - renderer.outputColorSpace = THREE.LinearSRGBColorSpace; + // The upscaler does not own presentation; examples choose ACES + sRGB. + renderer.toneMapping = THREE.ACESFilmicToneMapping; + renderer.outputColorSpace = THREE.SRGBColorSpace; (options.parent ?? document.body).appendChild(renderer.domElement); await renderer.init(); diff --git a/package.json b/package.json index 66d6c76..eb02f47 100644 --- a/package.json +++ b/package.json @@ -45,6 +45,9 @@ "scripts": { "dev": "vite --config bench/vite.config.ts", "bench": "vite --config bench/vite.config.ts", + "bench:run": "node scripts/run-benchmark.mjs --mode performance", + "bench:capture": "node scripts/run-benchmark.mjs --mode capture", + "bench:compare:rcas": "node scripts/compare-rcas.mjs", "examples": "vite --config examples/vite.config.ts", "examples:build": "vite build --config examples/vite.config.ts", "build": "vite build && tsc --declaration --emitDeclarationOnly -p tsconfig.build.json", diff --git a/scripts/benchmark-contract.mjs b/scripts/benchmark-contract.mjs new file mode 100644 index 0000000..14cdeab --- /dev/null +++ b/scripts/benchmark-contract.mjs @@ -0,0 +1,145 @@ +import { createHash } from 'node:crypto'; + +/** + * Checks the aggregate E00 timing noise gate. + * + * Per-pass noise remains recorded for claim eligibility, but only the compute + * sum controls the aggregate harness retry and blocker state. + * + * @param {Array<{ label: string; noiseFloorPass: boolean }>} analysis + * @returns {boolean} + */ +export function computeSumNoiseFloorPasses(analysis) { + const computeSums = analysis.filter((entry) => entry.label === 'compute-sum'); + return computeSums.length === 1 && computeSums[0].noiseFloorPass; +} + +/** + * Checks comparisons that are eligible to support authoritative claims. + * + * @param {Array<{ + * label: string; + * individualClaimEligible: boolean; + * median: { passes: boolean }; + * p95: { passes: boolean }; + * }>} analysis + * @returns {boolean} + */ +export function authoritativeComparisonPasses(analysis) { + return analysis + .filter( + (entry) => + entry.label === 'compute-sum' || entry.individualClaimEligible, + ) + .every((entry) => entry.median.passes && entry.p95.passes); +} + +/** + * Hashes a complete non-ignored working-tree inventory. + * + * @param {Array<{ path: string; bytes: Buffer | null }>} entries + * @returns {string} + */ +export function hashWorkingTreeEntries(entries) { + const hash = createHash('sha256'); + for (const entry of entries.toSorted((a, b) => a.path.localeCompare(b.path))) { + hash.update(entry.path); + hash.update('\0'); + hash.update(entry.bytes ?? Buffer.from('')); + hash.update('\0'); + } + return hash.digest('hex'); +} + +/** + * Builds a review persistence key bound to one evidence set and record order. + * + * @param {{ manifestDigest: string; workingTreeDigest: string }} binding + * @param {Array<{ id: string }>} records + * @returns {string} + */ +export function reviewStorageKey(binding, records) { + const digest = createHash('sha256') + .update(binding.manifestDigest) + .update('\0') + .update(binding.workingTreeDigest) + .update('\0') + .update(records.map((record) => record.id).join('\0')) + .digest('hex'); + return `fsr3-e00-review-${digest}`; +} + +/** + * Rejects review evidence that did not come from the active contract and tree. + * + * @param {{ + * run: { manifestDigest?: string; workingTreeDigest?: string }; + * analysis: { + * authoritative?: boolean; + * completeProtocol?: boolean; + * passes?: boolean; + * tupleCount?: number; + * expectedTupleCount?: number; + * failedPairCount?: number; + * }; + * manifestDigest: string; + * workingTreeDigest: string; + * }} input + * @returns {void} + */ +export function assertCaptureEvidenceBinding({ + run, + analysis, + manifestDigest, + workingTreeDigest, +}) { + if (run.manifestDigest !== manifestDigest) + throw new Error('Capture manifest digest does not match the active E00 contract.'); + if (run.workingTreeDigest !== workingTreeDigest) + throw new Error('Capture working-tree digest does not match the active source state.'); + if ( + analysis.authoritative !== true || + analysis.completeProtocol !== true || + analysis.passes !== true || + analysis.tupleCount !== analysis.expectedTupleCount || + analysis.failedPairCount !== 0 + ) + throw new Error('Review requires a complete passing authoritative capture analysis.'); +} + +/** + * Closes a target created inside a user-owned CDP browser and awaits removal. + * + * @param {string} cdpBase + * @param {string} targetId + * @param {{ + * fetchImpl?: typeof fetch; + * wait?: (milliseconds: number) => Promise; + * attempts?: number; + * }} [options] + * @returns {Promise} + */ +export async function closeExternalCdpTarget( + cdpBase, + targetId, + { + fetchImpl = fetch, + wait = (milliseconds) => + new Promise((resolveWait) => setTimeout(resolveWait, milliseconds)), + attempts = 20, + } = {}, +) { + const closeResponse = await fetchImpl(`${cdpBase}/json/close/${targetId}`); + if (!closeResponse.ok) + throw new Error(`Unable to close CDP target ${targetId}: ${closeResponse.status}`); + + for (let attempt = 0; attempt < attempts; attempt++) { + const listResponse = await fetchImpl(`${cdpBase}/json/list`); + if (!listResponse.ok) + throw new Error(`Unable to inspect CDP targets: ${listResponse.status}`); + const targets = await listResponse.json(); + if (!targets.some((target) => target.id === targetId)) return; + await wait(50); + } + throw new Error(`Timed out waiting for CDP target ${targetId} to close.`); +} diff --git a/scripts/benchmark-contract.test.mjs b/scripts/benchmark-contract.test.mjs new file mode 100644 index 0000000..789561c --- /dev/null +++ b/scripts/benchmark-contract.test.mjs @@ -0,0 +1,171 @@ +import { describe, expect, test } from 'vitest'; + +import { + assertCaptureEvidenceBinding, + authoritativeComparisonPasses, + closeExternalCdpTarget, + computeSumNoiseFloorPasses, + hashWorkingTreeEntries, + reviewStorageKey, +} from './benchmark-contract.mjs'; + +describe('E00 timing contract', () => { + test('accepts a stable compute sum while preserving ineligible per-pass evidence', () => { + const analysis = [ + { label: 'accumulate', noiseFloorPass: false }, + { label: 'compute-sum', noiseFloorPass: true }, + ]; + + expect(computeSumNoiseFloorPasses(analysis)).toBe(true); + expect(analysis[0].noiseFloorPass).toBe(false); + }); + + test('rejects a missing or unstable compute sum', () => { + expect(computeSumNoiseFloorPasses([])).toBe(false); + expect( + computeSumNoiseFloorPasses([ + { label: 'compute-sum', noiseFloorPass: false }, + ]), + ).toBe(false); + }); + + test('gates aggregate and eligible pass comparisons without claiming noisy passes', () => { + expect( + authoritativeComparisonPasses([ + { + label: 'accumulate', + individualClaimEligible: false, + median: { passes: false }, + p95: { passes: false }, + }, + { + label: 'rcas', + individualClaimEligible: true, + median: { passes: true }, + p95: { passes: true }, + }, + { + label: 'compute-sum', + individualClaimEligible: true, + median: { passes: true }, + p95: { passes: true }, + }, + ]), + ).toBe(true); + }); + +}); + +describe('E00 evidence binding', () => { + const run = { + manifestDigest: 'manifest-a', + workingTreeDigest: 'tree-a', + }; + const analysis = { + authoritative: true, + completeProtocol: true, + passes: true, + tupleCount: 264, + expectedTupleCount: 264, + failedPairCount: 0, + }; + + test('accepts only a complete passing capture from the current contract and tree', () => { + expect(() => + assertCaptureEvidenceBinding({ + run, + analysis, + manifestDigest: 'manifest-a', + workingTreeDigest: 'tree-a', + }), + ).not.toThrow(); + }); + + test('rejects stale manifests, changed trees, and incomplete capture evidence', () => { + expect(() => + assertCaptureEvidenceBinding({ + run, + analysis, + manifestDigest: 'manifest-b', + workingTreeDigest: 'tree-a', + }), + ).toThrow(/manifest digest/i); + expect(() => + assertCaptureEvidenceBinding({ + run, + analysis, + manifestDigest: 'manifest-a', + workingTreeDigest: 'tree-b', + }), + ).toThrow(/working-tree digest/i); + expect(() => + assertCaptureEvidenceBinding({ + run, + analysis: { ...analysis, completeProtocol: false }, + manifestDigest: 'manifest-a', + workingTreeDigest: 'tree-a', + }), + ).toThrow(/complete passing authoritative capture/i); + }); + + test('isolates persisted review state by evidence and record identity', () => { + const records = [{ id: 'first' }, { id: 'second' }]; + const baseline = reviewStorageKey(run, records); + + expect(reviewStorageKey(run, records)).toBe(baseline); + expect(reviewStorageKey({ ...run, manifestDigest: 'manifest-b' }, records)).not.toBe( + baseline, + ); + expect(reviewStorageKey({ ...run, workingTreeDigest: 'tree-b' }, records)).not.toBe( + baseline, + ); + expect(reviewStorageKey(run, records.toReversed())).not.toBe(baseline); + }); + + test('hashes working-tree entries independently of enumeration order', () => { + const entries = [ + { path: 'b.ts', bytes: Buffer.from('b') }, + { path: 'a.ts', bytes: Buffer.from('a') }, + ]; + + expect(hashWorkingTreeEntries(entries)).toBe( + hashWorkingTreeEntries(entries.toReversed()), + ); + expect( + hashWorkingTreeEntries([ + entries[0], + { path: 'a.ts', bytes: Buffer.from('changed') }, + ]), + ).not.toBe(hashWorkingTreeEntries(entries)); + }); +}); + +describe('external CDP cleanup', () => { + test('closes the created target and waits until it disappears', async () => { + const calls = []; + let listCount = 0; + const fetchImpl = async (url) => { + calls.push(url); + if (url.endsWith('/json/close/target-1')) return { ok: true, status: 200 }; + listCount++; + return { + ok: true, + status: 200, + async json() { + return listCount === 1 ? [{ id: 'target-1' }] : []; + }, + }; + }; + + await closeExternalCdpTarget('http://127.0.0.1:9333', 'target-1', { + fetchImpl, + wait: async () => {}, + }); + + expect(calls).toEqual([ + 'http://127.0.0.1:9333/json/close/target-1', + 'http://127.0.0.1:9333/json/list', + 'http://127.0.0.1:9333/json/list', + ]); + }); +}); diff --git a/scripts/compare-rcas.mjs b/scripts/compare-rcas.mjs new file mode 100644 index 0000000..ff5e190 --- /dev/null +++ b/scripts/compare-rcas.mjs @@ -0,0 +1,300 @@ +import { spawn } from 'node:child_process'; +import { createServer } from 'node:http'; +import { mkdir, readFile, writeFile } from 'node:fs/promises'; +import { extname, join, relative, resolve, sep } from 'node:path'; +import process from 'node:process'; + +const ROOT = resolve(import.meta.dirname, '..'); +const RUNNER = join(ROOT, 'scripts/run-benchmark.mjs'); + +//* Arguments ================================================================ + +function parseArguments(argv) { + const options = {}; + for (let index = 0; index < argv.length; index++) { + const value = argv[index]; + if (!value.startsWith('--')) continue; + const name = value.slice(2); + const next = argv[index + 1]; + if (next !== undefined && !next.startsWith('--')) options[name] = argv[++index]; + else options[name] = true; + } + return options; +} + +const options = parseArguments(process.argv.slice(2)); +const stamp = new Date().toISOString().replaceAll(':', '-'); +const outputDirectory = resolve( + options.reuse ?? + options.output ?? + join(ROOT, 'bench/results/raw/E01', `rcas-comparison-${stamp}`), +); + +//* Benchmark execution ====================================================== + +function runBenchmark(mode, output, arguments_) { + return new Promise((resolveRun, rejectRun) => { + const child = spawn( + process.execPath, + [ + RUNNER, + '--mode', + mode, + '--smoke', + '--output', + output, + ...arguments_, + ], + { + cwd: ROOT, + stdio: 'inherit', + }, + ); + child.once('error', rejectRun); + child.once('exit', (code, signal) => { + if (code === 0) resolveRun(); + else rejectRun( + new Error( + `Benchmark ${relative(ROOT, output)} stopped with ${signal ?? `code ${code}`}.`, + ), + ); + }); + }); +} + +const sharedTiming = [ + '--ratios', + '2', + '--blocks', + '3', + '--warmup', + '240', + '--samples', + '300', +]; +const sharedCapture = [ + '--ratios', + '2', + '--frames', + '0,120', + '--views', + 'final', + '--reloads', + '1', + '--allow-differences', + '--review-all', +]; + +if (!options.reuse) { + await mkdir(outputDirectory, { recursive: true }); + + const limiterTiming = join(outputDirectory, 'lower-limiter-timing'); + const limiterCapture = join(outputDirectory, 'lower-limiter-review'); + const denoiseTiming = join(outputDirectory, 'denoise-default-timing'); + const denoiseCapture = join(outputDirectory, 'denoise-default-review'); + + await runBenchmark('performance', limiterTiming, [ + '--variant', + 'local-baseline-5d6a65e', + '--comparison', + 'rcas-fsr315-limiter', + ...sharedTiming, +]); + await runBenchmark('capture', limiterCapture, [ + '--variant', + 'local-baseline-5d6a65e', + '--comparison', + 'rcas-fsr315-limiter', + '--scenarios', + 'Q0,Q1,Q6', + '--review-title', + 'RCAS: legacy vs FSR 3.1.5 lower limiter', + ...sharedCapture, +]); + await runBenchmark('performance', denoiseTiming, [ + '--variant', + 'rcas-fsr315-limiter', + '--comparison', + 'rcas-fsr315-numeric', + ...sharedTiming, +]); + await runBenchmark('capture', denoiseCapture, [ + '--variant', + 'rcas-fsr315-limiter', + '--comparison', + 'rcas-fsr315-numeric', + '--scenarios', + 'Q0,Q1', + '--review-title', + 'RCAS: lower limiter vs temporal denoise default', + ...sharedCapture, +]); + +//* Report =================================================================== + +function median(values) { + const sorted = values.toSorted((a, b) => a - b); + const middle = Math.floor(sorted.length / 2); + return sorted.length % 2 === 0 + ? (sorted[middle - 1] + sorted[middle]) / 2 + : sorted[middle]; +} + +function classify(delta) { + const magnitude = Math.abs(delta); + if (magnitude < 0.03) return 'tie / within practical noise'; + if (magnitude < 0.05) return 'uncertain'; + return delta > 0 ? 'candidate slower' : 'candidate faster'; +} + +async function timingRows(directory) { + const analysis = JSON.parse( + await readFile(join(directory, 'abba-analysis.json'), 'utf8'), + ); + return ['rcas', 'compute-sum'].map((label) => { + const entry = analysis.find((candidate) => candidate.label === label); + const deltas = entry.median.rows.map( + (row) => (row.meanB - row.meanA) / row.meanA, + ); + const result = median(deltas); + return { + label, + deltas, + median: result, + classification: classify(result), + }; + }); +} + +async function captureRows(directory) { + const analysis = JSON.parse( + await readFile(join(directory, 'capture-analysis.json'), 'utf8'), + ); + return analysis.pairs.map((pair) => ({ + tuple: pair.tuple, + max255: pair.metrics.full.maxAbsolute * 255, + rmse255: pair.metrics.full.rmse * 255, + })); +} + +function timingTable(rows) { + return rows + .map( + (row) => ` +${row.label} +${row.deltas.map((value) => `${(value * 100).toFixed(2)}%`).join(', ')} +${(row.median * 100).toFixed(2)}% +${row.classification} +`, + ) + .join(''); +} + +function captureTable(rows) { + return rows + .map( + (row) => ` +${row.tuple} +${row.max255.toFixed(2)} / 255 +${row.rmse255.toFixed(3)} / 255 +`, + ) + .join(''); +} + + const limiterTimingRows = await timingRows(limiterTiming); + const denoiseTimingRows = await timingRows(denoiseTiming); + const limiterCaptureRows = await captureRows(limiterCapture); + const denoiseCaptureRows = await captureRows(denoiseCapture); + + const report = ` + + + + +RCAS comparison + + + +

FSR 3.1.5 RCAS comparison

+

This is a directional engineering comparison, not a publication-grade benchmark. Candidate B deltas below 3% are treated as tied, 3–5% as uncertain, and at least 5% as actionable.

+ +
+

2. Lower limiter vs temporal denoise default

+

Open blinded visual review

+ +${timingTable(denoiseTimingRows)}
TimingThree A/B repetitionsMedianInterpretation
+

Pixel differences

+ +${captureTable(denoiseCaptureRows)}
CaptureMaximumRMSE
+
+

The review orientation is deterministically blinded. Use the ROI overlay and 4× zoom; visual regressions override timing wins. Raw JSON, PNGs, and heatmaps are retained beside this report.

+ +`; + + await writeFile(join(outputDirectory, 'index.html'), report); +} + +//* Local review server ====================================================== + +if (options['no-serve']) { + console.log(`RCAS comparison written to ${outputDirectory}`); + process.exit(0); +} + +const contentTypes = { + '.html': 'text/html; charset=utf-8', + '.json': 'application/json; charset=utf-8', + '.png': 'image/png', +}; +const server = createServer(async (request, response) => { + try { + const pathname = decodeURIComponent( + new URL(request.url ?? '/', 'http://localhost').pathname, + ); + const requested = pathname === '/' ? 'index.html' : pathname.slice(1); + const file = resolve(outputDirectory, requested); + if (file !== outputDirectory && !file.startsWith(`${outputDirectory}${sep}`)) + throw new Error('Invalid review path.'); + const body = await readFile(file); + response.writeHead(200, { + 'content-type': contentTypes[extname(file)] ?? 'application/octet-stream', + 'cache-control': 'no-store', + }); + response.end(body); + } catch { + response.writeHead(404, { 'content-type': 'text/plain; charset=utf-8' }); + response.end('Not found'); + } +}); + +await new Promise((resolveListen, rejectListen) => { + server.once('error', rejectListen); + server.listen(Number(options.port ?? 0), '127.0.0.1', resolveListen); +}); +const address = server.address(); +const url = `http://127.0.0.1:${address.port}/`; +console.log(`RCAS comparison ready: ${url}`); +console.log('Press Ctrl+C when review is complete.'); + +if (!options['no-open'] && process.platform === 'darwin') { + const opener = spawn('open', [url], { detached: true, stdio: 'ignore' }); + opener.unref(); +} diff --git a/scripts/run-benchmark.mjs b/scripts/run-benchmark.mjs new file mode 100644 index 0000000..19e36d0 --- /dev/null +++ b/scripts/run-benchmark.mjs @@ -0,0 +1,1746 @@ +import { execFile, spawn } from 'node:child_process'; +import { createHash } from 'node:crypto'; +import { existsSync } from 'node:fs'; +import { mkdir, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import process from 'node:process'; +import { promisify } from 'node:util'; +import { deflateSync, inflateSync } from 'node:zlib'; + +import { + assertCaptureEvidenceBinding, + authoritativeComparisonPasses, + closeExternalCdpTarget, + computeSumNoiseFloorPasses, + hashWorkingTreeEntries, + reviewStorageKey, +} from './benchmark-contract.mjs'; + +const ROOT = resolve(import.meta.dirname, '..'); +const MANIFEST_PATH = join(ROOT, 'bench/results/experiments/e00-harness.json'); +const DEFAULT_URL = 'http://127.0.0.1:5199'; +const execFileAsync = promisify(execFile); + +class UserDecisionRequired extends Error { + constructor(message) { + super(message); + this.name = 'USER_DECISION_REQUIRED'; + } +} + +class BlockedError extends Error { + constructor(message) { + super(message); + this.name = 'BLOCKED'; + } +} + +const PNG_SIGNATURE = Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]); +const CRC_TABLE = Array.from({ length: 256 }, (_, value) => { + let crc = value; + for (let bit = 0; bit < 8; bit++) crc = (crc & 1) !== 0 ? 0xedb88320 ^ (crc >>> 1) : crc >>> 1; + return crc >>> 0; +}); + +function crc32(buffer) { + let crc = 0xffffffff; + for (const value of buffer) crc = CRC_TABLE[(crc ^ value) & 0xff] ^ (crc >>> 8); + return (crc ^ 0xffffffff) >>> 0; +} + +function pngChunk(type, data) { + const name = Buffer.from(type); + const chunk = Buffer.alloc(data.length + 12); + chunk.writeUInt32BE(data.length, 0); + name.copy(chunk, 4); + data.copy(chunk, 8); + chunk.writeUInt32BE(crc32(Buffer.concat([name, data])), data.length + 8); + return chunk; +} + +function decodePng(bytes) { + if (!bytes.subarray(0, 8).equals(PNG_SIGNATURE)) throw new Error('Invalid PNG signature.'); + let offset = 8; + let width = 0; + let height = 0; + let bitDepth = 0; + let colorType = 0; + let interlace = 0; + const compressed = []; + while (offset < bytes.length) { + const length = bytes.readUInt32BE(offset); + const type = bytes.toString('ascii', offset + 4, offset + 8); + const data = bytes.subarray(offset + 8, offset + 8 + length); + const expectedCrc = bytes.readUInt32BE(offset + 8 + length); + if (crc32(Buffer.concat([Buffer.from(type), data])) !== expectedCrc) + throw new Error(`PNG ${type} CRC mismatch.`); + if (type === 'IHDR') { + width = data.readUInt32BE(0); + height = data.readUInt32BE(4); + bitDepth = data[8]; + colorType = data[9]; + interlace = data[12]; + } else if (type === 'IDAT') compressed.push(data); + else if (type === 'IEND') break; + offset += length + 12; + } + if (width < 1 || height < 1 || bitDepth !== 8 || ![2, 6].includes(colorType) || interlace !== 0) + throw new Error( + `PNG contract requires non-interlaced RGB8/RGBA8; got ${width}x${height}, depth=${bitDepth}, type=${colorType}, interlace=${interlace}.`, + ); + + const packed = inflateSync(Buffer.concat(compressed)); + const bytesPerPixel = colorType === 6 ? 4 : 3; + const stride = width * bytesPerPixel; + const rawPixels = Buffer.alloc(stride * height); + let source = 0; + for (let y = 0; y < height; y++) { + const filter = packed[source++]; + for (let x = 0; x < stride; x++) { + const raw = packed[source++]; + const left = + x >= bytesPerPixel ? rawPixels[y * stride + x - bytesPerPixel] : 0; + const above = y > 0 ? rawPixels[(y - 1) * stride + x] : 0; + const upperLeft = + y > 0 && x >= bytesPerPixel + ? rawPixels[(y - 1) * stride + x - bytesPerPixel] + : 0; + let predictor = 0; + if (filter === 1) predictor = left; + else if (filter === 2) predictor = above; + else if (filter === 3) predictor = Math.floor((left + above) / 2); + else if (filter === 4) { + const p = left + above - upperLeft; + const pa = Math.abs(p - left); + const pb = Math.abs(p - above); + const pc = Math.abs(p - upperLeft); + predictor = pa <= pb && pa <= pc ? left : pb <= pc ? above : upperLeft; + } else if (filter !== 0) throw new Error(`Unsupported PNG filter ${filter}.`); + rawPixels[y * stride + x] = (raw + predictor) & 0xff; + } + } + const rgba = Buffer.alloc(width * height * 4); + for (let pixel = 0; pixel < width * height; pixel++) { + rawPixels.copy( + rgba, + pixel * 4, + pixel * bytesPerPixel, + pixel * bytesPerPixel + bytesPerPixel, + ); + if (bytesPerPixel === 3) rgba[pixel * 4 + 3] = 255; + } + return { width, height, bitDepth, colorType, rgba }; +} + +function encodePng(width, height, rgba) { + const scanlines = Buffer.alloc((width * 4 + 1) * height); + for (let y = 0; y < height; y++) { + const row = y * (width * 4 + 1); + scanlines[row] = 0; + rgba.copy(scanlines, row + 1, y * width * 4, (y + 1) * width * 4); + } + const ihdr = Buffer.alloc(13); + ihdr.writeUInt32BE(width, 0); + ihdr.writeUInt32BE(height, 4); + ihdr[8] = 8; + ihdr[9] = 6; + return Buffer.concat([ + PNG_SIGNATURE, + pngChunk('IHDR', ihdr), + pngChunk('IDAT', deflateSync(scanlines)), + pngChunk('IEND', Buffer.alloc(0)), + ]); +} + +function parseArguments(argv) { + const options = {}; + for (let index = 0; index < argv.length; index++) { + const value = argv[index]; + if (!value.startsWith('--')) continue; + const [name, inline] = value.slice(2).split('=', 2); + const next = argv[index + 1]; + if (inline !== undefined) options[name] = inline; + else if (next !== undefined && !next.startsWith('--')) options[name] = argv[++index]; + else options[name] = true; + } + return options; +} + +function list(value, fallback) { + return String(value ?? fallback) + .split(',') + .map((entry) => entry.trim()) + .filter(Boolean); +} + +async function currentWorkingTreeDigest() { + const { stdout } = await execFileAsync( + 'git', + ['ls-files', '--cached', '--others', '--exclude-standard', '-z'], + { cwd: ROOT, encoding: 'buffer', maxBuffer: 16 * 1024 * 1024 }, + ); + const inventory = Buffer.from(stdout) + .toString('utf8') + .split('\0') + .filter(Boolean); + const entries = await Promise.all( + inventory.map(async (path) => { + try { + return { path, bytes: await readFile(join(ROOT, path)) }; + } catch (error) { + if (error?.code === 'ENOENT') return { path, bytes: null }; + throw error; + } + }), + ); + return hashWorkingTreeEntries(entries); +} + +function chromeExecutable(explicit) { + const candidates = [ + explicit, + process.env.CHROME_PATH, + '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', + '/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary', + '/Applications/Chromium.app/Contents/MacOS/Chromium', + '/usr/bin/google-chrome', + '/usr/bin/chromium', + ].filter(Boolean); + const executable = candidates.find(existsSync); + if (!executable) throw new Error('Chrome was not found. Pass --chrome /path/to/chrome.'); + return executable; +} + +async function waitForUrl(url, attempts = 100) { + for (let attempt = 0; attempt < attempts; attempt++) { + try { + const response = await fetch(url); + if (response.ok) return; + } catch { + // The process is still starting. + } + await new Promise((resolveWait) => setTimeout(resolveWait, 100)); + } + throw new Error(`Timed out waiting for ${url}`); +} + +class CdpClient { + constructor(url) { + this.socket = new WebSocket(url); + this.nextId = 1; + this.pending = new Map(); + this.listeners = new Map(); + this.opened = new Promise((resolveOpen, rejectOpen) => { + this.socket.addEventListener('open', resolveOpen, { once: true }); + this.socket.addEventListener('error', rejectOpen, { once: true }); + }); + this.socket.addEventListener('message', (event) => { + const message = JSON.parse(event.data); + if (message.id) { + const request = this.pending.get(message.id); + if (!request) return; + this.pending.delete(message.id); + if (message.error) request.reject(new Error(message.error.message)); + else request.resolve(message.result); + return; + } + for (const listener of this.listeners.get(message.method) ?? []) + listener(message.params); + }); + } + + async call(method, params = {}) { + await this.opened; + const id = this.nextId++; + const response = new Promise((resolveCall, rejectCall) => { + this.pending.set(id, { resolve: resolveCall, reject: rejectCall }); + }); + this.socket.send(JSON.stringify({ id, method, params })); + return response; + } + + on(method, listener) { + const listeners = this.listeners.get(method) ?? []; + listeners.push(listener); + this.listeners.set(method, listeners); + } + + close() { + this.socket.close(); + } +} + +function formatConsoleArgument(argument) { + if ('value' in argument) return String(argument.value); + if (argument.description) return argument.description; + return argument.type; +} + +async function createPage(cdpBase) { + const response = await fetch(`${cdpBase}/json/new?about:blank`, { method: 'PUT' }); + if (!response.ok) throw new Error(`Unable to create CDP page: ${response.status}`); + const target = await response.json(); + return { + client: new CdpClient(target.webSocketDebuggerUrl), + targetId: target.id, + }; +} + +async function waitForApi(client) { + for (let attempt = 0; attempt < 300; attempt++) { + const response = await client.call('Runtime.evaluate', { + expression: 'window.__UPSCALER_BENCH__?.ready === true', + returnByValue: true, + }); + if (response.result.value === true) return; + await new Promise((resolveWait) => setTimeout(resolveWait, 100)); + } + throw new Error('Timed out waiting for window.__UPSCALER_BENCH__.'); +} + +async function evaluate(client, expression) { + const response = await client.call('Runtime.evaluate', { + expression, + awaitPromise: true, + returnByValue: true, + }); + if (response.exceptionDetails) + throw new Error(response.exceptionDetails.exception?.description ?? 'Browser evaluation failed.'); + return response.result.value; +} + +async function navigate(client, url, logRecords) { + logRecords.length = 0; + await client.call('Page.navigate', { url }); + await waitForApi(client); +} + +function runUrl(baseUrl, options, variant, ratio, scenario = 'Q1', subrun = null) { + const url = new URL(baseUrl); + url.searchParams.set('experiment', options.experiment ?? 'E00'); + url.searchParams.set('benchMode', options.mode); + url.searchParams.set('variant', variant); + url.searchParams.set('comparison', options.comparison); + url.searchParams.set('ratio', String(ratio)); + url.searchParams.set('scenario', scenario); + url.searchParams.set('width', String(options.width ?? 1920)); + url.searchParams.set('height', String(options.height ?? 1080)); + url.searchParams.set('warmup', String(options.warmup)); + url.searchParams.set('samples', String(options.samples)); + if (subrun) url.searchParams.set('subrun', subrun); + return url.href; +} + +function assertCleanLogs(records) { + const failures = records.filter( + (record) => + record.channel === 'Runtime.exceptionThrown' || + (record.channel === 'Runtime.consoleAPICalled' && record.level === 'error') || + /device lost|validation|parsing wgsl|invalid (compute|bind|command)/i.test(record.text), + ); + if (failures.length > 0) + throw new Error(`Browser validation failed:\n${failures.map((entry) => entry.text).join('\n')}`); +} + +async function captureCanvas(client) { + await evaluate( + client, + `(async () => { + const device = window.__UPSCALER_BENCH__ && document.querySelector('canvas') + ? window.__UPSCALER_BENCH__ + : null; + if (!device) throw new Error('Benchmark API unavailable before screenshot.'); + return true; + })()`, + ); + const bounds = await evaluate( + client, + `(() => { + const canvas = document.querySelector('canvas'); + if (!canvas) throw new Error('Canvas not found.'); + const rect = canvas.getBoundingClientRect(); + return { x: rect.x, y: rect.y, width: rect.width, height: rect.height }; + })()`, + ); + const screenshot = await client.call('Page.captureScreenshot', { + format: 'png', + fromSurface: true, + captureBeyondViewport: true, + clip: { ...bounds, scale: 1 }, + }); + return { bytes: Buffer.from(screenshot.data, 'base64'), clip: bounds }; +} + +function resolveRoi(roi, width, height) { + return { + x: Math.floor(roi[0] * width), + y: Math.floor(roi[1] * height), + width: Math.ceil(roi[2] * width), + height: Math.ceil(roi[3] * height), + }; +} + +function validateDecodedCapture(decoded, expected) { + const errors = []; + if (decoded.width !== expected.width || decoded.height !== expected.height) + errors.push( + `Dimensions ${decoded.width}x${decoded.height} do not match ${expected.width}x${expected.height}.`, + ); + let nonOpaquePixels = 0; + for (let offset = 3; offset < decoded.rgba.length; offset += 4) + if (decoded.rgba[offset] !== 255) nonOpaquePixels++; + if (nonOpaquePixels > 0) errors.push(`${nonOpaquePixels} pixels have alpha other than 255.`); + return { + width: decoded.width, + height: decoded.height, + bitDepth: decoded.bitDepth, + colorType: decoded.colorType, + nonOpaquePixels, + errors, + passes: errors.length === 0, + }; +} + +function expectedCaptureDimensions(scenario, frame) { + if (scenario === 'Q10' && frame >= 120 && frame <= 179) + return { width: 1280, height: 720 }; + return { width: 1920, height: 1080 }; +} + +function differenceLegend() { + return { + mapping: 'red = max absolute RGB difference; green=0; blue=0; alpha=1', + domain: [0, 1], + exactThreshold: 1 / 255, + rmseThreshold: 0.25 / 255, + }; +} + +function compareRegion(a, b, region) { + let maxByteDifference = 0; + let squaredDifference = 0; + let channels = 0; + for (let y = region.y; y < Math.min(a.height, region.y + region.height); y++) { + for (let x = region.x; x < Math.min(a.width, region.x + region.width); x++) { + const offset = (y * a.width + x) * 4; + for (let channel = 0; channel < 3; channel++) { + const difference = Math.abs(a.rgba[offset + channel] - b.rgba[offset + channel]); + maxByteDifference = Math.max(maxByteDifference, difference); + squaredDifference += (difference / 255) ** 2; + channels++; + } + } + } + return { + ...region, + maxAbsolute: maxByteDifference / 255, + rmse: Math.sqrt(squaredDifference / channels), + passes: maxByteDifference <= 1 && Math.sqrt(squaredDifference / channels) <= 0.25 / 255, + }; +} + +function differenceHeatmap(a, b) { + const rgba = Buffer.alloc(a.rgba.length); + for (let offset = 0; offset < rgba.length; offset += 4) { + const difference = Math.max( + Math.abs(a.rgba[offset] - b.rgba[offset]), + Math.abs(a.rgba[offset + 1] - b.rgba[offset + 1]), + Math.abs(a.rgba[offset + 2] - b.rgba[offset + 2]), + ); + rgba[offset] = Math.min(255, difference * 255); + rgba[offset + 1] = 0; + rgba[offset + 2] = 0; + rgba[offset + 3] = 255; + } + return encodePng(a.width, a.height, rgba); +} + +function unorderedPairs(values) { + const pairs = []; + for (let left = 0; left < values.length; left++) + for (let right = left + 1; right < values.length; right++) + pairs.push([values[left], values[right]]); + return pairs; +} + +function timingCsv(timing) { + const rows = ['frame,sequence,label,milliseconds']; + for (const sample of timing.raw) { + let sum = 0; + for (const pass of sample.passes) { + rows.push(`${sample.frameTag},${sample.sequence},${pass.label},${pass.milliseconds}`); + sum += pass.milliseconds; + } + rows.push(`${sample.frameTag},${sample.sequence},compute-sum,${sum}`); + } + return `${rows.join('\n')}\n`; +} + +function relativeDelta(a, b) { + const mean = (a + b) / 2; + return mean === 0 ? 0 : Math.abs(a - b) / mean; +} + +function quantile95(values) { + const sorted = [...values].sort((a, b) => a - b); + const position = 0.95 * (sorted.length - 1); + const lower = Math.floor(position); + const upper = Math.ceil(position); + return sorted[lower] + (sorted[upper] - sorted[lower]) * (position - lower); +} + +function runStatistic(run, label, kind) { + const summary = + label === 'compute-sum' + ? run.result.timing.computeSum + : run.result.timing.passes.find((pass) => pass.label === label); + if (!summary || summary[kind] === null) + throw new Error(`Missing ${kind} statistic for ${label}.`); + return summary[kind]; +} + +function analyzeAbba(runs, repetitions) { + const analyses = []; + const ratios = [...new Set(runs.map((run) => run.ratio))]; + for (const ratio of ratios) { + const ratioRuns = runs.filter((run) => run.ratio === ratio); + // Cross-variant comparisons (production vs candidate) have disjoint pass + // graphs — only labels present in every run can be compared. compute-sum + // always qualifies. + const labels = new Set(['compute-sum']); + const [firstRun, ...restRuns] = ratioRuns; + for (const pass of firstRun.result.timing.passes) { + const everywhere = restRuns.every((run) => + run.result.timing.passes.some((other) => other.label === pass.label), + ); + if (everywhere) labels.add(pass.label); + } + + for (const label of labels) { + const statistics = {}; + for (const kind of ['median', 'p95']) { + const rows = []; + for (let repetition = 1; repetition <= repetitions; repetition++) { + const byPosition = Object.fromEntries( + ratioRuns + .filter((run) => run.repetition === repetition) + .map((run) => [run.position, run]), + ); + const A1 = runStatistic(byPosition.A1, label, kind); + const B1 = runStatistic(byPosition.B1, label, kind); + const B2 = runStatistic(byPosition.B2, label, kind); + const A2 = runStatistic(byPosition.A2, label, kind); + const dA = relativeDelta(A1, A2); + const dB = relativeDelta(B1, B2); + const meanA = (A1 + A2) / 2; + const meanB = (B1 + B2) / 2; + rows.push({ + repetition, + A1, + B1, + B2, + A2, + dA, + dB, + meanA, + meanB, + comparisonDelta: relativeDelta(meanA, meanB), + }); + } + const sortedDeltaA = rows.map((row) => row.dA).sort((a, b) => a - b); + const sortedDeltaB = rows.map((row) => row.dB).sort((a, b) => a - b); + const q95A = quantile95(sortedDeltaA); + const q95B = quantile95(sortedDeltaB); + const noiseFloor = Math.max(q95A, q95B); + const comparisonLimit = Math.max(kind === 'median' ? 0.03 : 0.05, 2 * noiseFloor); + const passCount = rows.filter( + (row) => row.comparisonDelta <= comparisonLimit, + ).length; + statistics[kind] = { + rows, + sortedDeltaA, + sortedDeltaB, + q95A, + q95B, + noiseFloor, + comparisonLimit, + passCount, + passes: passCount >= 3, + }; + } + const baselineMedian = statistics.median.rows.reduce( + (sum, row) => sum + row.meanA, + 0, + ) / repetitions; + const timerResolutionLimited = label !== 'compute-sum' && baselineMedian < 0.02; + const noiseFloorPass = + timerResolutionLimited || + (statistics.median.noiseFloor <= 0.015 && + statistics.p95.noiseFloor <= 0.025); + analyses.push({ + ratio, + label, + baselineMedian, + timerResolutionLimited, + individualClaimEligible: + label === 'compute-sum' || (!timerResolutionLimited && noiseFloorPass), + median: statistics.median, + p95: statistics.p95, + noiseFloorPass, + }); + } + } + return analyses; +} + +function noiseFloorPasses(analysis) { + return computeSumNoiseFloorPasses(analysis); +} + +function comparisonPasses(analysis) { + return authoritativeComparisonPasses(analysis); +} + +async function persistPageEvidence(client, outputDirectory, name, logRecords) { + let browserResult = null; + try { + browserResult = await evaluate(client, 'window.__UPSCALER_BENCH__?.result ?? null'); + } catch { + // Navigation or device failure can make the API unavailable; CDP logs remain authoritative. + } + await writeFile( + join(outputDirectory, `${name}-logs.json`), + JSON.stringify({ channels: logRecords, browserValidation: browserResult?.validation ?? [] }, null, 2), + ); +} + +async function performanceRun(client, context) { + const { options, outputDirectory, logRecords } = context; + const ratios = list(options.ratios, '1,1.5,2,3').map(Number); + const repetitions = Number(options.blocks ?? 4); + const sequence = [ + ['A1', options.variant], + ['B1', options.comparison], + ['B2', options.comparison], + ['A2', options.variant], + ]; + if (!options.smoke && (repetitions !== 4 || options.warmup !== 240 || options.samples !== 600)) + throw new Error('Authoritative E00 timing requires 4 blocks, 240 warmup frames, and 600 samples.'); + if (!options.smoke && ratios.join(',') !== '1,1.5,2,3') + throw new Error('Authoritative E00 timing requires ratios 1,1.5,2,3.'); + + async function runRatio(activeClient, activeLogs, ratio, retry) { + const ratioResults = []; + for (let repetition = 0; repetition < repetitions; repetition++) { + for (const [position, variant] of sequence) { + const url = runUrl(DEFAULT_URL, options, variant, ratio); + const name = `${retry ? 'retry-' : ''}ratio-${ratio}-r${repetition + 1}-${position}-${variant}`; + let result = null; + try { + await navigate(activeClient, url, activeLogs); + result = await evaluate( + activeClient, + `window.__UPSCALER_BENCH__.run({ + warmupFrames: ${options.warmup}, + sampleFrames: ${options.samples} + })`, + ); + if (result?.variant?.id !== variant) + throw new Error( + `Run ${name} reported variant ${result?.variant?.id ?? ''}, expected ${variant}.`, + ); + await writeFile(join(outputDirectory, `${name}.json`), JSON.stringify(result, null, 2)); + if (result.timing) + await writeFile(join(outputDirectory, `${name}.csv`), timingCsv(result.timing)); + } finally { + await persistPageEvidence(activeClient, outputDirectory, name, activeLogs); + } + assertCleanLogs(activeLogs); + if ( + !result?.timing || + result.timing.expectedFrames !== options.samples || + result.timing.receivedFrames !== options.samples || + result.timing.raw.length !== options.samples || + result.timing.invalidityCount !== 0 + ) + throw new Error(`Run ${name} did not produce an exact fresh timing set.`); + ratioResults.push({ ratio, repetition: repetition + 1, position, variant, result }); + } + } + return ratioResults; + } + + const allResults = []; + const analyses = []; + let activeClient = client; + let activeLogs = logRecords; + for (const ratio of ratios) { + let ratioResults = await runRatio(activeClient, activeLogs, ratio, false); + let ratioAnalysis = analyzeAbba(ratioResults, repetitions); + let retryState = 'not-required'; + if (!noiseFloorPasses(ratioAnalysis) && !options.smoke) { + retryState = 'cold-retry'; + const restarted = await context.restartBrowser(); + activeClient = restarted.client; + activeLogs = restarted.logRecords; + ratioResults = await runRatio(activeClient, activeLogs, ratio, true); + ratioAnalysis = analyzeAbba(ratioResults, repetitions); + if (!noiseFloorPasses(ratioAnalysis)) { + retryState = 'blocked-after-retry'; + await writeFile( + join(outputDirectory, `ratio-${ratio}-blocked.json`), + JSON.stringify( + { ratio, status: 'BLOCKED', retryState, analysis: ratioAnalysis }, + null, + 2, + ), + ); + throw new BlockedError( + `E00 timing noise floor remained too high at ratio ${ratio} after one cold retry.`, + ); + } + retryState = 'passed-after-retry'; + } + if (!comparisonPasses(ratioAnalysis) && !options.smoke) { + await writeFile( + join(outputDirectory, `ratio-${ratio}-failed.json`), + JSON.stringify( + { ratio, status: 'FAIL', reason: 'baseline-equivalence', analysis: ratioAnalysis }, + null, + 2, + ), + ); + throw new Error(`E00 baseline equivalence failed at ratio ${ratio}.`); + } + allResults.push(...ratioResults); + analyses.push(...ratioAnalysis.map((entry) => ({ ...entry, retryState }))); + } + await writeFile(join(outputDirectory, 'abba-analysis.json'), JSON.stringify(analyses, null, 2)); + return { runs: allResults, analysis: analyses }; +} + +function scenarioSubruns(scenario) { + if (scenario.id === 'Q6') return ['gtao', 'ssr', 'ssgi']; + if (scenario.id === 'Q8') return ['builtin', 'spatial', 'recurrent']; + return [null]; +} + +function captureFrames(expressions, period) { + return [...new Set(expressions.map((expression) => { + if (/^\d+$/.test(expression)) return Number(expression); + if (expression === 'P') return period; + if (expression === 'P-1') return period - 1; + if (expression === '2*P-1') return 2 * period - 1; + throw new Error(`Unsupported capture expression: ${expression}`); + }))]; +} + +function isHumanReviewTuple(manifest, record) { + const spec = + manifest.capture_protocol.harness_acceptance_matrix.human_review_scenarios[record.scenario]; + if (!spec) return false; + const period = + manifest.capture_protocol.jitter_period_by_ratio[String(record.ratio)]; + return ( + captureFrames(spec.frames, period).includes(record.frame) && + spec.debug_views.includes(record.debugView) + ); +} + +function normalizeRubricTemplate(records, manifest, includeAll = false) { + const groups = new Map(); + for (const record of records) { + if (!includeAll && !isHumanReviewTuple(manifest, record)) continue; + const key = [ + record.scenario, + record.subrun ?? 'default', + record.ratio, + record.frame, + record.debugView, + ].join('|'); + const group = groups.get(key) ?? []; + group.push(record); + groups.set(key, group); + } + + const reviewerCount = + manifest.capture_protocol.harness_acceptance_matrix.reviewer_count; + return [...groups.entries()].map(([key, group]) => { + const full = group.find((record) => record.roi === 'full') ?? group[0]; + return { + ...full, + id: `${key}|blinded-A-B|full-and-declared`, + roi: 'full-and-declared', + roiBounds: undefined, + inspectionRois: Object.fromEntries( + group.map((record) => [record.roi, record.roiBounds]), + ), + reviewerGrades: Array(reviewerCount).fill(null), + }; + }); +} + +function reviewHtml(records, binding, title = 'E00 blinded capture review') { + const data = JSON.stringify(records).replaceAll('<', '\\u003c'); + const safeTitle = String(title).replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>'); + const storageKey = reviewStorageKey(binding, records); + return ` + + + + +${safeTitle} + + + +
+ ${safeTitle} + + + + +
+
+
+
+

Left

Blinded left capture
+

Right

Blinded right capture
+
+

+
+ Grade +
+ Notes + +
+
+
+ + + Keys: 0–4 grade and advance · ←/→ navigate +
+ + +`; +} + +async function validateReviewerRubric(template, rubricPath, outputDirectory, manifest) { + const reviewedRubric = JSON.parse(await readFile(resolve(rubricPath), 'utf8')); + if (!Array.isArray(reviewedRubric) || reviewedRubric.length !== template.length) + throw new UserDecisionRequired('Reviewer rubric is missing required records.'); + const expectedRecords = new Map(template.map((record) => [record.id, record])); + if (expectedRecords.size !== template.length) + throw new Error('Generated reviewer rubric contains duplicate record IDs.'); + const reviewedIds = new Set(); + const reviewerCount = + manifest.capture_protocol.harness_acceptance_matrix.reviewer_count; + for (const record of reviewedRubric) { + const grades = record.reviewerGrades; + const expected = expectedRecords.get(record.id); + if ( + !expected || + reviewedIds.has(record.id) || + record.scenario !== expected.scenario || + record.subrun !== expected.subrun || + record.ratio !== expected.ratio || + record.frame !== expected.frame || + record.debugView !== expected.debugView || + record.roi !== expected.roi || + record.pairKind !== expected.pairKind || + JSON.stringify(record.blindPair) !== JSON.stringify(expected.blindPair) || + JSON.stringify(record.inspectionRois) !== JSON.stringify(expected.inspectionRois) || + !Array.isArray(grades) || + grades.length !== reviewerCount || + grades.some((grade) => !Number.isInteger(grade) || grade < 0 || grade > 4) + ) + throw new UserDecisionRequired( + `Reviewer rubric record ${record.id ?? ''} is incomplete.`, + ); + reviewedIds.add(record.id); + } + if ( + reviewedIds.size !== expectedRecords.size || + [...expectedRecords.keys()].some((id) => !reviewedIds.has(id)) + ) + throw new UserDecisionRequired('Reviewer rubric does not cover every required record ID.'); + + const grades = reviewedRubric.flatMap((record) => record.reviewerGrades); + const sorted = grades.toSorted((a, b) => a - b); + const middle = sorted.length / 2; + const median = + sorted.length % 2 === 0 + ? (sorted[middle - 1] + sorted[middle]) / 2 + : sorted[Math.floor(middle)]; + const reviewerPass = grades.every((grade) => grade <= 1) && median === 0; + await writeFile( + join(outputDirectory, 'rubric-reviewed.json'), + JSON.stringify(reviewedRubric, null, 2), + ); + if (!reviewerPass) throw new Error('One or more blinded reviewer rubric gates failed.'); + return { recordCount: reviewedRubric.length, reviewerCount, median, passes: true }; +} + +async function captureRun(client, context, manifest) { + const { options, outputDirectory, logRecords, binding } = context; + const ratios = list(options.ratios, '1,1.5,2,3').map(Number); + const acceptanceMatrix = manifest.capture_protocol.harness_acceptance_matrix.scenarios; + const acceptanceScenarioIds = Object.keys(acceptanceMatrix); + const requiredScenarioIds = manifest.scenarios.required.map((scenario) => scenario.id); + const defaultScenarios = options.smoke ? requiredScenarioIds : acceptanceScenarioIds; + const requested = new Set(list(options.scenarios, defaultScenarios.join(','))); + const variants = [ + ['A', options.variant], + ['B', options.comparison], + ]; + const reloads = Number(options.reloads ?? 5); + const frameOverride = options.frames ? list(options.frames, '').map(Number) : null; + const viewOverride = options.views ? list(options.views, '') : null; + const records = []; + const capturesByTuple = new Map(); + const authoritativeCoverage = + reloads === 5 && + ratios.join(',') === manifest.scenarios.ratios.join(',') && + frameOverride === null && + viewOverride === null && + requested.size === acceptanceScenarioIds.length && + acceptanceScenarioIds.every((id) => requested.has(id)); + if (!options.smoke && !authoritativeCoverage) + throw new Error( + `Authoritative capture requires ${acceptanceScenarioIds.join(',')}, all manifest ratios, the frozen E00 acceptance frames/views, and five reloads.`, + ); + + for (const scenario of manifest.scenarios.required.filter((entry) => requested.has(entry.id))) { + const captureSpec = options.smoke ? scenario.captures : acceptanceMatrix[scenario.id]; + if (!captureSpec) + throw new Error(`${scenario.id} is not part of the authoritative E00 capture matrix.`); + for (const subrun of scenarioSubruns(scenario)) { + for (const ratio of ratios) { + for (const [blind, variant] of variants) { + for (let reload = 1; reload <= reloads; reload++) { + const url = runUrl(DEFAULT_URL, options, variant, ratio, scenario.id, subrun); + const pageName = [ + blind, + variant, + `reload-${reload}`, + scenario.id, + subrun, + `ratio-${ratio}`, + ] + .filter(Boolean) + .join('_'); + try { + await navigate(client, url, logRecords); + const initial = await evaluate(client, 'window.__UPSCALER_BENCH__.result'); + if (initial.status === 'unsupported') + throw new Error(`${scenario.id}/${subrun ?? 'default'} is unsupported.`); + if (initial.variant?.id !== variant) + throw new Error( + `${pageName} reported variant ${initial.variant?.id ?? ''}, expected ${variant}.`, + ); + const period = manifest.capture_protocol.jitter_period_by_ratio[String(ratio)]; + for (const frame of frameOverride ?? captureFrames(captureSpec.frames, period)) { + for (const debugView of viewOverride ?? captureSpec.debug_views) { + const capture = await evaluate( + client, + `window.__UPSCALER_BENCH__.capture(${JSON.stringify({ frame, debugView })})`, + ); + const expectedDimensions = expectedCaptureDimensions( + scenario.id, + frame, + ); + const screenshot = await captureCanvas(client); + const png = screenshot.bytes; + const stem = [ + blind, + variant, + `reload-${reload}`, + scenario.id, + subrun, + `ratio-${ratio}`, + `frame-${frame}`, + debugView, + ] + .filter(Boolean) + .join('_'); + const pngPath = join(outputDirectory, `${stem}.png`); + await writeFile(pngPath, png); + let decoded; + try { + decoded = decodePng(png); + } catch (error) { + await writeFile( + join(outputDirectory, `${stem}-validation.json`), + JSON.stringify( + { + blind, + variant, + reload, + ratio, + scenario: scenario.id, + subrun, + capture, + expectedDimensions, + clip: screenshot.clip, + differenceLegend: differenceLegend(), + validation: { + passes: false, + errors: [ + error instanceof Error + ? error.message + : String(error), + ], + }, + }, + null, + 2, + ), + ); + throw error; + } + const validation = validateDecodedCapture( + decoded, + expectedDimensions, + ); + if ( + capture.width !== expectedDimensions.width || + capture.height !== expectedDimensions.height + ) + validation.errors.push( + `Browser metadata ${capture.width}x${capture.height} does not match independent expectation ${expectedDimensions.width}x${expectedDimensions.height}.`, + ); + if ( + screenshot.clip.width !== expectedDimensions.width || + screenshot.clip.height !== expectedDimensions.height + ) + validation.errors.push( + `Canvas clip ${screenshot.clip.width}x${screenshot.clip.height} does not match independent expectation ${expectedDimensions.width}x${expectedDimensions.height}.`, + ); + if (capture.jitterPeriod !== period) + validation.errors.push( + `Active resolver jitter period ${capture.jitterPeriod} does not match manifest P=${period}.`, + ); + validation.passes = validation.errors.length === 0; + const validationRecord = { + blind, + variant, + reload, + ratio, + scenario: scenario.id, + subrun, + capture, + expectedDimensions, + clip: screenshot.clip, + png: { + width: decoded.width, + height: decoded.height, + bitDepth: decoded.bitDepth, + colorType: decoded.colorType, + }, + validation, + differenceLegend: differenceLegend(), + }; + await writeFile( + join(outputDirectory, `${stem}-validation.json`), + JSON.stringify(validationRecord, null, 2), + ); + records.push(validationRecord); + if (!validation.passes) + throw new Error(`Capture validation failed for ${stem}: ${validation.errors.join(' ')}`); + + const tupleKey = [ + scenario.id, + subrun ?? 'default', + ratio, + frame, + debugView, + ].join('|'); + const tuple = capturesByTuple.get(tupleKey) ?? { + key: tupleKey, + scenario: scenario.id, + subrun, + ratio, + frame, + debugView, + rois: scenario.captures.rois, + A: [], + B: [], + }; + tuple[blind].push({ blind, variant, reload, stem, pngPath }); + capturesByTuple.set(tupleKey, tuple); + } + } + assertCleanLogs(logRecords); + } finally { + await persistPageEvidence(client, outputDirectory, pageName, logRecords); + } + } + } + } + } + } + const pairResults = []; + const rubric = []; + for (const tuple of capturesByTuple.values()) { + if (tuple.A.length !== reloads || tuple.B.length !== reloads) + throw new Error(`Capture tuple ${tuple.key} has incomplete reload sets.`); + const loadedCaptures = new Map(); + for (const capture of [...tuple.A, ...tuple.B]) { + const bytes = await readFile(capture.pngPath); + loadedCaptures.set(capture.pngPath, { bytes, decoded: decodePng(bytes) }); + } + const pairs = [ + ...unorderedPairs(tuple.A).map((pair) => ['A-A', pair]), + ...unorderedPairs(tuple.B).map((pair) => ['B-B', pair]), + ...tuple.A.flatMap((a) => tuple.B.map((b) => ['A-B', [a, b]])), + ]; + for (const [kind, [left, right]] of pairs) { + const { bytes: leftBytes, decoded: leftDecoded } = loadedCaptures.get(left.pngPath); + const { bytes: rightBytes, decoded: rightDecoded } = loadedCaptures.get(right.pngPath); + const regions = { + full: { x: 0, y: 0, width: leftDecoded.width, height: leftDecoded.height }, + ...Object.fromEntries( + Object.entries(tuple.rois).map(([name, roi]) => [ + name, + resolveRoi(roi, leftDecoded.width, leftDecoded.height), + ]), + ), + }; + const dimensionsPass = + leftDecoded.width === rightDecoded.width && + leftDecoded.height === rightDecoded.height; + const exactMatch = dimensionsPass && leftDecoded.rgba.equals(rightDecoded.rgba); + const metrics = dimensionsPass + ? Object.fromEntries( + Object.entries(regions).map(([name, region]) => [ + name, + exactMatch + ? { ...region, maxAbsolute: 0, rmse: 0, passes: true } + : compareRegion(leftDecoded, rightDecoded, region), + ]), + ) + : {}; + const passes = dimensionsPass && Object.values(metrics).every((metric) => metric.passes); + const pairStem = `${tuple.key.replaceAll('|', '_')}_${kind}_${left.reload}-${right.reload}`; + const result = { + tuple: tuple.key, + kind, + left: { blind: left.blind, variant: left.variant, reload: left.reload, stem: left.stem }, + right: { blind: right.blind, variant: right.variant, reload: right.reload, stem: right.stem }, + dimensionsPass, + alphaPass: true, + metrics, + thresholds: { maxAbsolute: 1 / 255, rmse: 0.25 / 255 }, + differenceLegend: differenceLegend(), + passes, + }; + if (!passes) { + await writeFile( + join(outputDirectory, `${pairStem}-metrics.json`), + JSON.stringify(result, null, 2), + ); + } + if (!passes && dimensionsPass) + await writeFile( + join(outputDirectory, `${pairStem}-heatmap.png`), + differenceHeatmap(leftDecoded, rightDecoded), + ); + pairResults.push(result); + + if ( + kind === 'A-B' && + left.reload === 1 && + right.reload === 1 && + (options['review-all'] || isHumanReviewTuple(manifest, tuple)) + ) { + const reviewId = createHash('sha256').update(tuple.key).digest('hex').slice(0, 16); + const reverse = (createHash('sha256').update(`${tuple.key}|orientation`).digest()[0] & 1) === 1; + const reviewLeft = reverse ? rightBytes : leftBytes; + const reviewRight = reverse ? leftBytes : rightBytes; + const leftImage = `review-${reviewId}-left.png`; + const rightImage = `review-${reviewId}-right.png`; + await writeFile(join(outputDirectory, leftImage), reviewLeft); + await writeFile(join(outputDirectory, rightImage), reviewRight); + + for (const [roi, roiBounds] of Object.entries(regions)) + rubric.push({ + id: [ + tuple.key, + 'blinded-A-B', + roi, + ].join('|'), + scenario: tuple.scenario, + subrun: tuple.subrun, + ratio: tuple.ratio, + frame: tuple.frame, + debugView: tuple.debugView, + roi, + roiBounds, + blindPair: ['left', 'right'], + pairKind: 'blinded-A-B', + leftReload: left.reload, + rightReload: right.reload, + leftImage, + rightImage, + reviewerGrades: [null, null], + notes: '', + }); + } + } + } + const expectedPairsPerTuple = reloads === 5 ? 45 : reloads * (reloads - 1) + reloads ** 2; + const selectedScenarios = manifest.scenarios.required.filter((scenario) => + requested.has(scenario.id), + ); + const expectedTupleCount = selectedScenarios.reduce( + (total, scenario) => { + const captureSpec = options.smoke ? scenario.captures : acceptanceMatrix[scenario.id]; + return total + + scenarioSubruns(scenario).length * + ratios.reduce( + (ratioTotal, ratio) => + ratioTotal + + captureFrames( + captureSpec.frames, + manifest.capture_protocol.jitter_period_by_ratio[String(ratio)], + ).length * + captureSpec.debug_views.length, + 0, + ); + }, + 0, + ); + const pairKindCounts = Object.fromEntries( + ['A-A', 'B-B', 'A-B'].map((kind) => [ + kind, + pairResults.filter((pair) => pair.kind === kind).length, + ]), + ); + const analysis = { + reloads, + authoritative: !options.smoke, + nonAuthoritativeReason: options.smoke ? 'smoke-mode' : null, + completeProtocol: + authoritativeCoverage && capturesByTuple.size === expectedTupleCount, + tupleCount: capturesByTuple.size, + expectedTupleCount, + pairCount: pairResults.length, + expectedPairsPerTuple, + pairKindCounts, + failedPairCount: pairResults.filter((pair) => !pair.passes).length, + passes: + pairResults.every((pair) => pair.passes) && + pairResults.length === capturesByTuple.size * expectedPairsPerTuple && + (options.smoke || + (capturesByTuple.size === expectedTupleCount && + pairKindCounts['A-A'] === expectedTupleCount * 10 && + pairKindCounts['B-B'] === expectedTupleCount * 10 && + pairKindCounts['A-B'] === expectedTupleCount * 25)), + pairs: pairResults, + }; + const reviewRubric = normalizeRubricTemplate(rubric, manifest, options['review-all']); + await writeFile(join(outputDirectory, 'capture-analysis.json'), JSON.stringify(analysis, null, 2)); + await writeFile( + join(outputDirectory, 'rubric-template.json'), + JSON.stringify(reviewRubric, null, 2), + ); + await writeFile( + join(outputDirectory, 'review.html'), + reviewHtml(reviewRubric, binding, options['review-title']), + ); + if (!analysis.passes && !options['allow-differences']) + throw new Error('One or more capture equivalence pairs failed.'); + if (!options.smoke) { + if (!options.rubric) + throw new UserDecisionRequired( + 'Blinded reviewer grades are required; complete rubric-template.json, then run --review-only with --output and --rubric.', + ); + await validateReviewerRubric(reviewRubric, options.rubric, outputDirectory, manifest); + } + return { + records, + analysis, + rubricTemplate: 'rubric-template.json', + status: options.smoke ? 'non-authoritative-smoke' : 'complete', + }; +} + +async function reviewExistingCapture(outputDirectory, rubricPath, manifest, binding) { + const run = JSON.parse(await readFile(join(outputDirectory, 'run.json'), 'utf8')); + const analysis = JSON.parse( + await readFile(join(outputDirectory, 'capture-analysis.json'), 'utf8'), + ); + assertCaptureEvidenceBinding({ + run, + analysis, + manifestDigest: binding.manifestDigest, + workingTreeDigest: binding.workingTreeDigest, + }); + const templatePath = join(outputDirectory, 'rubric-template.json'); + const existingTemplate = JSON.parse(await readFile(templatePath, 'utf8')); + if (!Array.isArray(existingTemplate)) + throw new Error('Existing rubric-template.json is not an array.'); + const normalizedTemplate = existingTemplate.every((record) => record.inspectionRois) + ? existingTemplate + : normalizeRubricTemplate(existingTemplate, manifest); + if (normalizedTemplate.length === 0) + throw new Error('Existing capture artifacts contain no required human-review tuples.'); + await writeFile(templatePath, JSON.stringify(normalizedTemplate, null, 2)); + await writeFile( + join(outputDirectory, 'review.html'), + reviewHtml(normalizedTemplate, binding), + ); + if (!rubricPath) + return { + status: 'USER_DECISION_REQUIRED', + recordCount: normalizedTemplate.length, + rubricTemplate: 'rubric-template.json', + }; + const validation = await validateReviewerRubric( + normalizedTemplate, + rubricPath, + outputDirectory, + manifest, + ); + return { status: 'complete', rubricTemplate: 'rubric-template.json', validation }; +} + +async function stopChild(child) { + if (!child || child.exitCode !== null || child.signalCode !== null) return; + const exited = new Promise((resolveExit) => child.once('exit', resolveExit)); + child.kill('SIGTERM'); + const graceful = await Promise.race([ + exited.then(() => true), + new Promise((resolveWait) => setTimeout(() => resolveWait(false), 3000)), + ]); + if (graceful) return; + child.kill('SIGKILL'); + await Promise.race([exited, new Promise((resolveWait) => setTimeout(resolveWait, 2000))]); +} + +function collectChildOutput(child) { + const output = { stdout: '', stderr: '' }; + child?.stdout?.on('data', (chunk) => { + output.stdout += chunk.toString(); + }); + child?.stderr?.on('data', (chunk) => { + output.stderr += chunk.toString(); + }); + return output; +} + +async function persistChildOutput(outputDirectory, name, output) { + if (!output) return; + await writeFile( + join(outputDirectory, `${name}-process.json`), + JSON.stringify(output, null, 2), + ); +} + +function attachLogCollection(client, logRecords) { + client.on('Log.entryAdded', ({ entry }) => { + logRecords.push({ + channel: 'Log.entryAdded', + level: entry.level, + text: entry.text, + timestamp: entry.timestamp, + }); + }); + client.on('Runtime.consoleAPICalled', (event) => { + logRecords.push({ + channel: 'Runtime.consoleAPICalled', + level: event.type, + text: event.args.map(formatConsoleArgument).join(' '), + timestamp: event.timestamp, + }); + }); + client.on('Runtime.exceptionThrown', ({ exceptionDetails }) => { + logRecords.push({ + channel: 'Runtime.exceptionThrown', + level: 'error', + text: exceptionDetails.exception?.description ?? exceptionDetails.text, + timestamp: exceptionDetails.timestamp, + }); + }); +} + +async function createBrowserRuntime(cli, cdpBase, port, outputDirectory, name) { + let chrome = null; + let profile = null; + let processOutput = null; + let page = null; + try { + if (!cli.cdp) { + profile = join(tmpdir(), `upscaler-e00-${process.pid}-${Date.now()}`); + chrome = spawn( + chromeExecutable(cli.chrome), + [ + '--headless=new', + '--enable-unsafe-webgpu', + '--disable-background-timer-throttling', + '--disable-renderer-backgrounding', + `--remote-debugging-port=${port}`, + `--user-data-dir=${profile}`, + '--window-size=1920,1080', + '--force-device-scale-factor=1', + 'about:blank', + ], + { stdio: ['ignore', 'pipe', 'pipe'] }, + ); + processOutput = collectChildOutput(chrome); + await waitForUrl(`${cdpBase}/json/version`); + } + page = await createPage(cdpBase); + const { client, targetId } = page; + const logRecords = []; + attachLogCollection(client, logRecords); + await Promise.all([ + client.call('Page.enable'), + client.call('Runtime.enable'), + client.call('Log.enable'), + ]); + return { + chrome, + profile, + client, + targetId, + cdpBase, + userOwnedCdp: Boolean(cli.cdp), + logRecords, + processOutput, + outputDirectory, + name, + }; + } catch (error) { + if (page && cli.cdp) { + try { + await closeExternalCdpTarget(cdpBase, page.targetId); + } catch { + // Preserve the startup error; cleanup failure is secondary here. + } finally { + page.client.close(); + } + } + await stopChild(chrome); + await persistChildOutput(outputDirectory, name, processOutput); + if (profile) await rm(profile, { recursive: true, force: true }); + throw error; + } +} + +async function closeBrowserRuntime(runtime) { + if (!runtime) return; + try { + if (runtime.userOwnedCdp) + await closeExternalCdpTarget(runtime.cdpBase, runtime.targetId); + } finally { + runtime.client.close(); + await stopChild(runtime.chrome); + await persistChildOutput(runtime.outputDirectory, runtime.name, runtime.processOutput); + if (runtime.profile) await rm(runtime.profile, { recursive: true, force: true }); + } +} + +async function main() { + const cli = parseArguments(process.argv.slice(2)); + if (cli.help || cli.h) { + console.log(`Usage: node scripts/run-benchmark.mjs [options] + --mode performance|capture (default performance) + --smoke relax E00 acceptance gates — required for candidate A/B runs + --variant --comparison A/B variant ids (see bench/src/benchmark/variants.ts) + --ratios 1,1.5,2,3 --blocks N --warmup N --samples N timing shape + --scenarios Q0,.. --frames 0,.. --views final,.. --reloads N --allow-differences --review-all capture shape + --output results directory (default bench/results/raw/E00/) + --chrome | --cdp browser selection +Without --smoke this runs the strict E00 baseline acceptance protocol (64 runs, hard noise gates).`); + return; + } + const manifestBytes = await readFile(MANIFEST_PATH); + const manifest = JSON.parse(manifestBytes); + const mode = cli.mode ?? 'performance'; + const smoke = cli.smoke === true || cli.smoke === 'true'; + const baselineRoleA = manifest.timing_protocol.variant_mapping.A; + const baselineRoleB = manifest.timing_protocol.variant_mapping.B; + const requestedVariant = cli.variant ?? 'baseline'; + const requestedComparison = cli.comparison ?? 'baseline'; + if (!smoke && requestedVariant !== 'baseline' && requestedVariant !== baselineRoleA) + throw new Error(`Authoritative E00 role A must be ${baselineRoleA}.`); + if (!smoke && requestedComparison !== 'baseline' && requestedComparison !== baselineRoleB) + throw new Error(`Authoritative E00 role B must be ${baselineRoleB}.`); + const options = { + ...cli, + mode, + experiment: cli.experiment ?? 'E00', + variant: requestedVariant === 'baseline' ? baselineRoleA : requestedVariant, + comparison: + requestedComparison === 'baseline' ? baselineRoleB : requestedComparison, + warmup: Number(cli.warmup ?? 240), + samples: Number(cli.samples ?? 600), + smoke, + }; + if (options.experiment !== 'E00') throw new Error(`Unsupported experiment: ${options.experiment}`); + + const stamp = new Date().toISOString().replaceAll(':', '-'); + const outputDirectory = resolve(cli.output ?? join(ROOT, 'bench/results/raw/E00', stamp)); + const binding = { + manifestDigest: createHash('sha256').update(manifestBytes).digest('hex'), + workingTreeDigest: await currentWorkingTreeDigest(), + }; + if (cli['prepare-review'] || cli['review-only']) { + if (!cli.output) + throw new Error('--prepare-review and --review-only require an existing --output directory.'); + const result = await reviewExistingCapture( + outputDirectory, + cli['review-only'] ? cli.rubric : null, + manifest, + binding, + ); + await writeFile( + join(outputDirectory, 'review-result.json'), + JSON.stringify( + { + ...result, + ...binding, + timestamp: new Date().toISOString(), + }, + null, + 2, + ), + ); + console.log(outputDirectory); + return; + } + + const { stdout: localSha } = await execFileAsync('git', ['rev-parse', 'HEAD'], { cwd: ROOT }); + await mkdir(outputDirectory, { recursive: true }); + await writeFile(join(outputDirectory, 'manifest.json'), manifestBytes); + await writeFile( + join(outputDirectory, 'run.json'), + JSON.stringify( + { + options, + ...binding, + localSha: localSha.trim(), + startedAt: new Date().toISOString(), + }, + null, + 2, + ), + ); + + const port = Number(cli.port ?? 9333); + const cdpBase = cli.cdp ?? `http://127.0.0.1:${port}`; + let server = null; + let serverOutput = null; + let runtime = null; + let browserGeneration = 0; + try { + try { + await waitForUrl(DEFAULT_URL, 1); + } catch { + server = spawn('npm', ['run', 'dev', '--', '--host', '127.0.0.1'], { + cwd: ROOT, + stdio: ['ignore', 'pipe', 'pipe'], + }); + serverOutput = collectChildOutput(server); + await waitForUrl(DEFAULT_URL); + } + + runtime = await createBrowserRuntime( + cli, + cdpBase, + port, + outputDirectory, + `chrome-${browserGeneration++}`, + ); + const browserVersion = await fetch(`${cdpBase}/json/version`).then((response) => response.json()); + await writeFile( + join(outputDirectory, 'browser.json'), + JSON.stringify( + { + product: browserVersion.Browser, + protocolVersion: browserVersion['Protocol-Version'], + userAgent: browserVersion['User-Agent'], + operatingSystem: `${process.platform} ${process.arch}`, + }, + null, + 2, + ), + ); + + const context = { + options, + outputDirectory, + binding, + logRecords: runtime.logRecords, + async restartBrowser() { + if (cli.cdp) + throw new Error('Cold-browser retry is unavailable with a user-owned --cdp session.'); + await closeBrowserRuntime(runtime); + runtime = await createBrowserRuntime( + cli, + cdpBase, + port, + outputDirectory, + `chrome-${browserGeneration++}`, + ); + return runtime; + }, + }; + const results = + mode === 'capture' + ? await captureRun(runtime.client, context, manifest) + : await performanceRun(runtime.client, context); + await writeFile(join(outputDirectory, 'results.json'), JSON.stringify(results, null, 2)); + console.log(outputDirectory); + } catch (error) { + await writeFile( + join(outputDirectory, 'failure.json'), + JSON.stringify( + { + status: + error instanceof UserDecisionRequired + ? 'USER_DECISION_REQUIRED' + : error instanceof BlockedError + ? 'BLOCKED' + : 'FAIL', + message: error instanceof Error ? error.message : String(error), + stack: error instanceof Error ? error.stack : null, + timestamp: new Date().toISOString(), + }, + null, + 2, + ), + ); + throw error; + } finally { + await closeBrowserRuntime(runtime); + await stopChild(server); + await persistChildOutput(outputDirectory, 'vite', serverOutput); + } +} + +await main(); diff --git a/src/UpscalePass.ts b/src/UpscalePass.ts index de2758f..d012f88 100644 --- a/src/UpscalePass.ts +++ b/src/UpscalePass.ts @@ -26,7 +26,7 @@ export interface UpscalePassConfig { * - MRT output count matched to the render-target attachment count (a `count: 2` * target rendered without a velocity output yields black) * - render resolution taken from the upscaler, float depth + half-float color - * - a `NoToneMapping` full-screen present of the display-referred FSR output + * - a full-screen present that uses the renderer's normal output transform * * Use {@link renderScene} for the common single-view case, or {@link draw} + * {@link outputTexture} when you want to present the result yourself (split @@ -80,7 +80,7 @@ export class UpscalePass { return this._rt; } - /** The upscaled result — sample it however you like (already sRGB). */ + /** The upscaled linear/HDR result — sample or post-process before presentation. */ get outputTexture(): THREE.Texture { return this.upscaler.outputTexture; } @@ -156,7 +156,7 @@ export class UpscalePass { ); } - /** Presents {@link outputTexture} full-screen (no re-tonemap). */ + /** Presents {@link outputTexture} using the renderer's output transform. */ present(): void { this._quad.render(this._renderer); } diff --git a/src/Upscaler.ts b/src/Upscaler.ts index b19a82c..4f3aa8c 100644 --- a/src/Upscaler.ts +++ b/src/Upscaler.ts @@ -1,4 +1,10 @@ -import { Matrix4, NoColorSpace, type OrthographicCamera, type PerspectiveCamera } from 'three'; +import { + HalfFloatType, + Matrix4, + NoColorSpace, + type OrthographicCamera, + type PerspectiveCamera, +} from 'three'; import { StorageTexture, type Texture, type WebGPURenderer } from 'three/webgpu'; import { ComputePass } from './internal/ComputePass'; @@ -9,10 +15,33 @@ import { JitterSequence } from './math/jitter'; import { getQualityModeRatio, getRenderResolution } from './math/resolution'; import { ACCUMULATE_SHADER } from './shaders/accumulate'; import { BLIT_SHADER } from './shaders/blit'; +import { + ACCUMULATE_SOURCE_FILTER_SHADER, + ACCUMULATE_SOURCE_STRUCTURAL_SHADER, + EASU_SOURCE_APPROX_SHADER, +} from './shaders/candidateFilters'; +import { + DEBUG_SOURCE_FILTER_SHADER, + DEBUG_SOURCE_RESOLVER_SHADER, + DEBUG_SOURCE_STRUCTURAL_SHADER, +} from './shaders/candidateDebug'; +import { + DEPTH_CLIP_SOURCE_SHADER, + GENERATE_REACTIVE_SOURCE_SHADER, + PREPARE_INPUTS_SOURCE_SHADER, + PREPARE_REACTIVITY_SOURCE_SHADER, +} from './shaders/candidateInputs'; +import { + ACCUMULATE_SOURCE_RESOLVER_SHADER, + EXPOSURE_HISTORY_SOURCE_SHADER, + LUMA_INSTABILITY_SOURCE_SHADER, + LUMA_SPD_SOURCE_SHADER, + SHADING_CHANGE_RESOLVE_SOURCE_SHADER, + SHADING_CHANGE_SPD_SOURCE_SHADER, +} from './shaders/candidateTemporal'; import { FLAG_AUTO_EXPOSURE, FLAG_EXTERNAL_EXPOSURE, - FLAG_INPUT_DISPLAY, FLAG_INPUT_REINHARD, FLAG_LOCKS, FLAG_PERSPECTIVE, @@ -38,6 +67,15 @@ import { } from './types'; type JitterableCamera = PerspectiveCamera | OrthographicCamera; +type CandidateBundleId = + | 'source-filter-bundle-v1' + | 'source-structural-bundle-v1' + | 'source-spd-resolver-bundle-v1'; +type UpscalerInternalOptions = { + renderer: WebGPURenderer; + _rcasShader?: string; + _candidateBundle?: string; +}; /** * FSR3-style upscaler for three's `WebGPURenderer`, implemented as raw WGSL @@ -85,6 +123,8 @@ export class Upscaler { //* Internals private readonly _renderer: WebGPURenderer; + private readonly _rcasShader: string; + private readonly _candidateBundle: CandidateBundleId | null; private _device!: GPUDevice; private _constants!: ConstantsBuffer; private _timer!: GpuTimer; @@ -98,6 +138,11 @@ export class Upscaler { private _exposurePass!: ComputePass; private _generateReactivePass!: ComputePass; private _debugPass!: ComputePass; + private _depthClipPass: ComputePass | null = null; + private _prepareReactivityPass: ComputePass | null = null; + private _shadingSpdPass: ComputePass | null = null; + private _shadingResolvePass: ComputePass | null = null; + private _lumaInstabilityPass: ComputePass | null = null; private _path: UpscalePath = 'temporal'; private _displayWidth = 0; @@ -135,9 +180,42 @@ export class Upscaler { // Render-res target the auto-generated reactive mask is written into when // the caller passes an opaque-only color to diff against the final color. private _reactiveGenerated: GPUTexture | null = null; - - constructor(options: { renderer: WebGPURenderer }) { + // Candidate-only source graph resources. Production never allocates these. + private _reconstructedDepth: GPUBuffer | null = null; + private _inputSignals: GPUTexture | null = null; + private _preparedMasks: GPUTexture | null = null; + private _accumulation: [GPUTexture, GPUTexture] | null = null; + private _newLocks: GPUBuffer | null = null; + private _lumaPyramid: GPUTexture | null = null; + private _shadingPyramid: GPUTexture | null = null; + private _shadingChange: GPUTexture | null = null; + private _lumaHistory: [GPUTexture, GPUTexture] | null = null; + private _lumaInstability: GPUTexture | null = null; + + constructor(options: { renderer: WebGPURenderer }); + constructor(options: UpscalerInternalOptions) { this._renderer = options.renderer; + this._rcasShader = options._rcasShader ?? RCAS_SHADER; + const candidate = options._candidateBundle; + if ( + candidate !== undefined && + candidate !== 'source-filter-bundle-v1' && + candidate !== 'source-structural-bundle-v1' && + candidate !== 'source-spd-resolver-bundle-v1' + ) + throw new Error(`@pmndrs/upscaler: unknown internal candidate bundle ${candidate}.`); + this._candidateBundle = candidate ?? null; + } + + private get _usesStructuralInputs(): boolean { + return ( + this._candidateBundle === 'source-structural-bundle-v1' || + this._candidateBundle === 'source-spd-resolver-bundle-v1' + ); + } + + private get _usesSourceResolver(): boolean { + return this._candidateBundle === 'source-spd-resolver-bundle-v1'; } /** @@ -158,13 +236,161 @@ export class Upscaler { }); this._blitPass = new ComputePass(device, 'blit', BLIT_SHADER); - this._easuPass = new ComputePass(device, 'easu', EASU_SHADER); - this._rcasPass = new ComputePass(device, 'rcas', RCAS_SHADER); - this._reconstructPass = new ComputePass(device, 'reconstruct', RECONSTRUCT_SHADER); - this._accumulatePass = new ComputePass(device, 'accumulate', ACCUMULATE_SHADER); - this._exposurePass = new ComputePass(device, 'exposure', LUMINANCE_PYRAMID_SHADER); - this._generateReactivePass = new ComputePass(device, 'gen-reactive', GENERATE_REACTIVE_SHADER); - this._debugPass = new ComputePass(device, 'debug', DEBUG_SHADER); + this._easuPass = new ComputePass( + device, + 'easu', + this._candidateBundle ? EASU_SOURCE_APPROX_SHADER : EASU_SHADER, + this._candidateBundle + ? { + shaderKey: 'fsr1-source-approx-v1', + assembledChunks: ['constants', 'easu-source-approx'], + } + : {}, + ); + this._rcasPass = new ComputePass(device, 'rcas', this._rcasShader); + this._reconstructPass = new ComputePass( + device, + 'reconstruct', + this._candidateBundle ? PREPARE_INPUTS_SOURCE_SHADER : RECONSTRUCT_SHADER, + this._candidateBundle + ? { + shaderKey: this._usesStructuralInputs + ? 'fsr315-prepare-inputs-structural-v1' + : 'fsr315-prepare-inputs-filter-v1', + assembledChunks: ['constants', 'color', 'depth', 'candidate-depth', 'prepare-inputs'], + constants: { + MOTION_INPUT_AT_DISPLAY_RESOLUTION: 0, + MOTION_CANCEL_JITTER: 0, + MOTION_SCALE_X: 1, + MOTION_SCALE_Y: 1, + PREPARE_STRUCTURAL_SIGNALS: this._usesStructuralInputs ? 1 : 0, + }, + } + : {}, + ); + const accumulateShader = this._usesSourceResolver + ? ACCUMULATE_SOURCE_RESOLVER_SHADER + : this._usesStructuralInputs + ? ACCUMULATE_SOURCE_STRUCTURAL_SHADER + : this._candidateBundle + ? ACCUMULATE_SOURCE_FILTER_SHADER + : ACCUMULATE_SHADER; + this._accumulatePass = new ComputePass(device, 'accumulate', accumulateShader, { + shaderKey: this._candidateBundle + ? this._usesSourceResolver + ? 'fsr315-source-resolver-v1' + : this._usesStructuralInputs + ? 'fsr315-source-filter-structural-v1' + : 'fsr315-source-filter-v1' + : 'baseline:accumulate', + assembledChunks: this._candidateBundle + ? ['constants', 'color', 'tonemap', 'candidate-accumulate'] + : [], + }); + this._exposurePass = new ComputePass( + device, + 'exposure', + this._usesSourceResolver + ? LUMA_SPD_SOURCE_SHADER + : this._candidateBundle + ? EXPOSURE_HISTORY_SOURCE_SHADER + : LUMINANCE_PYRAMID_SHADER, + this._candidateBundle + ? { + shaderKey: this._usesSourceResolver + ? 'fsr315-luma-spd-v1' + : 'fsr315-exposure-history-v1', + assembledChunks: ['constants', 'candidate-exposure'], + } + : {}, + ); + this._generateReactivePass = new ComputePass( + device, + 'gen-reactive', + this._usesStructuralInputs + ? GENERATE_REACTIVE_SOURCE_SHADER + : GENERATE_REACTIVE_SHADER, + this._usesStructuralInputs + ? { + shaderKey: 'fsr315-generate-reactive-policy-v1', + assembledChunks: ['constants', 'generate-reactive-source-policy'], + constants: { + REACTIVE_USE_COMPONENT_MAX: 1, + REACTIVE_APPLY_THRESHOLD: 1, + REACTIVE_BINARY: 0, + REACTIVE_THRESHOLD: 0.04, + REACTIVE_SCALE: 2, + REACTIVE_BINARY_VALUE: 1, + }, + } + : {}, + ); + const debugShader = this._usesSourceResolver + ? DEBUG_SOURCE_RESOLVER_SHADER + : this._usesStructuralInputs + ? DEBUG_SOURCE_STRUCTURAL_SHADER + : this._candidateBundle + ? DEBUG_SOURCE_FILTER_SHADER + : DEBUG_SHADER; + this._debugPass = new ComputePass(device, 'debug', debugShader, { + shaderKey: this._candidateBundle + ? `${this._candidateBundle}:debug-v1` + : 'baseline:debug', + assembledChunks: this._candidateBundle + ? ['constants', 'color', 'candidate-debug'] + : [], + }); + if (this._candidateBundle) { + this._depthClipPass = new ComputePass(device, 'depth-clip', DEPTH_CLIP_SOURCE_SHADER, { + shaderKey: this._usesStructuralInputs + ? 'fsr315-depth-clip-structural-v1' + : 'fsr315-depth-clip-filter-v1', + assembledChunks: ['constants', 'depth', 'candidate-depth', 'depth-clip'], + constants: { + DEPTH_CLIP_MOTION_DIVERGENCE: this._usesStructuralInputs ? 1 : 0, + }, + }); + } + if (this._usesStructuralInputs) { + this._prepareReactivityPass = new ComputePass( + device, + 'prepare-reactivity', + PREPARE_REACTIVITY_SOURCE_SHADER, + { + shaderKey: 'fsr315-prepare-reactivity-v1', + assembledChunks: ['constants', 'prepare-reactivity'], + }, + ); + } + if (this._usesSourceResolver) { + this._shadingSpdPass = new ComputePass( + device, + 'shading-spd', + SHADING_CHANGE_SPD_SOURCE_SHADER, + { + shaderKey: 'fsr315-shading-change-spd-v1', + assembledChunks: ['constants', 'shading-change-spd'], + }, + ); + this._shadingResolvePass = new ComputePass( + device, + 'shading-resolve', + SHADING_CHANGE_RESOLVE_SOURCE_SHADER, + { + shaderKey: 'fsr315-shading-change-resolve-v1', + assembledChunks: ['constants', 'shading-change-resolve'], + }, + ); + this._lumaInstabilityPass = new ComputePass( + device, + 'luma-instability', + LUMA_INSTABILITY_SOURCE_SHADER, + { + shaderKey: 'fsr315-luma-instability-v1', + assembledChunks: ['constants', 'luma-instability'], + }, + ); + } this._jitter = new JitterSequence(this._ratio); this._initialized = true; @@ -236,8 +462,8 @@ export class Upscaler { /** * The upscaled result as a three texture — sample it on a fullscreen - * quad (values are display-referred sRGB; disable further tone mapping - * and output encoding when presenting). + * quad or feed it into later post-processing. Values remain in the + * caller's linear/HDR domain; presentation is the caller's responsibility. */ get outputTexture(): Texture { if (!this._output) { @@ -382,7 +608,7 @@ export class Upscaler { this._easuPass.dispatch(easuPass, easuBindGroup, this._displayWidth, this._displayHeight); easuPass.end(); - //* RCAS — sharpen (input already display-referred) + //* RCAS — sharpen in the caller's color domain const exposureView = this._exposure![0].createView(); if (this.settings.sharpness > 0) { this._encodeRcas(encoder, this._easuOutput!.createView(), exposureView); @@ -399,6 +625,10 @@ export class Upscaler { if (!inputs.depth || !inputs.velocity) { throw new Error('@pmndrs/upscaler: the temporal path requires depth and velocity inputs.'); } + if (this._candidateBundle) { + this._encodeCandidateTemporal(encoder, colorGPU, inputs); + return; + } const depthGPU = getGPUTexture(this._renderer, inputs.depth); const velocityGPU = getGPUTexture(this._renderer, inputs.velocity); @@ -552,6 +782,357 @@ export class Upscaler { } } + private _encodeCandidateTemporal( + encoder: GPUCommandEncoder, + colorGPU: GPUTexture, + inputs: DispatchInputs, + ): void { + const depthGPU = getGPUTexture(this._renderer, inputs.depth!); + const velocityGPU = getGPUTexture(this._renderer, inputs.velocity!); + this._checkMsaa(depthGPU, 'depth'); + this._checkMsaa(velocityGPU, 'velocity'); + const depthView = depthGPU.createView( + depthGPU.format.includes('stencil') ? { aspect: 'depth-only' } : undefined, + ); + + const historyIn = this._history![this._historyIndex]; + const historyOut = this._history![1 - this._historyIndex]; + const locksIn = this._locks![this._historyIndex]; + const locksOut = this._locks![1 - this._historyIndex]; + const exposurePrev = this._exposure![this._historyIndex]; + const exposureCur = this._exposure![1 - this._historyIndex]; + const depthCur = this._dilatedDepth![this._depthIndex]; + const externalConditioning = inputs.exposureTexture + ? getGPUTexture(this._renderer, inputs.exposureTexture).createView() + : this._reactiveDummy!.createView(); + const hostPreExposure = inputs.preExposureTexture + ? getGPUTexture(this._renderer, inputs.preExposureTexture).createView() + : this._reactiveDummy!.createView(); + + // Atomic scatter/lock buffers are frame-transient. GPU clears preserve + // the pass boundary without a CPU upload proportional to resolution. + encoder.clearBuffer(this._reconstructedDepth!); + if (this._newLocks) encoder.clearBuffer(this._newLocks); + + //* Reactive Generation === + let reactiveView: GPUTextureView; + if (inputs.reactive) { + reactiveView = getGPUTexture(this._renderer, inputs.reactive).createView(); + } else if (inputs.reactiveOpaqueColor) { + const opaqueGPU = getGPUTexture(this._renderer, inputs.reactiveOpaqueColor); + const bindGroup = this._generateReactivePass.createBindGroup([ + { buffer: this._constants.buffer }, + opaqueGPU.createView(), + colorGPU.createView(), + this._reactiveGenerated!.createView(), + ]); + const pass = encoder.beginComputePass({ + label: 'upscale-gen-reactive', + timestampWrites: this._timer.passDescriptor('genReactive'), + }); + this._generateReactivePass.dispatch(pass, bindGroup, this._renderWidth, this._renderHeight); + pass.end(); + reactiveView = this._reactiveGenerated!.createView(); + } else { + reactiveView = this._reactiveDummy!.createView(); + } + + //* Prepare Inputs + Atomic Depth Scatter === + const prepareInputsBindGroup = this._reconstructPass.createBindGroup([ + { buffer: this._constants.buffer }, + depthView, + velocityGPU.createView(), + colorGPU.createView(), + { buffer: this._reconstructedDepth! }, + depthCur.createView(), + this._dilatedMotion!.createView(), + this._inputSignals!.createView(), + ]); + const prepareInputsPass = encoder.beginComputePass({ + label: 'upscale-prepare-inputs', + timestampWrites: this._timer.passDescriptor('prepareInputs'), + }); + this._reconstructPass.dispatch( + prepareInputsPass, + prepareInputsBindGroup, + this._renderWidth, + this._renderHeight, + ); + prepareInputsPass.end(); + + //* Exposure State / Luma SPD === + if (this._usesSourceResolver) { + const lumaSpdBindGroup = this._exposurePass.createBindGroup([ + { buffer: this._constants.buffer }, + this._inputSignals!.createView(), + exposurePrev.createView(), + externalConditioning, + hostPreExposure, + exposureCur.createView(), + this._mipView(this._lumaPyramid!, 0), + this._mipView(this._lumaPyramid!, 1), + this._mipView(this._lumaPyramid!, 2), + ]); + const lumaSpdPass = encoder.beginComputePass({ + label: 'upscale-luma-spd', + timestampWrites: this._timer.passDescriptor('lumaSpd'), + }); + this._exposurePass.dispatch( + lumaSpdPass, + lumaSpdBindGroup, + Math.max(1, Math.ceil(this._renderWidth / 2)), + Math.max(1, Math.ceil(this._renderHeight / 2)), + ); + lumaSpdPass.end(); + } else { + const exposureBindGroup = this._exposurePass.createBindGroup([ + { buffer: this._constants.buffer }, + colorGPU.createView(), + this._linearSampler, + exposurePrev.createView(), + exposureCur.createView(), + externalConditioning, + hostPreExposure, + ]); + const exposurePass = encoder.beginComputePass({ + label: 'upscale-exposure-history', + timestampWrites: this._timer.passDescriptor('exposure'), + }); + this._exposurePass.dispatch(exposurePass, exposureBindGroup, 8, 8); + exposurePass.end(); + } + + //* Reconstructed Depth Resolve === + const depthClipBindGroup = this._depthClipPass!.createBindGroup([ + { buffer: this._constants.buffer }, + { buffer: this._reconstructedDepth! }, + depthCur.createView(), + this._dilatedMotion!.createView(), + this._masks!.createView(), + ]); + const depthClipPass = encoder.beginComputePass({ + label: 'upscale-depth-clip', + timestampWrites: this._timer.passDescriptor('depthClip'), + }); + this._depthClipPass!.dispatch( + depthClipPass, + depthClipBindGroup, + this._renderWidth, + this._renderHeight, + ); + depthClipPass.end(); + + //* Signed-Difference Shading SPD === + if (this._usesSourceResolver) { + const lumaHistoryIn = this._lumaHistory![this._historyIndex]; + const shadingSpdBindGroup = this._shadingSpdPass!.createBindGroup([ + { buffer: this._constants.buffer }, + this._inputSignals!.createView(), + lumaHistoryIn.createView(), + this._dilatedMotion!.createView(), + exposureCur.createView(), + exposurePrev.createView(), + this._mipView(this._shadingPyramid!, 0), + this._mipView(this._shadingPyramid!, 1), + this._mipView(this._shadingPyramid!, 2), + ]); + const shadingSpdPass = encoder.beginComputePass({ + label: 'upscale-shading-spd', + timestampWrites: this._timer.passDescriptor('shadingSpd'), + }); + this._shadingSpdPass!.dispatch( + shadingSpdPass, + shadingSpdBindGroup, + Math.max(1, Math.ceil(this._renderWidth / 2)), + Math.max(1, Math.ceil(this._renderHeight / 2)), + ); + shadingSpdPass.end(); + + const shadingResolveBindGroup = this._shadingResolvePass!.createBindGroup([ + { buffer: this._constants.buffer }, + this._shadingPyramid!.createView({ + baseMipLevel: 0, + mipLevelCount: 3, + }), + this._shadingChange!.createView(), + ]); + const shadingResolvePass = encoder.beginComputePass({ + label: 'upscale-shading-resolve', + timestampWrites: this._timer.passDescriptor('shadingResolve'), + }); + this._shadingResolvePass!.dispatch( + shadingResolvePass, + shadingResolveBindGroup, + Math.max(1, Math.ceil(this._renderWidth / 2)), + Math.max(1, Math.ceil(this._renderHeight / 2)), + ); + shadingResolvePass.end(); + } + + //* Prepare Reactivity / Accumulation State === + if (this._usesStructuralInputs) { + const accumulationIn = this._accumulation![this._historyIndex]; + const accumulationOut = this._accumulation![1 - this._historyIndex]; + const compositionView = inputs.transparencyAndComposition + ? getGPUTexture(this._renderer, inputs.transparencyAndComposition).createView() + : this._reactiveDummy!.createView(); + const shadingView = this._usesSourceResolver + ? this._shadingChange!.createView() + : this._reactiveDummy!.createView(); + const prepareReactivityBindGroup = this._prepareReactivityPass!.createBindGroup([ + { buffer: this._constants.buffer }, + this._masks!.createView(), + this._dilatedMotion!.createView(), + this._inputSignals!.createView(), + reactiveView, + compositionView, + accumulationIn.createView(), + shadingView, + this._preparedMasks!.createView(), + accumulationOut.createView(), + { buffer: this._newLocks! }, + ]); + const prepareReactivityPass = encoder.beginComputePass({ + label: 'upscale-prepare-reactivity', + timestampWrites: this._timer.passDescriptor('prepareReactivity'), + }); + this._prepareReactivityPass!.dispatch( + prepareReactivityPass, + prepareReactivityBindGroup, + this._renderWidth, + this._renderHeight, + ); + prepareReactivityPass.end(); + } + + //* Four-Frame Luma Instability === + if (this._usesSourceResolver) { + const lumaHistoryIn = this._lumaHistory![this._historyIndex]; + const lumaHistoryOut = this._lumaHistory![1 - this._historyIndex]; + const instabilityBindGroup = this._lumaInstabilityPass!.createBindGroup([ + { buffer: this._constants.buffer }, + this._inputSignals!.createView(), + this._dilatedMotion!.createView(), + this._preparedMasks!.createView(), + lumaHistoryIn.createView(), + exposureCur.createView(), + exposurePrev.createView(), + lumaHistoryOut.createView(), + this._lumaInstability!.createView(), + ]); + const instabilityPass = encoder.beginComputePass({ + label: 'upscale-luma-instability', + timestampWrites: this._timer.passDescriptor('lumaInstability'), + }); + this._lumaInstabilityPass!.dispatch( + instabilityPass, + instabilityBindGroup, + this._renderWidth, + this._renderHeight, + ); + instabilityPass.end(); + } + + //* Candidate Accumulation === + let accumulateResources: GPUBindingResource[]; + if (this._usesSourceResolver) { + accumulateResources = [ + { buffer: this._constants.buffer }, + colorGPU.createView(), + this._dilatedMotion!.createView(), + this._preparedMasks!.createView(), + historyIn.createView(), + historyOut.createView(), + this._inputSignals!.createView(), + this._lumaInstability!.createView(), + { buffer: this._newLocks! }, + exposureCur.createView(), + exposurePrev.createView(), + ]; + } else if (this._usesStructuralInputs) { + accumulateResources = [ + { buffer: this._constants.buffer }, + colorGPU.createView(), + this._dilatedMotion!.createView(), + this._preparedMasks!.createView(), + historyIn.createView(), + this._linearSampler, + historyOut.createView(), + locksIn.createView(), + locksOut.createView(), + exposureCur.createView(), + exposurePrev.createView(), + ]; + } else { + accumulateResources = [ + { buffer: this._constants.buffer }, + colorGPU.createView(), + this._dilatedMotion!.createView(), + this._masks!.createView(), + historyIn.createView(), + this._linearSampler, + historyOut.createView(), + locksIn.createView(), + locksOut.createView(), + exposureCur.createView(), + reactiveView, + exposurePrev.createView(), + ]; + } + const accumulateBindGroup = this._accumulatePass.createBindGroup(accumulateResources); + const accumulatePass = encoder.beginComputePass({ + label: 'upscale-accumulate-candidate', + timestampWrites: this._timer.passDescriptor('accumulate'), + }); + this._accumulatePass.dispatch( + accumulatePass, + accumulateBindGroup, + this._displayWidth, + this._displayHeight, + ); + accumulatePass.end(); + + //* Output === + if (this.settings.debugView !== DebugView.None) { + const debugMasks = this._usesStructuralInputs + ? this._preparedMasks!.createView() + : this._masks!.createView(); + const debugAuxiliary = this._usesSourceResolver + ? this._lumaInstability!.createView() + : locksOut.createView(); + const debugReactive = this._usesStructuralInputs + ? this._preparedMasks!.createView() + : reactiveView; + const debugBindGroup = this._debugPass.createBindGroup([ + { buffer: this._constants.buffer }, + this._dilatedMotion!.createView(), + debugMasks, + this._inputSignals!.createView(), + historyOut.createView(), + debugAuxiliary, + exposureCur.createView(), + colorGPU.createView(), + debugReactive, + this._outputView(), + ]); + const debugPass = encoder.beginComputePass({ + label: 'upscale-debug', + timestampWrites: this._timer.passDescriptor('output'), + }); + this._debugPass.dispatch( + debugPass, + debugBindGroup, + this._displayWidth, + this._displayHeight, + ); + debugPass.end(); + } else if (this.settings.sharpness > 0) { + this._encodeRcas(encoder, historyOut.createView(), exposureCur.createView()); + } else { + this._encodeBlit(encoder, historyOut.createView(), exposureCur.createView()); + } + } + private _encodeRcas( encoder: GPUCommandEncoder, input: GPUTextureView, @@ -593,6 +1174,10 @@ export class Upscaler { return this._outputGPU!.createView({ baseMipLevel: 0, mipLevelCount: 1 }); } + private _mipView(texture: GPUTexture, level: number): GPUTextureView { + return texture.createView({ baseMipLevel: level, mipLevelCount: 1 }); + } + //* Constants Staging private _baseFlags(): number { @@ -635,8 +1220,7 @@ export class Upscaler { flags |= FLAG_RESET; } if ((camera as PerspectiveCamera).isPerspectiveCamera) flags |= FLAG_PERSPECTIVE; - if (this._path === 'temporal') flags |= FLAG_INPUT_REINHARD; - if (this._path === 'spatial') flags |= FLAG_INPUT_DISPLAY; + if (this._path === 'temporal' && !this._usesSourceResolver) flags |= FLAG_INPUT_REINHARD; if (this.settings.lockThinFeatures) flags |= FLAG_LOCKS; if (this.settings.autoExposure) flags |= FLAG_AUTO_EXPOSURE; if (this.settings.detectShadingChanges) flags |= FLAG_SHADING_CHANGE; @@ -675,6 +1259,7 @@ export class Upscaler { this._output = new StorageTexture(dw, dh); this._output.name = 'upscale-output'; this._output.colorSpace = NoColorSpace; + this._output.type = HalfFloatType; // Texture.generateMipmaps defaults to true, which would make three // allocate a mip chain — storage views must cover exactly one level. this._output.generateMipmaps = false; @@ -715,8 +1300,78 @@ export class Upscaler { this._createTexture('dilated-depth-1', rw, rh, 'r32float'), ]; this._dilatedMotion = this._createTexture('dilated-motion', rw, rh, 'rgba16float'); - this._masks = this._createTexture('masks', rw, rh, 'rgba8unorm'); + this._masks = this._createTexture( + 'masks', + rw, + rh, + this._candidateBundle ? 'rgba16float' : 'rgba8unorm', + ); this._reactiveGenerated = this._createTexture('reactive-gen', rw, rh, 'rgba8unorm'); + + if (this._candidateBundle) { + this._reconstructedDepth = this._device.createBuffer({ + label: 'upscale-reconstructed-depth-atomic', + size: Math.max(4, rw * rh * Uint32Array.BYTES_PER_ELEMENT), + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + this._inputSignals = this._createTexture('input-signals', rw, rh, 'rgba16float'); + } + + if (this._usesStructuralInputs) { + this._preparedMasks = this._createTexture( + 'prepared-masks', + rw, + rh, + 'rgba16float', + ); + this._accumulation = [ + this._createTexture('accumulation-0', rw, rh, 'r32float'), + this._createTexture('accumulation-1', rw, rh, 'r32float'), + ]; + this._newLocks = this._device.createBuffer({ + label: 'upscale-new-locks-atomic', + size: Math.max(4, dw * dh * Uint32Array.BYTES_PER_ELEMENT), + usage: GPUBufferUsage.STORAGE | GPUBufferUsage.COPY_DST, + }); + } + + if (this._usesSourceResolver) { + // Each successive mip uses floor(base / 2). Round the base to + // four so odd render sizes still have room for every ceil-sized + // direct reduction written by the single-dispatch shaders. + const halfWidth = Math.max(4, Math.ceil(rw / 8) * 4); + const halfHeight = Math.max(4, Math.ceil(rh / 8) * 4); + this._lumaPyramid = this._device.createTexture({ + label: 'upscale-luma-spd', + size: { width: halfWidth, height: halfHeight }, + mipLevelCount: 3, + format: 'rgba16float', + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.STORAGE_BINDING, + }); + this._shadingPyramid = this._device.createTexture({ + label: 'upscale-shading-change-spd', + size: { width: halfWidth, height: halfHeight }, + mipLevelCount: 3, + format: 'rgba16float', + usage: GPUTextureUsage.TEXTURE_BINDING | GPUTextureUsage.STORAGE_BINDING, + }); + this._shadingChange = this._createTexture( + 'shading-change', + Math.max(1, Math.ceil(rw / 2)), + Math.max(1, Math.ceil(rh / 2)), + 'r32float', + ); + this._lumaHistory = [ + this._createTexture('luma-history-0', rw, rh, 'rgba16float'), + this._createTexture('luma-history-1', rw, rh, 'rgba16float'), + ]; + this._lumaInstability = this._createTexture( + 'luma-instability', + rw, + rh, + 'r32float', + ); + } } } @@ -750,5 +1405,29 @@ export class Upscaler { this._reactiveDummy = null; this._reactiveGenerated?.destroy(); this._reactiveGenerated = null; + this._reconstructedDepth?.destroy(); + this._reconstructedDepth = null; + this._inputSignals?.destroy(); + this._inputSignals = null; + this._preparedMasks?.destroy(); + this._preparedMasks = null; + if (this._accumulation) { + this._accumulation.forEach((texture) => texture.destroy()); + this._accumulation = null; + } + this._newLocks?.destroy(); + this._newLocks = null; + this._lumaPyramid?.destroy(); + this._lumaPyramid = null; + this._shadingPyramid?.destroy(); + this._shadingPyramid = null; + this._shadingChange?.destroy(); + this._shadingChange = null; + if (this._lumaHistory) { + this._lumaHistory.forEach((texture) => texture.destroy()); + this._lumaHistory = null; + } + this._lumaInstability?.destroy(); + this._lumaInstability = null; } } diff --git a/src/UpscalerNode.ts b/src/UpscalerNode.ts index 8b7f358..ae502b6 100644 --- a/src/UpscalerNode.ts +++ b/src/UpscalerNode.ts @@ -108,9 +108,9 @@ export interface UpscalerNodeOptions { * that composites into a low-res target *outside* the post render still wants * the raw {@link Upscaler}; see `examples/06-screenspace-gi`.) * - * Present the result untouched — set `renderer.toneMapping = NoToneMapping` and - * `renderer.outputColorSpace = LinearSRGBColorSpace` (the examples' bootRenderer - * does this) so the PostProcessing output transform is identity. + * The result remains linear/HDR. When it is the final output node, three's + * render pipeline applies the renderer's configured tone mapping and output + * color-space transform; it can also feed later linear post-processing. */ export class UpscalerNode extends TempNode { readonly isFSR3Node = true; diff --git a/src/internal/ComputePass.ts b/src/internal/ComputePass.ts index 9837f67..17c591b 100644 --- a/src/internal/ComputePass.ts +++ b/src/internal/ComputePass.ts @@ -6,24 +6,50 @@ * the upscaler whenever textures resize or ping-pong, so `dispatch` takes * the group per call instead of caching it here. */ +export interface ComputePassOptions { + /** Compile-time values for WGSL `override` declarations. */ + constants?: Record; + /** Stable shader identity included in benchmark evidence. */ + shaderKey?: string; + /** Ordered TypeScript-assembled WGSL chunk identities. */ + assembledChunks?: readonly string[]; +} + +/** Immutable pipeline construction metadata used by benchmark evidence. */ +export interface ComputePassMetadata { + shaderKey: string; + constants: Readonly>; + assembledChunks: readonly string[]; +} + export class ComputePass { static readonly WORKGROUP_SIZE = 8; readonly label: string; readonly pipeline: GPUComputePipeline; + readonly metadata: ComputePassMetadata; private readonly _device: GPUDevice; - constructor(device: GPUDevice, label: string, code: string) { + constructor(device: GPUDevice, label: string, code: string, options: ComputePassOptions = {}) { this._device = device; this.label = label; + this.metadata = Object.freeze({ + shaderKey: options.shaderKey ?? `baseline:${label}`, + constants: Object.freeze({ ...options.constants }), + assembledChunks: Object.freeze([...(options.assembledChunks ?? [])]), + }); const module = device.createShaderModule({ label: `upscale-${label}`, code }); // 'auto' layout keeps the TS side free of duplicated binding tables — // the WGSL source is the single source of truth for bindings. this.pipeline = device.createComputePipeline({ label: `upscale-${label}`, layout: 'auto', - compute: { module, entryPoint: 'main' }, + compute: { + module, + entryPoint: 'main', + constants: options.constants, + }, }); } diff --git a/src/internal/GpuTimer.ts b/src/internal/GpuTimer.ts index 0d10ef4..4b802f2 100644 --- a/src/internal/GpuTimer.ts +++ b/src/internal/GpuTimer.ts @@ -1,23 +1,47 @@ +/** One fresh, complete timestamp-query sample. */ +export interface GpuTimerFrameSample { + frameTag: number; + sequence: number; + passes: Array<{ label: string; milliseconds: number }>; +} + +interface GpuTimerSlot { + querySet: GPUQuerySet; + resolveBuffer: GPUBuffer; + readBuffer: GPUBuffer; + labels: string[]; + frameTag: number; + sequence: number; + epoch: number; + state: 'idle' | 'encoding' | 'pending'; + pending: Promise | null; +} + /** - * Lightweight GPU pass profiler built on WebGPU timestamp queries. + * Lightweight multi-slot GPU profiler built on WebGPU timestamp queries. * - * Degrades to a no-op when the device lacks `timestamp-query` (three - * requests every adapter feature at init, so if the hardware supports it, - * the device has it). Results resolve asynchronously a few frames behind — - * fine for a bench readout. + * Normal library use remains a graceful no-op without `timestamp-query`. + * Authoritative benchmark mode fails early instead of emitting an invalid + * performance claim. */ export class GpuTimer { readonly enabled: boolean; + private static readonly MAX_PASSES = 16; + private static readonly SLOT_COUNT = 8; + private readonly _device: GPUDevice; - private _querySet: GPUQuerySet | null = null; - private _resolveBuffer: GPUBuffer | null = null; - private _readBuffer: GPUBuffer | null = null; - private _labels: string[] = []; - private _pending = false; + private readonly _slots: GpuTimerSlot[] = []; + private _active: GpuTimerSlot | null = null; private _results = new Map(); - - private static readonly MAX_PASSES = 16; + private _samples: GpuTimerFrameSample[] = []; + private _nextFrameTag: number | null = null; + private _sequence = 0; + private _latestCompletedSequence = -1; + private _epoch = 0; + private _authoritative = false; + private _authoritativeError: Error | null = null; + private _disposed = false; constructor(device: GPUDevice) { this._device = device; @@ -25,39 +49,87 @@ export class GpuTimer { if (!this.enabled) return; const count = GpuTimer.MAX_PASSES * 2; - this._querySet = device.createQuerySet({ - label: 'upscale-timer', - type: 'timestamp', - count, - }); - this._resolveBuffer = device.createBuffer({ - label: 'upscale-timer-resolve', - size: count * 8, - usage: GPUBufferUsage.QUERY_RESOLVE | GPUBufferUsage.COPY_SRC, - }); - this._readBuffer = device.createBuffer({ - label: 'upscale-timer-read', - size: count * 8, - usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ, - }); + for (let index = 0; index < GpuTimer.SLOT_COUNT; index++) { + this._slots.push({ + querySet: device.createQuerySet({ + label: `upscale-timer-${index}`, + type: 'timestamp', + count, + }), + resolveBuffer: device.createBuffer({ + label: `upscale-timer-resolve-${index}`, + size: count * 8, + usage: GPUBufferUsage.QUERY_RESOLVE | GPUBufferUsage.COPY_SRC, + }), + readBuffer: device.createBuffer({ + label: `upscale-timer-read-${index}`, + size: count * 8, + usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ, + }), + labels: [], + frameTag: -1, + sequence: -1, + epoch: 0, + state: 'idle', + pending: null, + }); + } + } + + /** Makes the next sample use an explicit deterministic frame tag. */ + setNextFrameTag(frameTag: number): void { + this._nextFrameTag = frameTag; } - /** Starts a new frame of measurements. */ + /** Enables benchmark-only hard failures for unavailable or dropped timing. */ + setAuthoritative(authoritative: boolean): void { + this._authoritative = authoritative; + if (authoritative && !this.enabled) + throw new Error('Authoritative benchmark timing requires timestamp-query support.'); + } + + /** Starts a new frame without replacing the latest completed interactive result. */ beginFrame(): void { - this._labels = []; + this._active = null; + this._throwAuthoritativeError(); + if (!this.enabled) { + if (this._authoritative) + throw new Error('Authoritative benchmark timing requires timestamp-query support.'); + return; + } + + const slot = this._slots.find((candidate) => candidate.state === 'idle'); + if (!slot) { + if (this._authoritative) + throw new Error('No fresh GPU timestamp readback slot is available.'); + return; + } + + slot.labels = []; + slot.frameTag = this._nextFrameTag ?? this._sequence; + slot.sequence = this._sequence++; + slot.epoch = this._epoch; + slot.state = 'encoding'; + this._nextFrameTag = null; + this._active = slot; } /** - * Returns `timestampWrites` for a labeled compute pass, or undefined - * when timing is unavailable or the read-back is still in flight. + * Returns `timestampWrites` for a labeled compute pass. + * @param label - Stable compute-pass label + * @returns Timestamp writes for the active frame, when available */ passDescriptor(label: string): GPUComputePassTimestampWrites | undefined { - if (!this.enabled || !this._querySet || this._pending) return undefined; - if (this._labels.length >= GpuTimer.MAX_PASSES) return undefined; - const index = this._labels.length; - this._labels.push(label); + const slot = this._active; + if (!slot) return undefined; + if (slot.labels.length >= GpuTimer.MAX_PASSES) { + if (this._authoritative) throw new Error('GPU timer pass capacity exceeded.'); + return undefined; + } + const index = slot.labels.length; + slot.labels.push(label); return { - querySet: this._querySet, + querySet: slot.querySet, beginningOfPassWriteIndex: index * 2, endOfPassWriteIndex: index * 2 + 1, }; @@ -65,46 +137,129 @@ export class GpuTimer { /** Encodes query resolution; call after all passes, before submit. */ resolve(encoder: GPUCommandEncoder): void { - if (!this.enabled || this._pending || this._labels.length === 0) return; - if (!this._querySet || !this._resolveBuffer || !this._readBuffer) return; - const count = this._labels.length * 2; - encoder.resolveQuerySet(this._querySet, 0, count, this._resolveBuffer, 0); - encoder.copyBufferToBuffer(this._resolveBuffer, 0, this._readBuffer, 0, count * 8); + const slot = this._active; + if (!slot || slot.labels.length === 0) return; + const count = slot.labels.length * 2; + encoder.resolveQuerySet(slot.querySet, 0, count, slot.resolveBuffer, 0); + encoder.copyBufferToBuffer(slot.resolveBuffer, 0, slot.readBuffer, 0, count * 8); } - /** Kicks off the async read-back; results appear in {@link timings}. */ + /** Kicks off asynchronous readback into the fresh-sample queue. */ readback(): void { - if (!this.enabled || this._pending || this._labels.length === 0) return; - const readBuffer = this._readBuffer; - if (!readBuffer) return; - const labels = [...this._labels]; - this._pending = true; - readBuffer - .mapAsync(GPUMapMode.READ) + const slot = this._active; + this._active = null; + if (!slot) return; + if (slot.labels.length === 0) { + slot.state = 'idle'; + if (this._authoritative) throw new Error('Authoritative GPU frame contained no timed passes.'); + return; + } + + const labels = [...slot.labels]; + const frameTag = slot.frameTag; + const sequence = slot.sequence; + const epoch = slot.epoch; + const authoritative = this._authoritative; + const byteLength = labels.length * 2 * 8; + slot.state = 'pending'; + slot.pending = slot.readBuffer + .mapAsync(GPUMapMode.READ, 0, byteLength) .then(() => { - const values = new BigUint64Array(readBuffer.getMappedRange()); - for (let i = 0; i < labels.length; i++) { - const ns = Number(values[i * 2 + 1] - values[i * 2]); - this._results.set(labels[i], ns / 1e6); - } - readBuffer.unmap(); + const values = new BigUint64Array(slot.readBuffer.getMappedRange(0, byteLength)); + const passes = labels.map((label, index) => ({ + label, + milliseconds: Number(values[index * 2 + 1] - values[index * 2]) / 1e6, + })); + slot.readBuffer.unmap(); + if (epoch !== this._epoch || this._disposed) return; + + this._samples.push({ frameTag, sequence, passes }); + if (sequence <= this._latestCompletedSequence) return; + this._latestCompletedSequence = sequence; + this._results = new Map(passes.map((pass) => [pass.label, pass.milliseconds])); }) - .catch(() => { - // Device loss/teardown mid-map — timings just stop updating. + .catch((error: unknown) => { + if (authoritative && epoch === this._epoch && !this._disposed) + this._authoritativeError = + error instanceof Error + ? error + : new Error(`Authoritative GPU timestamp readback failed: ${String(error)}`); }) .finally(() => { - this._pending = false; + slot.pending = null; + slot.state = 'idle'; }); } - /** Latest resolved per-pass GPU times in milliseconds. */ + /** Waits until another frame can be timestamped without dropping a sample. */ + async waitForAvailableSlot(): Promise { + this._throwAuthoritativeError(); + if (!this.enabled) { + if (this._authoritative) + throw new Error('Authoritative benchmark timing requires timestamp-query support.'); + return; + } + while (!this._slots.some((slot) => slot.state === 'idle')) { + const pending = this._slots.flatMap((slot) => (slot.pending ? [slot.pending] : [])); + if (pending.length === 0) throw new Error('GPU timer slots are unavailable without readbacks.'); + await Promise.race(pending); + this._throwAuthoritativeError(); + } + } + + /** Waits for the queue and all timestamp readbacks to settle. */ + async drain(): Promise { + if (!this.enabled) return; + await this._device.queue.onSubmittedWorkDone(); + while (this._slots.some((slot) => slot.pending)) { + const pending = this._slots.flatMap((slot) => (slot.pending ? [slot.pending] : [])); + await Promise.all(pending); + } + this._throwAuthoritativeError(); + } + + /** Returns and clears all fresh samples, ordered by submission sequence. */ + takeSamples(): GpuTimerFrameSample[] { + const samples = this._samples.sort((a, b) => a.sequence - b.sequence); + this._samples = []; + return samples; + } + + /** Clears labels/results and invalidates pending samples from an old graph. */ + reset(): void { + this._epoch++; + this._active = null; + this._results = new Map(); + this._samples = []; + this._nextFrameTag = null; + this._latestCompletedSequence = -1; + this._authoritativeError = null; + for (const slot of this._slots) { + if (slot.state === 'encoding') slot.state = 'idle'; + slot.labels = []; + } + } + + /** Latest complete resolved frame, retained for the interactive readout. */ get timings(): ReadonlyMap { return this._results; } + private _throwAuthoritativeError(): void { + if (!this._authoritative || !this._authoritativeError) return; + const cause = this._authoritativeError; + this._authoritativeError = null; + throw new Error('Authoritative GPU timestamp readback failed.', { cause }); + } + dispose(): void { - this._querySet?.destroy(); - this._resolveBuffer?.destroy(); - this._readBuffer?.destroy(); + this._disposed = true; + this._epoch++; + for (const slot of this._slots) { + slot.querySet.destroy(); + slot.resolveBuffer.destroy(); + slot.readBuffer.destroy(); + } + this._slots.length = 0; } } diff --git a/src/shaders/README.md b/src/shaders/README.md index 0e597b1..6b826d0 100644 --- a/src/shaders/README.md +++ b/src/shaders/README.md @@ -10,8 +10,9 @@ Every pass is a WGSL compute module assembled from shared chunks (`common.ts` + - **Color and exposure domains** — the local temporal pipeline multiplies input by the selected local conditioning exposure (auto, fixed, or external), then accumulates in _invertible-tonemap space_ (`c / (1 + max(c))`). Before RCAS it inverse-tonemaps, divides - by the current local exposure, and applies ACES + sRGB through callbacks. FSR Upscaler - 3.1.5 instead keeps three concepts separate: the host's input `preExposure`, + by the current local exposure, and returns to the caller's linear/HDR domain. Tone + mapping and output encoding are integration concerns outside the upscaler. FSR Upscaler + 3.1.5 also keeps three exposure concepts separate: the host's input `preExposure`, `DeltaPreExposure()` for moving reprojected history into the current host pre-exposure domain, and internal/app `Exposure()` for conditioning. It removes only `Exposure()` before storage/output, so the result remains in the same color and host pre-exposure @@ -27,8 +28,8 @@ tap placement, and deringing. The WebGPU port uses native WGSL division and `inverseSqrt`, plus per-tap `textureLoad` calls, instead of AMD's approximation helpers and packed gathers. Those are implementation and profiling differences, not known algorithm gaps. The local path also assumes an -exact-sized input resource and applies the library's display conversion through its load -callback. **Next action — Keep / Benchmark:** keep EASU as the documented FSR1 fallback; +exact-sized input resource and leaves the input color domain unchanged. **Next action — +Keep / Benchmark:** keep EASU as the documented FSR1 fallback; benchmark the math/load variants before changing them, and generalize viewport or output handling only when an integration requires it. @@ -72,6 +73,54 @@ FSR Upscaler 3.1.5 dispatches: `prepare inputs` → `luma SPD` → `shading-change SPD` → `shading change` → `prepare reactivity` → `luma instability` → `accumulate` → optional `RCAS` +### Authored benchmark candidates — not measured or adopted + +The remaining audit solutions now exist as three cumulative, internally selected compiled +graphs. They are registered in the benchmark but have deliberately not been run. Their +presence is not evidence of a quality or performance improvement, and none changes the +default production path: + +1. `source-filter-bundle-v1` + - source-style radial approximate Lanczos2 with adaptive kernel bias for the current + frame; + - full 4×4 bicubic Lanczos history reconstruction and deringing; + - FSR1-style approximate reciprocal/reciprocal-square-root EASU math while retaining the + source 12-load topology as the WebGPU comparison against the exact production shader; + - WebGPU-safe reconstructed-depth scatter through `atomic` storage buffers and an + explicit pass boundary, followed by viewport/depth-scaled disocclusion; + - previous/current conditioning plus host pre-exposure state, with history corrected + into the current domain before filtering. Internal conditioning is removed at output; + host pre-exposure remains in caller-domain linear/HDR color. +2. `source-structural-bundle-v1` + - cumulatively includes the filter/reconstruction graph; + - explicit lower-level motion/depth conventions specialized to three.js defaults; + - farthest depth, current luma, motion divergence, and configurable source reactive + generation (component max or vector length, threshold, scale, binary policy); + - max-dilated application reactivity in the aggressive reset/shading channel; + - T&C plus motion divergence in a distinct softer rectification channel; + - render-resolution accumulation/reset state and atomic transient new-lock preparation. +3. `source-spd-resolver-bundle-v1` + - cumulatively includes both earlier graphs; + - a single-dispatch luma mip chain and a separate signed-difference shading-change mip + chain, plus a three-mip shading resolve; + - persistent four-frame render-resolution luma history/instability; + - coordinated source-style accumulation, dynamic rectification, ridge locks, and state + packing. History alpha is lock lifetime here, not the production resolver's sample + age, so these paths are intentionally separate. + +The mip candidates use direct higher-level reductions from the prepared source instead of +AMD's device-wide atomic SPD counter. This is a legal WebGPU single-dispatch solution with +the required mip signals, but it duplicates source reads at coarse levels; benchmark data +must determine whether that trade is acceptable. Reconstructed depth and new-lock scatter +use storage-buffer atomics because portable WebGPU does not expose the floating-point +storage-texture atomics used by native implementations. + +`DispatchInputs.preExposureTexture` and +`DispatchInputs.transparencyAndComposition` are the only new source-compatible dispatch +inputs. Both are optional; the production fallback ignores them. All reactive generation +and resolve stages remain in caller-domain color with no internal ACES, transfer function, +or other presentation transform. + ### Core recommendation Do not replace the local resolver wholesale or preserve divergences by default. Build a @@ -88,74 +137,43 @@ are in [Parity evaluation plan](#parity-evaluation-plan). #### RCAS bounds and denoise (`rcas.ts`) -- **Current status:** Diverges. The lower-limiter omission and denoise mismatch are likely - incomplete parity; native WGSL division and dispatch layout are separate implementation - differences. -- **Local implementation:** Omits - `lowerLimiterMultiplier = saturate(eL / min(neighbor lumas))` from `hitMin`. Denoise uses - green only, excludes the center from its denoise range, and defaults off. Lobe math uses - native WGSL division, and dispatch is one pixel per invocation in 8×8 workgroups. +- **Current status:** Numeric parity adopted; implementation strategy still differs. +- **Local implementation:** Applies + `lowerLimiterMultiplier = saturate(eL / min(neighbor lumas))`, uses + `L = 0.5R + G + 0.5B`, and includes the center in the denoise range. Denoise remains a + runtime policy and currently defaults off. Lobe math uses native WGSL division, and + dispatch is one pixel per invocation in 8×8 workgroups. - **FSR 3.1.5 behavior:** Applies the green-weighted lower limiter. Denoise uses `L = 0.5R + G + 0.5B`, includes the center in the denoise min/max, and is enabled by the temporal RCAS pass. Center RGB remains excluded from `mn4`, `mx4`, and `hitMax`. The source uses high-precision reciprocals for the lobe bounds, an approximate-medium reciprocal for final normalization, and quad-remapped 16×16 coverage. -- **Why it differs / evidence confidence — Unclear:** No platform requirement has been - identified for the missing lower limiter or different denoise math/default. Local - division matches the source's high-precision intent for the lobe bounds; final - normalization and dispatch layout could affect GPU cost, but no local measurement - currently demonstrates an advantage. - -**Keep the local path** - -- **Pros:** Preserves the current output and opt-in denoise policy. Native WGSL division is - straightforward and retains high-precision lobe-bound behavior. -- **Cons:** Default sharpening and isolated-luma attenuation do not match the source. - Native final normalization and the local dispatch may cost more or less depending on the - GPU; that has not been measured. - -**Adopt FSR parity** - -- **Pros:** Restores the source limiter, denoise response, and temporal default. Reduces an - obvious math divergence before evaluating larger temporal changes. -- **Cons:** The approximate-medium final normalization and quad remapping may behave - differently across WebGPU implementations. Matching those implementation details - without timing evidence could change cost without a quality benefit. +- **Why it differs / evidence confidence — Measured:** E01 found the source limiter and + denoise math effectively tied in total compute and visually safe. The user adopted both + on 2026-07-17. Native final normalization and local dispatch remain unmeasured + implementation choices. -**Next action:** **Target parity.** Restore the lower limiter, source denoise luma/range, -and temporal denoise default. Separately benchmark native versus approximate-medium final -normalization and 8×8 per-pixel dispatch versus quad-remapped coverage. +**Decision:** **Adopted.** The production shader now uses the source lower limiter and +denoise luma/range. Re-evaluate the temporal denoise default after the linear/HDR-domain +conversion on representative noisy content. Separately benchmark native versus +approximate-medium final normalization and 8×8 per-pixel dispatch versus quad-remapped +coverage only if RCAS performance becomes material. #### RCAS color domain -- **Current status:** Diverges materially in color space. +- **Current status:** Source-aligned output/color domain. - **Local implementation:** Inverse-tonemaps, divides by the current local exposure, and - applies ACES + sRGB per tap before RCAS. + filters in the caller's linear/HDR domain. It applies no presentation transform. - **FSR 3.1.5 behavior:** Filters color conditioned by `Exposure()`, reverses `Exposure()`, and applies no presentation transform. Host `preExposure` remains, so linear HDR input remains linear HDR output. -- **Why it differs / evidence confidence — Unclear:** Folding presentation into the - compute output produces a directly presentable texture, but no evidence establishes - that this was an intended departure from the original direct port or that it improves - performance. +- **Why it differs / evidence confidence — Resolved:** The fixed per-tap ACES+sRGB transform + was an integration-layer policy inside the upscaler and had no source or platform + justification. The user required its removal on 2026-07-17. -**Keep the local path** - -- **Pros:** Produces the library's current directly presentable output without requiring a - separate caller-managed presentation stage. -- **Cons:** RCAS operates on presentation-transformed neighborhoods, limiting HDR, - alternate tone mapping, gamut choices, and later linear post-processing. - -**Adopt FSR parity** - -- **Pros:** Keeps RCAS and output in the caller's linear color/pre-exposure domain. Improves - composability and makes the temporal color pipeline source-aligned. -- **Cons:** Requires presentation to happen elsewhere. A careless migration could - duplicate or omit the final display transform. - -**Next action:** **Target parity.** Add a linear/HDR resolver output and route it through -the same existing final presentation transform for comparison. Keep directly presentable -output only as an explicit integration mode if it remains useful. +**Decision:** **Adopted.** RCAS and the `rgba16float` output stay in the caller's +linear/HDR domain. Benchmarks and examples apply their chosen renderer presentation only +when drawing to the screen; library users may instead continue linear post-processing. #### Reconstruction and disocclusion (`reconstruct.ts`) @@ -594,31 +612,18 @@ required. #### Output (`rcas.ts`, `blit.ts`) -- **Current status:** Adapted port with a material output-domain divergence. -- **Local implementation:** Fixes output to ACES + sRGB in `rgba8unorm`. +- **Current status:** Source-aligned output ownership. +- **Local implementation:** Writes caller-domain color to `rgba16float` with no tone + mapping, gamut conversion, or transfer function. - **FSR 3.1.5 behavior:** Removes internal/app `Exposure()` while preserving host `preExposure`, returning the caller's input color/pre-exposure domain and leaving presentation to integration. -- **Why it differs / evidence confidence — Likely:** A fixed display transform creates a - directly presentable three texture. This is a plausible integration convenience, not a - measured optimization or evidence that source-domain output was intentionally rejected. - -**Keep the local path** - -- **Pros:** Simple direct presentation and consistent current demo output. -- **Cons:** Prevents alternate tone mapping, gamut/transfer choices, HDR output, and later - linear post-processing. - -**Adopt FSR parity** - -- **Pros:** Preserves caller color semantics and supports composition before one final - presentation transform. -- **Cons:** Requires integrations to own or select the final transform and may require a - higher-precision output resource. +- **Why it differs / evidence confidence — Resolved:** The former fixed ACES+sRGB output + was misplaced presentation policy. It was removed by user decision on 2026-07-17. -**Next action:** **Target parity selectively.** Add a linear/HDR output variant and compare -it through exactly the same final presentation transform as the local output; retain fixed -display output only as an explicit convenience mode. +**Decision:** **Adopted.** The library always exposes linear/HDR output. Integrations own +the final transform; the examples choose three's ACES filmic tone mapping and sRGB output +only as an example presentation policy. #### Raw WGSL and three.js integration @@ -698,8 +703,8 @@ runtime flags, while structural changes need separate resource graphs and pipeli 5. **Temporal stability graph.** Add luma SPD, shading-change SPD, and luma instability. 6. **Coordinated resolver variant.** Evaluate accumulation, rectification, locks, and state packing as one compatible model. -7. **Output domain.** Route a linear/HDR variant through the same final presentation - transform for a fair visual comparison. +7. **Output domain — adopted.** Keep the upscaler linear/HDR and apply presentation only + in the consuming renderer or post-processing graph. #### Measurement protocol @@ -759,16 +764,16 @@ maps it to an auto-exposure target, clamps that target, and eases toward it. Fix than passed through the auto-target clamp. This local conditioning `exposure` should not be conflated with FSR's caller-provided host `preExposure`. `accumulate.ts` applies the local factor before the invertible tonemap; `rcas.ts` or `blit.ts` later divides by the current -factor before the local display transform. +factor before linear/HDR output. This conditions the local accumulation range, but it does not guarantee unchanged final brightness. In particular, stored history is not corrected when exposure changes, so adaptation can cause pumping or trails. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); with it off, `settings.exposure` follows the same local path. Passing `dispatch({ exposureTexture })` overrides both values with the texture's red -channel, but still feeds this local conditioning/history/display path. It is not a way to +channel, but still feeds this local conditioning/history path. It is not a way to declare AMD-style host `preExposure`; using it as one will divide that factor back out -during local output and will not provide `DeltaPreExposure()` history correction. +during output and will not provide `DeltaPreExposure()` history correction. `DebugView.Exposure` visualizes clamped exposed luma, not the selected exposure scalar. ### Custom shading-change heuristic diff --git a/src/shaders/blit.ts b/src/shaders/blit.ts index 144aaae..ceefef4 100644 --- a/src/shaders/blit.ts +++ b/src/shaders/blit.ts @@ -1,31 +1,29 @@ -import { WGSL_CONSTANTS, WGSL_DISPLAY_TRANSFORM, WGSL_TONEMAP } from './common'; +import { WGSL_CONSTANTS, WGSL_TONEMAP } from './common'; import { assembleShader } from './wgsl'; /** * Output/blit pass. * * Samples the input (bilinear — which doubles as the naive-upscale - * comparison mode in the bench), converts to display space, and writes the - * final rgba8unorm output. Input interpretation by flag: + * comparison mode in the bench) and writes in the caller's color domain. + * Input interpretation by flag: * - `FLAG_INPUT_REINHARD` — temporal history in invertible-tonemap space - * - `FLAG_INPUT_DISPLAY` — already display-referred (EASU output), pass through - * - neither — linear HDR scene color (native/bilinear bench modes) + * - otherwise — caller-domain color, passed through unchanged * * Bindings: * - 1: input color (filterable float) * - 2: linear clamp sampler * - 3: exposure, 1×1 (rgba16float; r = pre-exposure to undo on the temporal path) - * - 4: output storage (rgba8unorm, display size) + * - 4: output storage (rgba16float, display size) */ export const BLIT_SHADER = assembleShader( WGSL_CONSTANTS, WGSL_TONEMAP, - WGSL_DISPLAY_TRANSFORM, /* wgsl */ ` @group(0) @binding(1) var inputColor : texture_2d; @group(0) @binding(2) var linearSampler : sampler; @group(0) @binding(3) var exposureTex : texture_2d; -@group(0) @binding(4) var outputColor : texture_storage_2d; +@group(0) @binding(4) var outputColor : texture_storage_2d; @compute @workgroup_size(8, 8) fn main(@builtin(global_invocation_id) gid : vec3u) { @@ -37,9 +35,7 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { if (hasFlag(FLAG_INPUT_REINHARD)) { // Undo the pre-exposure the accumulate pass baked in before tonemapping. let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); - c = displayTransform(tonemapInvert(c) / exposure); - } else if (!hasFlag(FLAG_INPUT_DISPLAY)) { - c = displayTransform(c * C.exposure); + c = tonemapInvert(c) / exposure; } textureStore(outputColor, gid.xy, vec4f(c, 1.0)); diff --git a/src/shaders/candidateDebug.ts b/src/shaders/candidateDebug.ts new file mode 100644 index 0000000..ab636fc --- /dev/null +++ b/src/shaders/candidateDebug.ts @@ -0,0 +1,108 @@ +import { WGSL_COLOR, WGSL_CONSTANTS } from './common'; +import { assembleShader } from './wgsl'; + +function createCandidateDebugShader( + preparedChannels: boolean, + sourceHistoryState: boolean, +): string { + const disocclusion = preparedChannels + ? 'textureLoad(candidateMasks, renderCoord, 0).g' + : 'textureLoad(candidateMasks, renderCoord, 0).r'; + const accumulation = sourceHistoryState + ? 'textureLoad(candidateMasks, renderCoord, 0).a' + : 'textureLoad(historyIn, vec2i(gid.xy), 0).a'; + const locks = sourceHistoryState + ? /* wgsl */ ` + let lock = textureLoad(historyIn, vec2i(gid.xy), 0).a * 0.5; + let instability = textureLoad(auxiliaryState, renderCoord, 0).r; + c = vec3f(lock, max(lock, instability * 0.25), lock); +` + : 'c = vec3f(textureLoad(auxiliaryState, vec2i(gid.xy), 0).r);'; + const shading = preparedChannels + ? 'textureLoad(candidateMasks, renderCoord, 0).b' + : 'textureLoad(auxiliaryState, vec2i(gid.xy), 0).b'; + const reactivity = preparedChannels + ? /* wgsl */ ` + let prepared = textureLoad(candidateMasks, renderCoord, 0); + let direct = textureLoad(reactiveInput, renderCoord, 0).r; + c = vec3f(max(direct, max(prepared.r, prepared.b))); +` + : /* wgsl */ ` + let dimensions = vec2i(textureDimensions(reactiveInput)); + let reactiveCoord = clamp(renderCoord, vec2i(0), dimensions - 1); + c = vec3f(textureLoad(reactiveInput, reactiveCoord, 0).r); +`; + + return assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + /* wgsl */ ` +@group(0) @binding(1) var dilatedMotion : texture_2d; +@group(0) @binding(2) var candidateMasks : texture_2d; +@group(0) @binding(3) var inputSignals : texture_2d; +@group(0) @binding(4) var historyIn : texture_2d; +@group(0) @binding(5) var auxiliaryState : texture_2d; +@group(0) @binding(6) var frameInfo : texture_2d; +@group(0) @binding(7) var inputColor : texture_2d; +@group(0) @binding(8) var reactiveInput : texture_2d; +@group(0) @binding(9) var outputColor : texture_storage_2d; + +fn motionToColor(motion : vec2f) -> vec3f { + let magnitude = clamp(sqrt(length(motion) * 8.0), 0.0, 1.0); + let direction = normalize(select(motion, vec2f(1.0, 0.0), length(motion) < 1.0e-6)); + return vec3f( + 0.5 + 0.5 * direction.x * magnitude, + 0.5 + 0.5 * direction.y * magnitude, + magnitude, + ); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.displaySize)) { return; } + let uv = (vec2f(gid.xy) + 0.5) * C.displaySizeInv; + let renderCoord = clamp(vec2i(uv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + var c = vec3f(0.0); + switch (C.debugMode) { + case 1u: { + c = motionToColor(textureLoad(dilatedMotion, renderCoord, 0).xy); + } + case 2u: { + c = vec3f(${disocclusion}); + } + case 3u: { + let depthMeters = textureLoad(inputSignals, renderCoord, 0).b; + c = vec3f(clamp(log2(1.0 + depthMeters) / 8.0, 0.0, 1.0)); + } + case 4u: { + c = vec3f(${accumulation}); + } + case 5u: { +${locks} + } + case 6u: { + let conditioning = textureLoad(frameInfo, vec2i(0), 0).r; + c = vec3f(clamp(luma(textureLoad(inputColor, renderCoord, 0).rgb) * conditioning, 0.0, 1.0)); + } + case 7u: { + c = vec3f(${shading}); + } + case 8u: { +${reactivity} + } + default: {} + } + textureStore(outputColor, gid.xy, vec4f(c, 1.0)); +} +`, + ); +} + +/** Candidate debug shader for source filters with local state packing. */ +export const DEBUG_SOURCE_FILTER_SHADER = createCandidateDebugShader(false, false); + +/** Candidate debug shader for prepared reactivity with local state packing. */ +export const DEBUG_SOURCE_STRUCTURAL_SHADER = createCandidateDebugShader(true, false); + +/** Candidate debug shader for source resolver state packing. */ +export const DEBUG_SOURCE_RESOLVER_SHADER = createCandidateDebugShader(true, true); diff --git a/src/shaders/candidateFilters.ts b/src/shaders/candidateFilters.ts new file mode 100644 index 0000000..0737805 --- /dev/null +++ b/src/shaders/candidateFilters.ts @@ -0,0 +1,280 @@ +import { EASU_SHADER } from './easu'; +import { WGSL_COLOR, WGSL_CONSTANTS, WGSL_TONEMAP } from './common'; +import { assembleShader } from './wgsl'; + +const EASU_APPROXIMATION_HELPERS = /* wgsl */ ` +// FSR1's low-precision helpers trade exact division for a bit estimate plus +// one Newton step. Keeping this in a separate module makes the ALU experiment +// compile independently from the reviewed production EASU shader. +fn easuApproxRcp(v : f32) -> f32 { + let x = max(v, 1.0e-8); + var estimate = bitcast(0x7ef311c3u - bitcast(x)); + estimate = estimate * (2.0 - x * estimate); + return estimate; +} + +fn easuApproxRsqrt(v : f32) -> f32 { + let x = max(v, 1.0e-12); + var estimate = bitcast(0x5f3759dfu - (bitcast(x) >> 1u)); + estimate = estimate * (1.5 - 0.5 * x * estimate * estimate); + return estimate; +} +`; + +/** + * Source-math EASU candidate using FSR1-style approximate reciprocal helpers. + * The production shader remains byte-identical and available as the fallback. + */ +export const EASU_SOURCE_APPROX_SHADER = EASU_SHADER.replace( + '@group(0) @binding(1) var inputColor', + `${EASU_APPROXIMATION_HELPERS}\n@group(0) @binding(1) var inputColor`, +) + .replaceAll('1.0 / max(lenX, 1.0e-5)', 'easuApproxRcp(max(lenX, 1.0e-5))') + .replaceAll('1.0 / max(lenY, 1.0e-5)', 'easuApproxRcp(max(lenY, 1.0e-5))') + .replaceAll('inverseSqrt(max(dirR, 1.0e-12))', 'easuApproxRsqrt(dirR)') + .replaceAll('1.0 / lob', 'easuApproxRcp(lob)'); + +function createSourceFilterAccumulateShader(structuralInputs: boolean): string { + const candidateBindings = structuralInputs + ? /* wgsl */ ` +@group(0) @binding(9) var exposureCur : texture_2d; +@group(0) @binding(10) var exposurePrev : texture_2d; +` + : /* wgsl */ ` +@group(0) @binding(9) var exposureCur : texture_2d; +@group(0) @binding(10) var reactiveMask : texture_2d; +@group(0) @binding(11) var exposurePrev : texture_2d; +`; + const maskLoad = structuralInputs + ? /* wgsl */ ` + let preparedMasks = textureLoad(masks, renderCoord, 0); + let softReactivity = clamp(preparedMasks.r, 0.0, 1.0); + let disocclusion = clamp(preparedMasks.g, 0.0, 1.0); + let resetReactivity = clamp(preparedMasks.b, 0.0, 1.0); + let reactivity = max(resetReactivity, softReactivity * 0.45); +` + : /* wgsl */ ` + let disocclusion = textureLoad(masks, renderCoord, 0).r; + var reactivity = 0.0; + if (hasFlag(FLAG_REACTIVE)) { + let reactiveSize = vec2i(textureDimensions(reactiveMask)); + let reactiveCoord = clamp(renderCoord, vec2i(0), reactiveSize - 1); + reactivity = clamp(textureLoad(reactiveMask, reactiveCoord, 0).r, 0.0, 1.0); + } + let softReactivity = 0.0; + let resetReactivity = reactivity; +`; + + return assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + WGSL_TONEMAP, + /* wgsl */ ` +@group(0) @binding(1) var inputColor : texture_2d; +@group(0) @binding(2) var dilatedMotion : texture_2d; +@group(0) @binding(3) var masks : texture_2d; +@group(0) @binding(4) var historyIn : texture_2d; +@group(0) @binding(5) var linearSampler : sampler; +@group(0) @binding(6) var historyOut : texture_storage_2d; +@group(0) @binding(7) var locksIn : texture_2d; +@group(0) @binding(8) var locksOut : texture_storage_2d; +${candidateBindings} + +const PI : f32 = 3.14159265358979; +const REACTIVE_STRENGTH : f32 = 0.9; +const LOCK_DECAY : f32 = 0.08; +const LOCK_GROW : f32 = 0.5; +const LOCK_CLAMP_RELAX : f32 = 8.0; +const LOCK_HISTORY_BOOST : f32 = 0.65; + +fn lanczos2(x : f32) -> f32 { + let ax = abs(x); + if (ax < 1.0e-4) { return 1.0; } + if (ax >= 2.0) { return 0.0; } + let px = PI * ax; + return 2.0 * sin(px) * sin(px * 0.5) / (px * px); +} + +// FSR's polynomial Lanczos2 approximation accepts squared radial distance. +fn lanczos2ApproxSq(distanceSquared : f32) -> f32 { + let x2 = min(distanceSquared, 4.0); + let a = (2.0 / 5.0) * x2 - 1.0; + let b = 0.25 * x2 - 1.0; + return ((25.0 / 16.0) * a * a - (9.0 / 16.0)) * b * b; +} + +fn historyLoad(coord : vec2i) -> vec4f { + let maximum = vec2i(C.displaySize) - 1; + return textureLoad(historyIn, clamp(coord, vec2i(0), maximum), 0); +} + +// The source resolver reconstructs history with a full 4x4 bicubic Lanczos +// footprint. Deringing against the central 2x2 prevents negative lobes from +// manufacturing values outside the local history range. +fn sampleHistoryLanczos(uv : vec2f) -> vec4f { + let samplePosition = uv * C.displaySize - 0.5; + let base = vec2i(floor(samplePosition)); + let fraction = fract(samplePosition); + var rows = array(); + var centerMin = vec4f(1.0e6); + var centerMax = vec4f(-1.0e6); + + for (var y = 0; y < 4; y++) { + var row = vec4f(0.0); + var rowWeight = 0.0; + for (var x = 0; x < 4; x++) { + let sample = historyLoad(base + vec2i(x - 1, y - 1)); + let weight = lanczos2(f32(x - 1) - fraction.x); + row += sample * weight; + rowWeight += weight; + if (x >= 1 && x <= 2 && y >= 1 && y <= 2) { + centerMin = min(centerMin, sample); + centerMax = max(centerMax, sample); + } + } + rows[y] = row / max(abs(rowWeight), 1.0e-5); + } + + var result = vec4f(0.0); + var weightSum = 0.0; + for (var y = 0; y < 4; y++) { + let weight = lanczos2(f32(y - 1) - fraction.y); + result += rows[y] * weight; + weightSum += weight; + } + return clamp(result / max(abs(weightSum), 1.0e-5), centerMin, centerMax); +} + +fn historyInCurrentDomain(history : vec3f) -> vec3f { + let previousConditioning = max(textureLoad(exposurePrev, vec2i(0), 0).r, 1.0e-4); + let currentConditioning = max(textureLoad(exposureCur, vec2i(0), 0).r, 1.0e-4); + let previousHost = max(textureLoad(exposurePrev, vec2i(0), 0).b, 1.0e-4); + let currentHost = max(textureLoad(exposureCur, vec2i(0), 0).b, 1.0e-4); + let previousLinear = tonemapInvert(history) / previousConditioning; + return tonemapInvertible(previousLinear * (currentHost / previousHost) * currentConditioning); +} + +fn clipToAABB(center : vec3f, extents : vec3f, color : vec3f) -> vec3f { + let direction = color - center; + let scale = extents / max(abs(direction), vec3f(1.0e-6)); + return center + direction * min(1.0, min(scale.x, min(scale.y, scale.z))); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.displaySize)) { return; } + + let uv = (vec2f(gid.xy) + 0.5) * C.displaySizeInv; + let renderCoord = clamp(vec2i(uv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + let motion = textureLoad(dilatedMotion, renderCoord, 0).xy; +${maskLoad} + let conditioning = max(textureLoad(exposureCur, vec2i(0), 0).r, 1.0e-4); + + //* Radial Current-Frame Reconstruction === + let sourcePosition = uv * C.renderSize - 0.5 - C.jitter; + let sourceBase = vec2i(floor(sourcePosition)); + let maximum = vec2i(C.renderSize) - 1; + let kernelBiasMax = min(1.99, max(C.displaySize.x / C.renderSize.x, 1.0)); + let historyAge = textureSampleLevel(historyIn, linearSampler, uv - motion, 0.0).a; + let biasWeight = min(1.0 - disocclusion * 0.5, clamp(historyAge * 5.0, 0.0, 1.0)); + let kernelBias = mix(max(1.0, (1.0 + kernelBiasMax) * 0.3), kernelBiasMax, biasWeight); + + var colorSum = vec3f(0.0); + var colorWeight = 0.0; + var momentWeight = 0.0; + var momentOne = vec3f(0.0); + var momentTwo = vec3f(0.0); + var neighborhoodMin = vec3f(1.0e6); + var neighborhoodMax = vec3f(-1.0e6); + + for (var y = -1; y <= 1; y++) { + for (var x = -1; x <= 1; x++) { + let tapCoord = sourceBase + vec2i(x, y); + let coord = clamp(tapCoord, vec2i(0), maximum); + let offset = vec2f(tapCoord) - sourcePosition; + let distanceSquared = dot(offset, offset); + let color = tonemapInvertible(max(textureLoad(inputColor, coord, 0).rgb, vec3f(0.0)) * conditioning); + let weight = lanczos2ApproxSq(distanceSquared * kernelBias * kernelBias); + let boxWeight = exp(-2.3 * distanceSquared); + let ycc = rgbToYCoCg(color); + colorSum += color * weight; + colorWeight += weight; + momentOne += ycc * boxWeight; + momentTwo += ycc * ycc * boxWeight; + momentWeight += boxWeight; + neighborhoodMin = min(neighborhoodMin, ycc); + neighborhoodMax = max(neighborhoodMax, ycc); + } + } + + let mean = momentOne / max(momentWeight, 1.0e-5); + let variance = max(momentTwo / max(momentWeight, 1.0e-5) - mean * mean, vec3f(0.0)); + let currentYcc = clamp( + rgbToYCoCg(colorSum / max(abs(colorWeight), 1.0e-5)), + neighborhoodMin, + neighborhoodMax, + ); + let current = max(yCoCgToRgb(currentYcc), vec3f(0.0)); + + //* Exposure-Corrected History === + let previousUv = uv - motion; + let offscreen = any(previousUv < vec2f(0.0)) || any(previousUv > vec2f(1.0)); + if (hasFlag(FLAG_RESET) || offscreen) { + textureStore(historyOut, gid.xy, vec4f(current, 1.0 / C.maxAccumulation)); + textureStore(locksOut, gid.xy, vec4f(0.0)); + return; + } + + let historySample = sampleHistoryLanczos(previousUv); + let history = historyInCurrentDomain(historySample.rgb); + var sampleCount = historySample.a * C.maxAccumulation; + let historyYcc = rgbToYCoCg(history); + let contrast = neighborhoodMax.x - neighborhoodMin.x; + + //* Local Lock Compatibility === + var lockLife = 0.0; + var lockedLuma = currentYcc.x; + if (hasFlag(FLAG_LOCKS)) { + let previousLock = textureSampleLevel(locksIn, linearSampler, previousUv, 0.0); + let peakiness = abs(currentYcc.x - mean.x) / max(sqrt(variance.x), 1.0e-3); + let feature = smoothstep(0.6, 2.0, peakiness) * smoothstep(0.02, 0.12, contrast); + lockedLuma = select(previousLock.g, currentYcc.x, previousLock.r < 0.05); + lockLife = previousLock.r + select(-LOCK_DECAY, LOCK_GROW * feature, feature > 0.1); + let lockChange = smoothstep(max(contrast, 1.0e-3), max(contrast, 1.0e-3) * 2.0, abs(currentYcc.x - lockedLuma)); + lockLife = clamp(lockLife * (1.0 - disocclusion) * (1.0 - lockChange) * (1.0 - reactivity), 0.0, 1.0); + } + + //* Dynamic Rectification === + let velocity4K = length(motion * vec2f(3840.0, 2160.0)); + let boxScaleFactor = max( + clamp(velocity4K / 20.0, 0.0, 1.0), + max(1.0 - historySample.a, sqrt(reactivity)), + ); + let boxScale = mix(3.0, 1.0, boxScaleFactor); + let extents = sqrt(variance) * vec3f(1.7, 1.0, 1.0) * boxScale * (1.0 + lockLife * LOCK_CLAMP_RELAX); + let clippedYcc = clipToAABB(mean, max(extents, vec3f(1.193e-7)), historyYcc); + let rectified = max(yCoCgToRgb(clippedYcc), vec3f(0.0)); + let clipAmount = clamp(length(clippedYcc - historyYcc) / max(length(extents), 1.0e-4), 0.0, 1.0); + + sampleCount *= 1.0 - disocclusion; + sampleCount *= 1.0 - 0.5 * clipAmount * (1.0 - lockLife); + sampleCount *= 1.0 - REACTIVE_STRENGTH * reactivity; + let newCount = min(sampleCount + 1.0, C.maxAccumulation); + let confidence = clamp(1.0 - length(sourcePosition - round(sourcePosition)), 0.25, 1.0); + var alpha = clamp(confidence / newCount, 1.0 / C.maxAccumulation, 1.0); + alpha = max(alpha * (1.0 - LOCK_HISTORY_BOOST * lockLife), 1.0 / (C.maxAccumulation * 2.0)); + alpha = mix(alpha, 1.0, REACTIVE_STRENGTH * reactivity); + let result = mix(rectified, current, alpha); + + textureStore(historyOut, gid.xy, vec4f(result, newCount / C.maxAccumulation)); + textureStore(locksOut, gid.xy, vec4f(lockLife, lockedLuma, resetReactivity, softReactivity)); +} +`, + ); +} + +/** Source filter candidate retaining the local input/resource graph. */ +export const ACCUMULATE_SOURCE_FILTER_SHADER = createSourceFilterAccumulateShader(false); + +/** Source filter candidate consuming the structural prepared-mask channels. */ +export const ACCUMULATE_SOURCE_STRUCTURAL_SHADER = createSourceFilterAccumulateShader(true); diff --git a/src/shaders/candidateInputs.ts b/src/shaders/candidateInputs.ts new file mode 100644 index 0000000..e4bff53 --- /dev/null +++ b/src/shaders/candidateInputs.ts @@ -0,0 +1,392 @@ +import { WGSL_COLOR, WGSL_CONSTANTS, WGSL_DEPTH } from './common'; +import { assembleShader } from './wgsl'; + +/** + * Source-policy reactive generator. Numeric/policy controls are compile-time + * overrides so benchmark variants can specialize without changing bindings. + * Presentation transforms are intentionally absent: both inputs stay in the + * caller's linear/HDR domain. + */ +export const GENERATE_REACTIVE_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +override REACTIVE_USE_COMPONENT_MAX : bool = true; +override REACTIVE_APPLY_THRESHOLD : bool = true; +override REACTIVE_BINARY : bool = false; +override REACTIVE_THRESHOLD : f32 = 0.04; +override REACTIVE_SCALE : f32 = 2.0; +override REACTIVE_BINARY_VALUE : f32 = 1.0; + +@group(0) @binding(1) var opaqueColor : texture_2d; +@group(0) @binding(2) var finalColor : texture_2d; +@group(0) @binding(3) var reactiveOut : texture_storage_2d; + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let coord = vec2i(gid.xy); + let difference = abs( + textureLoad(finalColor, coord, 0).rgb - + textureLoad(opaqueColor, coord, 0).rgb + ); + let componentMaximum = max(difference.r, max(difference.g, difference.b)); + let vectorLength = length(difference); + var value = select(vectorLength, componentMaximum, REACTIVE_USE_COMPONENT_MAX); + value *= REACTIVE_SCALE; + if (REACTIVE_APPLY_THRESHOLD && value < REACTIVE_THRESHOLD) { value = 0.0; } + if (REACTIVE_BINARY && value > 0.0) { value = REACTIVE_BINARY_VALUE; } + textureStore(reactiveOut, coord, vec4f(clamp(value, 0.0, 1.0), 0.0, 0.0, 1.0)); +} +`, +); + +const WGSL_CANDIDATE_DEPTH = /* wgsl */ ` +// WGSL has no isInf(); an f32 far plane past ~3.4e38 can only be +inf. +fn candidateFarIsInfinite(farPlane : f32) -> bool { + return farPlane > 3.0e38; +} + +fn candidateLinearizeDepth(depth : f32) -> f32 { + let nearPlane = C.depthNearFar.x; + let farPlane = C.depthNearFar.y; + if (!hasFlag(FLAG_PERSPECTIVE)) { + let normalized = select(depth, 1.0 - depth, hasFlag(FLAG_REVERSED_DEPTH)); + return select( + nearPlane + normalized * (farPlane - nearPlane), + nearPlane + normalized * 1.0e6, + candidateFarIsInfinite(farPlane), + ); + } + if (candidateFarIsInfinite(farPlane)) { + return select( + nearPlane / max(1.0 - depth, 1.0e-7), + nearPlane / max(depth, 1.0e-7), + hasFlag(FLAG_REVERSED_DEPTH), + ); + } + return linearizeDepth(depth); +} + +// Encoding "nearness" monotonically lets both depth conventions use +// atomicMax and lets commandEncoder.clearBuffer provide the empty sentinel. +fn encodeNearestDepth(depth : f32) -> u32 { + let nearness = select(1.0 - depth, depth, hasFlag(FLAG_REVERSED_DEPTH)); + return bitcast(max(nearness, 0.0)); +} + +fn decodeNearestDepth(encoded : u32) -> f32 { + let nearness = bitcast(encoded); + return select(1.0 - nearness, nearness, hasFlag(FLAG_REVERSED_DEPTH)); +} +`; + +/** + * Source-style prepare-inputs and reconstructed-depth scatter candidate. + * A storage-buffer atomic boundary is used because WebGPU storage textures do + * not provide portable floating-point atomics. + */ +export const PREPARE_INPUTS_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + WGSL_DEPTH, + WGSL_CANDIDATE_DEPTH, + /* wgsl */ ` +override MOTION_INPUT_AT_DISPLAY_RESOLUTION : bool = false; +override MOTION_CANCEL_JITTER : bool = false; +override MOTION_SCALE_X : f32 = 1.0; +override MOTION_SCALE_Y : f32 = 1.0; +override PREPARE_STRUCTURAL_SIGNALS : bool = false; + +struct AtomicDepthBuffer { + values : array>, +} + +@group(0) @binding(1) var sceneDepth : texture_depth_2d; +@group(0) @binding(2) var sceneVelocity : texture_2d; +@group(0) @binding(3) var inputColor : texture_2d; +@group(0) @binding(4) var reconstructedDepth : AtomicDepthBuffer; +@group(0) @binding(5) var dilatedDepth : texture_storage_2d; +@group(0) @binding(6) var dilatedMotion : texture_storage_2d; +@group(0) @binding(7) var inputSignals : texture_storage_2d; + +fn motionLoadCoord(renderCoord : vec2i) -> vec2i { + if (!MOTION_INPUT_AT_DISPLAY_RESOLUTION) { return renderCoord; } + let uv = (vec2f(renderCoord) + 0.5) * C.renderSizeInv; + let dimensions = vec2i(textureDimensions(sceneVelocity)); + return clamp(vec2i(uv * vec2f(dimensions)), vec2i(0), dimensions - 1); +} + +fn scatterDepth(coord : vec2i, encoded : u32) { + if (any(coord < vec2i(0)) || any(coord >= vec2i(C.renderSize))) { return; } + let index = u32(coord.y) * u32(C.renderSize.x) + u32(coord.x); + atomicMax(&reconstructedDepth.values[index], encoded); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let center = vec2i(gid.xy); + let maximum = vec2i(C.renderSize) - 1; + let reversed = hasFlag(FLAG_REVERSED_DEPTH); + var nearestDepth = textureLoad(sceneDepth, center, 0); + var farthestDepth = nearestDepth; + var nearestCoord = center; + + //* Depth Extents === + for (var y = -1; y <= 1; y++) { + for (var x = -1; x <= 1; x++) { + let coord = clamp(center + vec2i(x, y), vec2i(0), maximum); + let depth = textureLoad(sceneDepth, coord, 0); + let nearer = select((depth < nearestDepth), (depth > nearestDepth), reversed); + let farther = select((depth > farthestDepth), (depth < farthestDepth), reversed); + if (nearer) { + nearestDepth = depth; + nearestCoord = coord; + } + if (PREPARE_STRUCTURAL_SIGNALS && farther) { farthestDepth = depth; } + } + } + + //* Motion Convention Adapter === + let velocityCoord = motionLoadCoord(nearestCoord); + var motion = textureLoad(sceneVelocity, velocityCoord, 0).xy; + motion *= C.motionScale * vec2f(MOTION_SCALE_X, MOTION_SCALE_Y); + if (MOTION_CANCEL_JITTER) { + motion -= (C.jitter - C.jitterPrev) * C.renderSizeInv; + } + + let nearestMeters = min(candidateLinearizeDepth(nearestDepth), 65504.0); + let motionPixels4K = length(motion * vec2f(3840.0, 2160.0)); + let motionThreshold = mix(0.25, 0.75, clamp(nearestMeters / 100.0, 0.0, 1.0)); + if (motionPixels4K <= motionThreshold) { motion = vec2f(0.0); } + + //* Atomic Previous-Depth Scatter === + let uv = (vec2f(center) + 0.5) * C.renderSizeInv; + let previousPosition = (uv - motion) * C.renderSize - 0.5; + let base = vec2i(floor(previousPosition)); + let fraction = fract(previousPosition); + let weights = vec4f( + (1.0 - fraction.x) * (1.0 - fraction.y), + fraction.x * (1.0 - fraction.y), + (1.0 - fraction.x) * fraction.y, + fraction.x * fraction.y + ); + let encoded = encodeNearestDepth(nearestDepth); + if (weights.x > 6.1e-4) { scatterDepth(base, encoded); } + if (weights.y > 6.1e-4) { scatterDepth(base + vec2i(1, 0), encoded); } + if (weights.z > 6.1e-4) { scatterDepth(base + vec2i(0, 1), encoded); } + if (weights.w > 6.1e-4) { scatterDepth(base + vec2i(1, 1), encoded); } + + var farthestMeters = nearestMeters; + var currentLuma = 0.0; + if (PREPARE_STRUCTURAL_SIGNALS) { + farthestMeters = min(candidateLinearizeDepth(farthestDepth), 65504.0); + currentLuma = min( + luma(max(textureLoad(inputColor, center, 0).rgb, vec3f(0.0))), + 65504.0, + ); + } + textureStore(dilatedDepth, center, vec4f(nearestDepth, 0.0, 0.0, 0.0)); + textureStore(dilatedMotion, center, vec4f(motion, 0.0, 0.0)); + textureStore(inputSignals, center, vec4f(farthestMeters, currentLuma, nearestMeters, 0.0)); +} +`, +); + +/** + * Source-style disocclusion and motion-divergence candidate consuming the + * synchronized reconstructed-depth buffer. + */ +export const DEPTH_CLIP_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + WGSL_DEPTH, + WGSL_CANDIDATE_DEPTH, + /* wgsl */ ` +override DEPTH_CLIP_MOTION_DIVERGENCE : bool = false; + +struct AtomicDepthBuffer { + values : array>, +} + +@group(0) @binding(1) var reconstructedDepth : AtomicDepthBuffer; +@group(0) @binding(2) var dilatedDepth : texture_2d; +@group(0) @binding(3) var dilatedMotion : texture_2d; +@group(0) @binding(4) var masksOut : texture_storage_2d; + +fn reconstructedLoad(coord : vec2i) -> f32 { + let clamped = clamp(coord, vec2i(0), vec2i(C.renderSize) - 1); + let index = u32(clamped.y) * u32(C.renderSize.x) + u32(clamped.x); + return decodeNearestDepth(atomicLoad(&reconstructedDepth.values[index])); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let coord = vec2i(gid.xy); + let uv = (vec2f(gid.xy) + 0.5) * C.renderSizeInv; + let motion = textureLoad(dilatedMotion, coord, 0).xy; + let previousUv = uv - motion; + if (any(previousUv < vec2f(0.0)) || any(previousUv > vec2f(1.0))) { + textureStore(masksOut, coord, vec4f(1.0, 0.0, 0.0, 0.0)); + return; + } + + //* Viewport/Depth-Scaled Disocclusion === + let samplePosition = previousUv * C.renderSize - 0.5; + let base = vec2i(floor(samplePosition)); + let fraction = fract(samplePosition); + let offsets = array(vec2i(0, 0), vec2i(1, 0), vec2i(0, 1), vec2i(1, 1)); + let weights = vec4f( + (1.0 - fraction.x) * (1.0 - fraction.y), + fraction.x * (1.0 - fraction.y), + (1.0 - fraction.x) * fraction.y, + fraction.x * fraction.y + ); + let currentDepth = candidateLinearizeDepth(textureLoad(dilatedDepth, coord, 0).r); + let halfViewportWidth = length(C.renderSize * 0.5); + var separationConfidence = 0.0; + var weightSum = 0.0; + var potentialDisocclusion = true; + + for (var index = 0; index < 4; index++) { + let weight = weights[index]; + if (weight <= 6.1e-4) { continue; } + let previousRaw = reconstructedLoad(base + offsets[index]); + let previousDepth = candidateLinearizeDepth(previousRaw); + let difference = currentDepth - previousDepth; + potentialDisocclusion = potentialDisocclusion && difference > 1.175e-38; + if (potentialDisocclusion) { + let required = 1.37e-5 * halfViewportWidth * max(currentDepth, previousDepth); + separationConfidence += clamp(required / max(difference, 1.0e-7), 0.0, 1.0) * weight; + weightSum += weight; + } + } + let disocclusion = select( + 0.0, + clamp(1.0 - separationConfidence / max(weightSum, 1.0e-6), 0.0, 1.0), + potentialDisocclusion && weightSum > 0.0, + ); + + //* Motion Divergence === + var motionDivergence = 0.0; + if (DEPTH_CLIP_MOTION_DIVERGENCE) { + let reprojectedCoord = clamp(vec2i(previousUv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + let reprojectedMotion = textureLoad(dilatedMotion, reprojectedCoord, 0).xy; + let reprojectedDepth = candidateLinearizeDepth(textureLoad(dilatedDepth, reprojectedCoord, 0).r); + let velocity = length(motion * vec2f(3840.0, 2160.0)); + let reprojectedVelocity = length(reprojectedMotion * vec2f(3840.0, 2160.0)); + let depthRatio = min(currentDepth, reprojectedDepth) / max(max(currentDepth, reprojectedDepth), 1.0e-6); + motionDivergence = + (1.0 - clamp(reprojectedVelocity / max(velocity, 1.0e-6), 0.0, 1.0)) * + depthRatio * + clamp(velocity / 10.0, 0.0, 1.0); + } + + textureStore(masksOut, coord, vec4f(disocclusion, motionDivergence, 0.0, 0.0)); +} +`, +); + +/** + * Prepare-reactivity candidate. Aggressive application reactivity drives the + * reset/shading channel; T&C and motion divergence remain a softer, + * separately packed rectification signal. + */ +export const PREPARE_REACTIVITY_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +struct AtomicLockBuffer { + values : array>, +} + +@group(0) @binding(1) var depthMotionMasks : texture_2d; +@group(0) @binding(2) var dilatedMotion : texture_2d; +@group(0) @binding(3) var inputSignals : texture_2d; +@group(0) @binding(4) var reactiveMask : texture_2d; +@group(0) @binding(5) var transparencyCompositionMask : texture_2d; +@group(0) @binding(6) var accumulationIn : texture_2d; +@group(0) @binding(7) var shadingChange : texture_2d; +@group(0) @binding(8) var preparedMasks : texture_storage_2d; +@group(0) @binding(9) var accumulationOut : texture_storage_2d; +@group(0) @binding(10) var newLocks : AtomicLockBuffer; + +fn maskLoad(mask : texture_2d, coord : vec2i) -> f32 { + let dimensions = vec2i(textureDimensions(mask)); + let uv = (vec2f(coord) + 0.5) * C.renderSizeInv; + let sampleCoord = clamp(vec2i(uv * vec2f(dimensions)), vec2i(0), dimensions - 1); + return clamp(textureLoad(mask, sampleCoord, 0).r, 0.0, 1.0); +} + +fn currentLuma(coord : vec2i) -> f32 { + return textureLoad(inputSignals, clamp(coord, vec2i(0), vec2i(C.renderSize) - 1), 0).g; +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let coord = vec2i(gid.xy); + let uv = (vec2f(gid.xy) + 0.5) * C.renderSizeInv; + let motion = textureLoad(dilatedMotion, coord, 0).xy; + let depthMotion = textureLoad(depthMotionMasks, coord, 0); + + //* Max-Dilated Application Reactive === + var aggressiveReactive = 0.0; + for (var y = -1; y <= 1; y++) { + for (var x = -1; x <= 1; x++) { + aggressiveReactive = max(aggressiveReactive, maskLoad(reactiveMask, coord + vec2i(x, y))); + } + } + + let composition = maskLoad(transparencyCompositionMask, coord); + let softReactive = max(composition, clamp(depthMotion.g, 0.0, 1.0)); + let shading = max(aggressiveReactive, maskLoad(shadingChange, coord)); + let disocclusion = clamp(depthMotion.r, 0.0, 1.0); + + //* Accumulation Reset Coupling === + let previousUv = uv - motion; + var accumulation = 0.0; + if ( + !hasFlag(FLAG_RESET) && + all(previousUv >= vec2f(0.0)) && + all(previousUv <= vec2f(1.0)) + ) { + let previousCoord = clamp(vec2i(previousUv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + accumulation = textureLoad(accumulationIn, previousCoord, 0).r - 0.333; + } + accumulation = min(accumulation + 1.0 / max(C.maxAccumulation, 1.0), 1.0); + accumulation = mix(accumulation, 0.0, shading); + accumulation = mix(accumulation, min(-0.333, accumulation), disocclusion); + let storedAccumulation = clamp(accumulation + 0.333, 0.0, 1.0); + textureStore(accumulationOut, coord, vec4f(storedAccumulation, 0.0, 0.0, 1.0)); + + //* Ridge Lock Candidate === + var minimumLuma = 1.0e6; + var maximumLuma = 0.0; + let nucleus = currentLuma(coord); + var similarQuadrants = 0u; + for (var y = -1; y <= 1; y++) { + for (var x = -1; x <= 1; x++) { + let sample = currentLuma(coord + vec2i(x, y)); + minimumLuma = min(minimumLuma, sample); + maximumLuma = max(maximumLuma, sample); + if (abs(sample - nucleus) <= 0.1 * max(maximumLuma - minimumLuma, 1.0e-5)) { + similarQuadrants += 1u; + } + } + } + let ridge = (nucleus > maximumLuma - 1.0e-5 || nucleus < minimumLuma + 1.0e-5) && similarQuadrants < 6u; + let lockStrength = select(0.0, 1.0 - minimumLuma / max(maximumLuma, 1.0e-5), ridge); + if (lockStrength > 0.01) { + let displayCoord = clamp( + vec2i(floor((vec2f(coord) + 0.5 - C.jitter) * C.displaySize / C.renderSize)), + vec2i(0), + vec2i(C.displaySize) - 1, + ); + let lockIndex = u32(displayCoord.y) * u32(C.displaySize.x) + u32(displayCoord.x); + atomicMax(&newLocks.values[lockIndex], u32(clamp(lockStrength, 0.0, 1.0) * 65535.0)); + } + + textureStore(preparedMasks, coord, vec4f(softReactive, disocclusion, shading, clamp(accumulation, 0.0, 1.0))); +} +`, +); diff --git a/src/shaders/candidateTemporal.ts b/src/shaders/candidateTemporal.ts new file mode 100644 index 0000000..46f9e56 --- /dev/null +++ b/src/shaders/candidateTemporal.ts @@ -0,0 +1,521 @@ +import { WGSL_COLOR, WGSL_CONSTANTS, WGSL_TONEMAP } from './common'; +import { assembleShader } from './wgsl'; + +/** + * Scalar exposure candidate used by the filter and structural bundles. It + * tracks conditioning and host pre-exposure independently while retaining the + * production metering workload for an attributable A/B. + */ +export const EXPOSURE_HISTORY_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + /* wgsl */ ` +@group(0) @binding(1) var inputColor : texture_2d; +@group(0) @binding(2) var linearSampler : sampler; +@group(0) @binding(3) var previousFrameInfo : texture_2d; +@group(0) @binding(4) var frameInfoOut : texture_storage_2d; +@group(0) @binding(5) var externalConditioning : texture_2d; +@group(0) @binding(6) var hostPreExposure : texture_2d; + +const EXPOSURE_KEY : f32 = 0.18; +const EXPOSURE_MIN : f32 = 0.02; +const EXPOSURE_MAX : f32 = 80.0; +const ADAPT_SPEED : f32 = 2.5; + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.displaySize)) { return; } + if (any(gid.xy != vec2u(0u))) { return; } + var logSum = 0.0; + for (var y = 0u; y < 32u; y++) { + for (var x = 0u; x < 32u; x++) { + let uv = (vec2f(f32(x), f32(y)) + 0.5) / 32.0; + let color = textureSampleLevel(inputColor, linearSampler, uv, 0.0).rgb; + logSum += log2(max(luma(color), 1.0e-4)); + } + } + let averageLuma = exp2(logSum / 1024.0); + let targetConditioning = clamp(EXPOSURE_KEY / max(averageLuma, 1.0e-4), EXPOSURE_MIN, EXPOSURE_MAX); + let previous = textureLoad(previousFrameInfo, vec2i(0), 0); + var conditioning = targetConditioning; + if (!hasFlag(FLAG_RESET) && previous.r > 0.0) { + let adaptation = clamp(1.0 - exp2(-C.deltaTime * ADAPT_SPEED), 0.0, 1.0); + conditioning = mix(previous.r, targetConditioning, adaptation); + } + conditioning = select(C.exposure, conditioning, hasFlag(FLAG_AUTO_EXPOSURE)); + let externalConditioningValue = textureLoad(externalConditioning, vec2i(0), 0).r; + conditioning = select(conditioning, externalConditioningValue, hasFlag(FLAG_EXTERNAL_EXPOSURE)); + let hostValue = textureLoad(hostPreExposure, vec2i(0), 0).r; + let host = select(1.0, hostValue, hostValue > 0.0); + textureStore(frameInfoOut, vec2i(0), vec4f(conditioning, averageLuma, host, log2(max(averageLuma, 1.0e-4)))); +} +`, +); + +/** + * Single-dispatch luma mip candidate. Higher levels are evaluated directly + * from the prepared luma source to avoid cross-workgroup dependencies while + * preserving an SPD-shaped resource and dispatch boundary on WebGPU. + */ +export const LUMA_SPD_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +@group(0) @binding(1) var inputSignals : texture_2d; +@group(0) @binding(2) var previousFrameInfo : texture_2d; +@group(0) @binding(3) var externalConditioning : texture_2d; +@group(0) @binding(4) var hostPreExposure : texture_2d; +@group(0) @binding(5) var frameInfoOut : texture_storage_2d; +@group(0) @binding(6) var lumaMip0 : texture_storage_2d; +@group(0) @binding(7) var lumaMip1 : texture_storage_2d; +@group(0) @binding(8) var lumaMip2 : texture_storage_2d; + +const EXPOSURE_KEY : f32 = 0.18; +const EXPOSURE_MIN : f32 = 0.02; +const EXPOSURE_MAX : f32 = 80.0; +const ADAPT_SPEED : f32 = 2.5; + +fn signalLoad(coord : vec2i) -> vec2f { + let clamped = clamp(coord, vec2i(0), vec2i(C.renderSize) - 1); + let signal = textureLoad(inputSignals, clamped, 0); + return vec2f(max(signal.g, 1.0e-5), signal.r); +} + +fn reduceBlock(origin : vec2i, extent : i32) -> vec2f { + var sum = vec2f(0.0); + var count = 0.0; + for (var y = 0; y < extent; y++) { + for (var x = 0; x < extent; x++) { + sum += signalLoad(origin + vec2i(x, y)); + count += 1.0; + } + } + return sum / count; +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let mip0Size = max(vec2u(1u), vec2u(C.renderSize + 1.0) / 2u); + if (any(gid.xy >= mip0Size)) { return; } + + let coord = vec2i(gid.xy); + let mip0 = reduceBlock(coord * 2, 2); + textureStore(lumaMip0, coord, vec4f(log(max(mip0.x, 1.0e-5)), mip0.x, mip0.y, 0.0)); + + // Direct reductions keep this a legal single dispatch without relying on + // unavailable device-wide barriers between mip levels. + if (all((gid.xy % vec2u(2u)) == vec2u(0u))) { + let mip1 = reduceBlock(coord * 2, 4); + textureStore(lumaMip1, coord / 2, vec4f(log(max(mip1.x, 1.0e-5)), mip1.x, mip1.y, 0.0)); + } + if (all((gid.xy % vec2u(4u)) == vec2u(0u))) { + let mip2 = reduceBlock(coord * 2, 8); + textureStore(lumaMip2, coord / 4, vec4f(log(max(mip2.x, 1.0e-5)), mip2.x, mip2.y, 0.0)); + } + + if (any(gid.xy != vec2u(0u))) { return; } + + //* Frame Exposure State === + var logSum = 0.0; + for (var y = 0; y < 32; y++) { + for (var x = 0; x < 32; x++) { + let sampleCoord = clamp( + vec2i((vec2f(f32(x), f32(y)) + 0.5) * C.renderSize / 32.0), + vec2i(0), + vec2i(C.renderSize) - 1, + ); + logSum += log2(max(textureLoad(inputSignals, sampleCoord, 0).g, 1.0e-4)); + } + } + let averageLuma = exp2(logSum / 1024.0); + let targetConditioning = clamp(EXPOSURE_KEY / max(averageLuma, 1.0e-4), EXPOSURE_MIN, EXPOSURE_MAX); + let previous = textureLoad(previousFrameInfo, vec2i(0), 0); + var conditioning = targetConditioning; + if (!hasFlag(FLAG_RESET) && previous.r > 0.0) { + let adaptation = clamp(1.0 - exp2(-C.deltaTime * ADAPT_SPEED), 0.0, 1.0); + conditioning = mix(previous.r, targetConditioning, adaptation); + } + conditioning = select(C.exposure, conditioning, hasFlag(FLAG_AUTO_EXPOSURE)); + let externalConditioningValue = textureLoad(externalConditioning, vec2i(0), 0).r; + conditioning = select(conditioning, externalConditioningValue, hasFlag(FLAG_EXTERNAL_EXPOSURE)); + + let hostValue = textureLoad(hostPreExposure, vec2i(0), 0).r; + let host = select(1.0, hostValue, hostValue > 0.0); + textureStore(frameInfoOut, vec2i(0), vec4f(conditioning, averageLuma, host, log2(max(averageLuma, 1.0e-4)))); +} +`, +); + +/** + * Signed current/previous luma-difference pyramid. Previous luma is corrected + * into the current host pre-exposure domain before every comparison. + */ +export const SHADING_CHANGE_SPD_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +@group(0) @binding(1) var inputSignals : texture_2d; +@group(0) @binding(2) var lumaHistory : texture_2d; +@group(0) @binding(3) var dilatedMotion : texture_2d; +@group(0) @binding(4) var frameInfoCur : texture_2d; +@group(0) @binding(5) var frameInfoPrev : texture_2d; +@group(0) @binding(6) var shadingMip0 : texture_storage_2d; +@group(0) @binding(7) var shadingMip1 : texture_storage_2d; +@group(0) @binding(8) var shadingMip2 : texture_storage_2d; + +fn signedDifference(coord : vec2i) -> vec2f { + if (hasFlag(FLAG_RESET)) { return vec2f(0.0); } + let clamped = clamp(coord, vec2i(0), vec2i(C.renderSize) - 1); + let uv = (vec2f(clamped) + 0.5) * C.renderSizeInv; + let motion = textureLoad(dilatedMotion, clamped, 0).xy; + let previousUv = uv - motion; + if (any(previousUv < vec2f(0.0)) || any(previousUv > vec2f(1.0))) { + return vec2f(0.0); + } + + let previousCoord = clamp(vec2i(previousUv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + let currentFrameInfo = textureLoad(frameInfoCur, vec2i(0), 0); + let previousFrameInfo = textureLoad(frameInfoPrev, vec2i(0), 0); + let hostRatio = currentFrameInfo.b / max(previousFrameInfo.b, 1.0e-4); + let conditioning = currentFrameInfo.r; + let currentLuma = textureLoad(inputSignals, clamped, 0).g * conditioning; + let previousLuma = textureLoad(lumaHistory, previousCoord, 0).r * hostRatio * conditioning; + let maximum = max(currentLuma, previousLuma); + if (maximum <= 1.0e-5) { return vec2f(0.0); } + let difference = sign(currentLuma - previousLuma) * + (1.0 - min(currentLuma, previousLuma) / maximum); + return vec2f(difference, select(0.0, sign(difference), difference != 0.0)); +} + +fn reduceBlock(origin : vec2i, extent : i32) -> vec2f { + var sum = vec2f(0.0); + var count = 0.0; + for (var y = 0; y < extent; y++) { + for (var x = 0; x < extent; x++) { + sum += signedDifference(origin + vec2i(x, y)); + count += 1.0; + } + } + return sum / count; +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let mip0Size = max(vec2u(1u), vec2u(C.renderSize + 1.0) / 2u); + if (any(gid.xy >= mip0Size)) { return; } + let coord = vec2i(gid.xy); + textureStore(shadingMip0, coord, vec4f(reduceBlock(coord * 2, 2), 0.0, 0.0)); + if (all((gid.xy % vec2u(2u)) == vec2u(0u))) { + textureStore(shadingMip1, coord / 2, vec4f(reduceBlock(coord * 2, 4), 0.0, 0.0)); + } + if (all((gid.xy % vec2u(4u)) == vec2u(0u))) { + textureStore(shadingMip2, coord / 4, vec4f(reduceBlock(coord * 2, 8), 0.0, 0.0)); + } +} +`, +); + +/** Resolves the three signed-difference mips into one half-resolution mask. */ +export const SHADING_CHANGE_RESOLVE_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +@group(0) @binding(1) var shadingPyramid : texture_2d; +@group(0) @binding(2) var shadingChangeOut : texture_storage_2d; + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let outputSize = vec2u(textureDimensions(shadingChangeOut)); + if (any(gid.xy >= outputSize)) { return; } + let baseCoord = vec2i(gid.xy); + let mip0 = textureLoad(shadingPyramid, baseCoord, 0).xy; + let mip1 = textureLoad(shadingPyramid, baseCoord / 2, 1).xy; + let mip2 = textureLoad(shadingPyramid, baseCoord / 4, 2).xy; + let response = clamp( + max(abs(mip0.x * mip0.y), max(abs(mip1.x * mip1.y), abs(mip2.x * mip2.y))) * + (1.0 + 2.0 / 3.0), + 0.0, + 1.0, + ); + textureStore(shadingChangeOut, baseCoord, vec4f(response, 0.0, 0.0, 1.0)); +} +`, +); + +/** + * Four-frame render-resolution luma instability candidate. History channels + * remain in caller/host pre-exposure space; conditioning is applied only for + * comparisons and removed again before storage. + */ +export const LUMA_INSTABILITY_SOURCE_SHADER = assembleShader( + WGSL_CONSTANTS, + /* wgsl */ ` +@group(0) @binding(1) var inputSignals : texture_2d; +@group(0) @binding(2) var dilatedMotion : texture_2d; +@group(0) @binding(3) var preparedMasks : texture_2d; +@group(0) @binding(4) var lumaHistoryIn : texture_2d; +@group(0) @binding(5) var frameInfoCur : texture_2d; +@group(0) @binding(6) var frameInfoPrev : texture_2d; +@group(0) @binding(7) var lumaHistoryOut : texture_storage_2d; +@group(0) @binding(8) var instabilityOut : texture_storage_2d; + +fn similarity(a : f32, b : f32) -> f32 { + return min(a, b) / max(max(a, b), 1.0e-5); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let coord = vec2i(gid.xy); + let uv = (vec2f(gid.xy) + 0.5) * C.renderSizeInv; + let motion = textureLoad(dilatedMotion, coord, 0).xy; + let previousUv = uv - motion + C.jitterPrev * C.renderSizeInv; + let currentFrameInfo = textureLoad(frameInfoCur, vec2i(0), 0); + let previousFrameInfo = textureLoad(frameInfoPrev, vec2i(0), 0); + let conditioning = max(currentFrameInfo.r, 1.0e-4); + let currentHostLuma = max(textureLoad(inputSignals, coord, 0).g, 0.0); + // Seed every slot after reset/offscreen reprojection. Leaving zeros here + // makes the next shading pass compare valid current luma with empty state. + var history = vec4f(currentHostLuma); + var instability = 0.0; + + if ( + !hasFlag(FLAG_RESET) && + all(previousUv >= vec2f(0.0)) && + all(previousUv <= vec2f(1.0)) + ) { + let previousCoord = clamp(vec2i(previousUv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + let hostRatio = currentFrameInfo.b / max(previousFrameInfo.b, 1.0e-4); + history = textureLoad(lumaHistoryIn, previousCoord, 0) * hostRatio * conditioning; + let current = currentHostLuma * conditioning; + let firstDifference = current - history.x; + let firstSimilarity = similarity(current, history.x); + var maximumSimilarity = firstSimilarity; + if (firstSimilarity < 1.0) { + for (var index = 1; index < 4; index++) { + let difference = current - history[index]; + if (sign(firstDifference) == sign(difference)) { + maximumSimilarity = max(maximumSimilarity, similarity(current, history[index])); + } + } + instability = select(0.0, 1.0, maximumSimilarity > firstSimilarity); + } + + let masks = textureLoad(preparedMasks, coord, 0); + let velocityWeight = 1.0 - clamp(length(motion * vec2f(3840.0, 2160.0)) / 20.0, 0.0, 1.0); + let mature = select(0.0, 1.0, masks.a > 0.9); + instability *= mature * velocityWeight * (1.0 - masks.r) * (1.0 - masks.g) * (1.0 - masks.b); + history = vec4f(current, history.xyz); + history /= conditioning; + if (history.w == 0.0) { instability = 0.0; } + } + + textureStore(lumaHistoryOut, coord, history); + textureStore(instabilityOut, coord, vec4f(instability, 0.0, 0.0, 1.0)); +} +`, +); + +/** + * Coordinated source-style resolver. History alpha stores lock lifetime, not + * local sample age; accumulation and instability are render-resolution state. + */ +export const ACCUMULATE_SOURCE_RESOLVER_SHADER = assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + WGSL_TONEMAP, + /* wgsl */ ` +struct AtomicLockBuffer { + values : array>, +} + +@group(0) @binding(1) var inputColor : texture_2d; +@group(0) @binding(2) var dilatedMotion : texture_2d; +@group(0) @binding(3) var preparedMasks : texture_2d; +@group(0) @binding(4) var historyIn : texture_2d; +// No sampler: history is reconstructed with explicit textureLoad taps, and a +// statically-unused binding would be dropped from the 'auto' layout. +@group(0) @binding(5) var historyOut : texture_storage_2d; +@group(0) @binding(6) var inputSignals : texture_2d; +@group(0) @binding(7) var lumaInstability : texture_2d; +@group(0) @binding(8) var newLocks : AtomicLockBuffer; +@group(0) @binding(9) var frameInfoCur : texture_2d; +@group(0) @binding(10) var frameInfoPrev : texture_2d; + +const PI : f32 = 3.14159265358979; +const LOCK_THRESHOLD : f32 = 1.0; +const LOCK_MAX : f32 = 2.0; +const AVERAGE_LANCZOS_WEIGHT : f32 = 0.74 / 16.0; + +fn lanczos2(value : f32) -> f32 { + let x = abs(value); + if (x < 1.0e-4) { return 1.0; } + if (x >= 2.0) { return 0.0; } + let px = PI * x; + return 2.0 * sin(px) * sin(px * 0.5) / (px * px); +} + +fn lanczos2ApproxSq(value : f32) -> f32 { + let x2 = min(value, 4.0); + let a = (2.0 / 5.0) * x2 - 1.0; + let b = 0.25 * x2 - 1.0; + return ((25.0 / 16.0) * a * a - (9.0 / 16.0)) * b * b; +} + +fn historyLoad(coord : vec2i) -> vec4f { + return textureLoad(historyIn, clamp(coord, vec2i(0), vec2i(C.displaySize) - 1), 0); +} + +fn sampleHistoryLanczos(uv : vec2f) -> vec4f { + let position = uv * C.displaySize - 0.5; + let base = vec2i(floor(position)); + let fraction = fract(position); + var rows = array(); + var centerMin = vec4f(1.0e6); + var centerMax = vec4f(-1.0e6); + for (var y = 0; y < 4; y++) { + var row = vec4f(0.0); + var rowWeight = 0.0; + for (var x = 0; x < 4; x++) { + let sample = historyLoad(base + vec2i(x - 1, y - 1)); + let weight = lanczos2(f32(x - 1) - fraction.x); + row += sample * weight; + rowWeight += weight; + if (x >= 1 && x <= 2 && y >= 1 && y <= 2) { + centerMin = min(centerMin, sample); + centerMax = max(centerMax, sample); + } + } + rows[y] = row / max(abs(rowWeight), 1.0e-5); + } + var result = vec4f(0.0); + var weightSum = 0.0; + for (var y = 0; y < 4; y++) { + let weight = lanczos2(f32(y - 1) - fraction.y); + result += rows[y] * weight; + weightSum += weight; + } + return clamp(result / max(abs(weightSum), 1.0e-5), centerMin, centerMax); +} + +fn clipToEllipsoid(center : vec3f, extents : vec3f, color : vec3f) -> vec3f { + let safeExtents = max(extents, vec3f(1.193e-7)); + let transformed = (color - center) / safeExtents; + let distance = length(transformed); + return select(color, center + normalize(transformed) * safeExtents, distance > 1.0); +} + +@compute @workgroup_size(8, 8) +fn main(@builtin(global_invocation_id) gid : vec3u) { + if (any(vec2f(gid.xy) >= C.displaySize)) { return; } + let uv = (vec2f(gid.xy) + 0.5) * C.displaySizeInv; + let renderCoord = clamp(vec2i(uv * C.renderSize), vec2i(0), vec2i(C.renderSize) - 1); + let motion = textureLoad(dilatedMotion, renderCoord, 0).xy; + let masks = textureLoad(preparedMasks, renderCoord, 0); + let instability = textureLoad(lumaInstability, renderCoord, 0).r; + let currentFrameInfo = textureLoad(frameInfoCur, vec2i(0), 0); + let previousFrameInfo = textureLoad(frameInfoPrev, vec2i(0), 0); + let conditioning = max(currentFrameInfo.r, 1.0e-4); + + //* Source Radial Reconstruction === + let sourcePosition = uv * C.renderSize - 0.5 - C.jitter; + let sourceBase = vec2i(floor(sourcePosition)); + let kernelBiasMax = min(1.99, max(C.displaySize.x / C.renderSize.x, 1.0)); + let kernelBiasMin = max(1.0, (1.0 + kernelBiasMax) * 0.3); + let kernelBiasWeight = min(1.0 - masks.g * 0.5, min(1.0 - masks.b, clamp(masks.a * 5.0, 0.0, 1.0))); + let kernelBias = mix(kernelBiasMin, kernelBiasMax, kernelBiasWeight); + var upsampled = vec3f(0.0); + var upsampledWeight = 0.0; + var boxCenter = vec3f(0.0); + var boxSecondMoment = vec3f(0.0); + var boxWeight = 0.0; + var aabbMinimum = vec3f(1.0e6); + var aabbMaximum = vec3f(-1.0e6); + + for (var y = -1; y <= 1; y++) { + for (var x = -1; x <= 1; x++) { + let tapCoord = sourceBase + vec2i(x, y); + let coord = clamp(tapCoord, vec2i(0), vec2i(C.renderSize) - 1); + let offset = vec2f(tapCoord) - sourcePosition; + let distanceSquared = dot(offset, offset); + let prepared = rgbToYCoCg(max(textureLoad(inputColor, coord, 0).rgb, vec3f(0.0)) * conditioning); + let reconstructionWeight = lanczos2ApproxSq(distanceSquared * kernelBias * kernelBias); + let rectificationWeight = exp(-2.3 * distanceSquared); + upsampled += prepared * reconstructionWeight; + upsampledWeight += reconstructionWeight; + boxCenter += prepared * rectificationWeight; + boxSecondMoment += prepared * prepared * rectificationWeight; + boxWeight += rectificationWeight; + aabbMinimum = min(aabbMinimum, prepared); + aabbMaximum = max(aabbMaximum, prepared); + } + } + upsampled /= max(abs(upsampledWeight), 1.0e-5); + upsampled = clamp(upsampled, aabbMinimum, aabbMaximum); + upsampledWeight = max(upsampledWeight, 0.0) * AVERAGE_LANCZOS_WEIGHT; + boxCenter /= max(boxWeight, 1.0e-5); + let boxVector = sqrt(abs(boxSecondMoment / max(boxWeight, 1.0e-5) - boxCenter * boxCenter)); + + //* Reprojected Linear/HDR History === + let previousUv = uv - motion; + let existing = !hasFlag(FLAG_RESET) && + all(previousUv >= vec2f(0.0)) && + all(previousUv <= vec2f(1.0)); + var historyYcc = upsampled; + var lock = 0.0; + if (existing) { + let history = sampleHistoryLanczos(previousUv); + let hostRatio = currentFrameInfo.b / max(previousFrameInfo.b, 1.0e-4); + historyYcc = rgbToYCoCg(max(history.rgb * hostRatio, vec3f(0.0)) * conditioning); + lock = history.a; + } + + //* Coordinated Lock Lifetime === + let decrease = max(masks.b, max(masks.r, masks.g)); + lock = max(0.0, lock - decrease * LOCK_MAX); + let lockContribution = clamp(clamp(lock - LOCK_THRESHOLD, 0.0, 1.0) * (LOCK_MAX - LOCK_THRESHOLD), 0.0, 1.0); + let lockIndex = gid.y * u32(C.displaySize.x) + gid.x; + let newLock = f32(atomicLoad(&newLocks.values[lockIndex])) / 65535.0; + lock = min(lock + newLock * (1.0 - masks.r), LOCK_MAX); + lock = max(0.0, lock - (0.1 / max(C.maxAccumulation, 1.0)) * (1.0 - decrease)); + lock *= select(0.0, 1.0, all((uv - motion) >= vec2f(0.0)) && all((uv - motion) <= vec2f(1.0))); + + //* Source-Driven Dynamic Rectification === + let velocity4K = length(motion * vec2f(3840.0, 2160.0)); + let farthestDepth = textureLoad(inputSignals, renderCoord, 0).r; + let boxScaleSignal = max( + clamp(velocity4K / 20.0, 0.0, 1.0), + max( + clamp(0.75 - farthestDepth / 20.0, 0.0, 1.0), + max(1.0 - masks.a, max(sqrt(masks.r), masks.b)), + ), + ); + let boxScale = mix(3.0, 1.0, boxScaleSignal); + let scaledBox = boxVector * vec3f(1.7, 1.0, 1.0) * boxScale; + let rectifiedHistory = clipToEllipsoid(boxCenter, scaledBox, historyYcc); + let preserveHistory = max(instability, lockContribution) * masks.a * (1.0 - masks.g); + historyYcc = mix(rectifiedHistory, historyYcc, clamp(preserveHistory, 0.0, 1.0)); + + //* Source Weight Model === + var historyWeight = masks.a; + historyWeight = min( + historyWeight, + // 20-pixel 4K-motion normalization, matching every other velocity + // falloff in the candidate graph (0.5 saturated at half a pixel and + // collapsed history under any camera motion). + mix(historyWeight, 0.15, clamp(max(0.0, velocity4K / 20.0), 0.0, 1.0)), + ); + if (!existing) { historyWeight = 0.0; } + let totalWeight = max(6.1e-5, historyWeight + upsampledWeight); + let alpha = clamp(upsampledWeight / totalWeight, 0.0, 1.0); + + // Tonemap only for the blend, then restore linear/HDR and remove internal + // conditioning. Host pre-exposure remains part of the caller's domain. + let historyTone = rgbToYCoCg(tonemapInvertible(yCoCgToRgb(historyYcc))); + let currentTone = rgbToYCoCg(tonemapInvertible(yCoCgToRgb(upsampled))); + let resultTone = mix(historyTone, currentTone, alpha); + let result = max(tonemapInvert(yCoCgToRgb(resultTone)) / conditioning, vec3f(0.0)); + textureStore(historyOut, gid.xy, vec4f(result, lock)); +} +`, +); diff --git a/src/shaders/common.ts b/src/shaders/common.ts index 4356e36..d6a2cb7 100644 --- a/src/shaders/common.ts +++ b/src/shaders/common.ts @@ -97,28 +97,6 @@ fn tonemapInvert(c : vec3f) -> vec3f { } `; -/** - * Display transform: ACES filmic approximation (Narkowicz) + sRGB OETF. - * Every output path (blit, EASU, RCAS) funnels through this so all bench - * modes are visually comparable. - */ -export const WGSL_DISPLAY_TRANSFORM = /* wgsl */ ` -fn acesFilm(x : vec3f) -> vec3f { - let a = 2.51; let b = 0.03; let c = 2.43; let d = 0.59; let e = 0.14; - return clamp((x * (a * x + b)) / (x * (c * x + d) + e), vec3f(0.0), vec3f(1.0)); -} - -fn srgbEncode(c : vec3f) -> vec3f { - let lo = c * 12.92; - let hi = 1.055 * pow(max(c, vec3f(0.0)), vec3f(1.0 / 2.4)) - 0.055; - return select(hi, lo, c <= vec3f(0.0031308)); -} - -fn displayTransform(linearHdr : vec3f) -> vec3f { - return srgbEncode(acesFilm(linearHdr)); -} -`; - /** * Depth linearization for three's WebGPU projection conventions * (see Matrix4.makePerspective — both standard and reversed [0,1] depth). diff --git a/src/shaders/debug.ts b/src/shaders/debug.ts index b38f773..2cd4fda 100644 --- a/src/shaders/debug.ts +++ b/src/shaders/debug.ts @@ -14,7 +14,7 @@ import { assembleShader } from './wgsl'; * - 6: exposure (1×1; r = pre-exposure, g = avg luma) * - 7: scene color (render size) * - 8: reactive mask (render size; r = reactivity) - * - 9: output storage (rgba8unorm, display size) + * - 9: output storage (rgba16float, display size) */ export const DEBUG_SHADER = assembleShader( WGSL_CONSTANTS, @@ -28,7 +28,7 @@ export const DEBUG_SHADER = assembleShader( @group(0) @binding(6) var exposureTex : texture_2d; @group(0) @binding(7) var inputColor : texture_2d; @group(0) @binding(8) var reactiveMask : texture_2d; -@group(0) @binding(9) var outputColor : texture_storage_2d; +@group(0) @binding(9) var outputColor : texture_storage_2d; // Simple HSV-ish direction coloring for motion vectors. fn motionToColor(m : vec2f) -> vec3f { diff --git a/src/shaders/easu.ts b/src/shaders/easu.ts index 7ddb1d7..a6a86b5 100644 --- a/src/shaders/easu.ts +++ b/src/shaders/easu.ts @@ -1,4 +1,4 @@ -import { WGSL_CONSTANTS, WGSL_DISPLAY_TRANSFORM, WGSL_TONEMAP } from './common'; +import { WGSL_CONSTANTS } from './common'; import { assembleShader } from './wgsl'; /** @@ -19,9 +19,9 @@ import { assembleShader } from './wgsl'; * `textureLoad`s instead of packed `textureGather`s — an acceptable trade * for a test bench (noted in the package README as a Phase 5 optimization). * - * Per the FSR1 spec, EASU runs on display-referred (tonemapped, perceptual) - * data, so the display transform is applied per tap here and RCAS runs on - * this pass's output space directly. + * Per the FSR1 spec, EASU expects perceptual input. The upscaler does not + * choose or bake a presentation transform; callers that use the spatial path + * are responsible for supplying the intended color domain. * * Bindings: * - 1: input color, render resolution (linear HDR) @@ -29,17 +29,14 @@ import { assembleShader } from './wgsl'; */ export const EASU_SHADER = assembleShader( WGSL_CONSTANTS, - WGSL_TONEMAP, - WGSL_DISPLAY_TRANSFORM, /* wgsl */ ` @group(0) @binding(1) var inputColor : texture_2d; @group(0) @binding(2) var outputColor : texture_storage_2d; -// Loads a render-resolution texel in display space (clamped at the borders). +// Loads a render-resolution texel in the caller's color domain. fn easuLoad(p : vec2i) -> vec3f { let clamped = clamp(p, vec2i(0), vec2i(C.renderSize) - 1); - let c = textureLoad(inputColor, clamped, 0).rgb; - return displayTransform(c * C.exposure); + return textureLoad(inputColor, clamped, 0).rgb; } // EASU operates on a green-weighted luma: L = 0.5*R + G + 0.5*B. diff --git a/src/shaders/rcas.ts b/src/shaders/rcas.ts index 62cf279..da48d34 100644 --- a/src/shaders/rcas.ts +++ b/src/shaders/rcas.ts @@ -1,46 +1,63 @@ -import { WGSL_CONSTANTS, WGSL_DISPLAY_TRANSFORM, WGSL_TONEMAP } from './common'; +import { WGSL_CONSTANTS, WGSL_TONEMAP } from './common'; import { assembleShader } from './wgsl'; -/** - * RCAS — Robust Contrast Adaptive Sharpening, the sharpening half of FSR1 - * and the final pass of FSR2/3. - * - * A faithful WGSL port of the f32 reference (`FsrRcasF`) from AMD's - * `ffx_fsr1.h` (MIT licensed). Unlike plain CAS, RCAS derives its maximum - * sharpening lobe analytically from the local min/max ring so it cannot - * over-shoot (ring) regardless of the sharpness setting. - * - * Runs at display resolution over the upscaled image: - * - spatial path — input is EASU output, already display-referred - * - temporal path — input is accumulation history in invertible-tonemap - * space (`FLAG_INPUT_REINHARD`), expanded + display-transformed per tap - * - * Bindings: - * - 1: input color (display size) - * - 2: exposure, 1×1 (rgba16float; r = pre-exposure to undo on the temporal path) - * - 3: output storage (rgba8unorm, display size) - */ -export const RCAS_SHADER = assembleShader( - WGSL_CONSTANTS, - WGSL_TONEMAP, - WGSL_DISPLAY_TRANSFORM, - /* wgsl */ ` +function createRcasShader(fsr315NumericParity: boolean): string { + const luma = fsr315NumericParity + ? /* wgsl */ ` + // FSR's inexpensive luma is scaled by two; the scale cancels in ratios. + let bL = 0.5 * b.r + b.g + 0.5 * b.b; + let dL = 0.5 * d.r + d.g + 0.5 * d.b; + let eL = 0.5 * e.r + e.g + 0.5 * e.b; + let fL = 0.5 * f.r + f.g + 0.5 * f.b; + let hL = 0.5 * h.r + h.g + 0.5 * h.b; +` + : ''; + const lowerLimiter = fsr315NumericParity + ? /* wgsl */ ` + let lowerLimiterMultiplier = clamp( + eL / min(min(bL, dL), min(fL, hL)), + 0.0, + 1.0 + ); +` + : ''; + const hitMinMultiplier = fsr315NumericParity ? ' * lowerLimiterMultiplier' : ''; + const denoise = fsr315NumericParity + ? /* wgsl */ ` + let mn = min(min(min(bL, dL), eL), min(fL, hL)); + let mx = max(max(max(bL, dL), eL), max(fL, hL)); + var nz = 0.25 * (bL + dL + fL + hL) - eL; + nz = clamp(abs(nz) / max(mx - mn, 1.0e-4), 0.0, 1.0); + lobe *= 1.0 - 0.5 * nz; +` + : /* wgsl */ ` + let mn = min(min(b.g, d.g), min(f.g, h.g)); + let mx = max(max(b.g, d.g), max(f.g, h.g)); + var nz = 0.25 * (b.g + d.g + f.g + h.g) - e.g; + nz = clamp(abs(nz) / max(mx - mn, 1.0e-4), 0.0, 1.0); + lobe *= 1.0 - 0.5 * nz; +`; + + return assembleShader( + WGSL_CONSTANTS, + WGSL_TONEMAP, + /* wgsl */ ` @group(0) @binding(1) var inputColor : texture_2d; @group(0) @binding(2) var exposureTex : texture_2d; -@group(0) @binding(3) var outputColor : texture_storage_2d; +@group(0) @binding(3) var outputColor : texture_storage_2d; // Maximum sharpening lobe magnitude — set so a single tap cannot exceed the // local contrast ring (0.25 - 1/16 in the reference). const RCAS_LIMIT : f32 = 0.25 - (1.0 / 16.0); -// Loads a display-resolution texel in final display space. +// Loads a display-resolution texel in the caller's linear/HDR domain. fn rcasLoad(p : vec2i) -> vec3f { let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); let c = textureLoad(inputColor, clamped, 0).rgb; if (hasFlag(FLAG_INPUT_REINHARD)) { // Undo the pre-exposure the accumulate pass baked in before tonemapping. let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); - return displayTransform(tonemapInvert(c) / exposure); + return tonemapInvert(c) / exposure; } return c; } @@ -59,13 +76,15 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { let e = rcasLoad(sp); let f = rcasLoad(sp + vec2i(1, 0)); let h = rcasLoad(sp + vec2i(0, 1)); +${luma} //* Sharpening Lobe // Min/max ring per channel bounds how strong the negative lobe may be // before the output would exceed local contrast. let mn4 = min(min(b, d), min(f, h)); let mx4 = max(max(b, d), max(f, h)); - let hitMin = mn4 / (4.0 * mx4); +${lowerLimiter} + let hitMin = mn4 / (4.0 * mx4)${hitMinMultiplier}; let hitMax = (vec3f(1.0) - mx4) / (4.0 * mn4 - 4.0); let lobeRGB = max(-hitMin, hitMax); // C.sharpness 1 -> 0 attenuation stops (sharpest), 0 -> 2 stops. @@ -77,11 +96,7 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { // range, reads as noise; attenuate the lobe there (up to 50%) so RCAS // doesn't amplify grain from noisy inputs (e.g. reduced-res SSR/GI). if (hasFlag(FLAG_RCAS_DENOISE)) { - let mn = min(min(b.g, d.g), min(f.g, h.g)); - let mx = max(max(b.g, d.g), max(f.g, h.g)); - var nz = 0.25 * (b.g + d.g + f.g + h.g) - e.g; - nz = clamp(abs(nz) / max(mx - mn, 1.0e-4), 0.0, 1.0); - lobe *= 1.0 - 0.5 * nz; +${denoise} } //* Resolve @@ -91,4 +106,15 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { textureStore(outputColor, gid.xy, vec4f(pix, 1.0)); } `, -); + ); +} + +/** + * Legacy RCAS shader retained only for benchmark comparisons. + */ +export const RCAS_LEGACY_SHADER = createRcasShader(false); + +/** + * Production RCAS shader with FSR 3.1.5 lower-limiter and denoise parity. + */ +export const RCAS_SHADER = createRcasShader(true); diff --git a/src/shaders/shaders.test.ts b/src/shaders/shaders.test.ts index 28d6c6f..23de986 100644 --- a/src/shaders/shaders.test.ts +++ b/src/shaders/shaders.test.ts @@ -1,12 +1,47 @@ import { describe, expect, it } from 'vitest'; +import { BenchmarkClock } from '../../bench/src/benchmark/clock'; +import { BenchmarkCollector } from '../../bench/src/benchmark/collector'; +import { + getBenchmarkScenario, + resolveCaptureFrames, +} from '../../bench/src/benchmark/scenarios'; +import { + SingleVariantRegistry, + getActiveResolverCount, +} from '../../bench/src/benchmark/variants'; +import { ComputePass } from '../internal/ComputePass'; import { ACCUMULATE_SHADER } from './accumulate'; import { BLIT_SHADER } from './blit'; +import { + DEBUG_SOURCE_FILTER_SHADER, + DEBUG_SOURCE_RESOLVER_SHADER, + DEBUG_SOURCE_STRUCTURAL_SHADER, +} from './candidateDebug'; +import { + ACCUMULATE_SOURCE_FILTER_SHADER, + ACCUMULATE_SOURCE_STRUCTURAL_SHADER, + EASU_SOURCE_APPROX_SHADER, +} from './candidateFilters'; +import { + DEPTH_CLIP_SOURCE_SHADER, + GENERATE_REACTIVE_SOURCE_SHADER, + PREPARE_INPUTS_SOURCE_SHADER, + PREPARE_REACTIVITY_SOURCE_SHADER, +} from './candidateInputs'; +import { + ACCUMULATE_SOURCE_RESOLVER_SHADER, + EXPOSURE_HISTORY_SOURCE_SHADER, + LUMA_INSTABILITY_SOURCE_SHADER, + LUMA_SPD_SOURCE_SHADER, + SHADING_CHANGE_RESOLVE_SOURCE_SHADER, + SHADING_CHANGE_SPD_SOURCE_SHADER, +} from './candidateTemporal'; import { DEBUG_SHADER } from './debug'; import { EASU_SHADER } from './easu'; import { GENERATE_REACTIVE_SHADER } from './generateReactive'; import { LUMINANCE_PYRAMID_SHADER } from './luminancePyramid'; -import { RCAS_SHADER } from './rcas'; +import { RCAS_LEGACY_SHADER, RCAS_SHADER } from './rcas'; import { RECONSTRUCT_SHADER } from './reconstruct'; import { assembleShader } from './wgsl'; @@ -21,6 +56,75 @@ const ALL_SHADERS: Record = { debug: DEBUG_SHADER, }; +const BASELINE_BINDING_COUNTS: Record = { + blit: 5, + easu: 3, + rcas: 4, + reconstruct: 7, + accumulate: 11, + luminancePyramid: 6, + generateReactive: 4, + debug: 10, +}; + +const BASELINE_FINGERPRINTS: Record = { + blit: '673108e1', + easu: '11632358', + rcas: 'c803572b', + reconstruct: '1ced83aa', + accumulate: 'd0973222', + luminancePyramid: '7a806c41', + generateReactive: '6ed4b549', + debug: 'e30ebd6c', +}; + +const CANDIDATE_SHADERS: Record = { + easuSourceApprox: EASU_SOURCE_APPROX_SHADER, + exposureHistory: EXPOSURE_HISTORY_SOURCE_SHADER, + generateReactiveSource: GENERATE_REACTIVE_SOURCE_SHADER, + prepareInputsSource: PREPARE_INPUTS_SOURCE_SHADER, + depthClipSource: DEPTH_CLIP_SOURCE_SHADER, + prepareReactivitySource: PREPARE_REACTIVITY_SOURCE_SHADER, + accumulateSourceFilter: ACCUMULATE_SOURCE_FILTER_SHADER, + accumulateSourceStructural: ACCUMULATE_SOURCE_STRUCTURAL_SHADER, + lumaSpdSource: LUMA_SPD_SOURCE_SHADER, + shadingSpdSource: SHADING_CHANGE_SPD_SOURCE_SHADER, + shadingResolveSource: SHADING_CHANGE_RESOLVE_SOURCE_SHADER, + lumaInstabilitySource: LUMA_INSTABILITY_SOURCE_SHADER, + accumulateSourceResolver: ACCUMULATE_SOURCE_RESOLVER_SHADER, + debugSourceFilter: DEBUG_SOURCE_FILTER_SHADER, + debugSourceStructural: DEBUG_SOURCE_STRUCTURAL_SHADER, + debugSourceResolver: DEBUG_SOURCE_RESOLVER_SHADER, +}; + +const CANDIDATE_BINDING_COUNTS: Record = { + easuSourceApprox: 3, + exposureHistory: 7, + generateReactiveSource: 4, + prepareInputsSource: 8, + depthClipSource: 5, + prepareReactivitySource: 11, + accumulateSourceFilter: 12, + accumulateSourceStructural: 11, + lumaSpdSource: 9, + shadingSpdSource: 9, + shadingResolveSource: 3, + lumaInstabilitySource: 9, + accumulateSourceResolver: 11, + debugSourceFilter: 10, + debugSourceStructural: 10, + debugSourceResolver: 10, +}; + +function fingerprint(source: string): string { + let hash = 0x811c9dc5; + for (let index = 0; index < source.length; index++) { + hash ^= source.charCodeAt(index); + hash = Math.imul(hash, 0x01000193); + } + return (hash >>> 0).toString(16).padStart(8, '0'); +} + describe('assembleShader', () => { it('deduplicates shared chunks', () => { const chunk = 'fn shared() -> f32 { return 1.0; }'; @@ -63,4 +167,347 @@ describe.each(Object.entries(ALL_SHADERS))('%s shader', (_name, source) => { /if \(any\(vec2f\(gid\.xy\) >= C\.(displaySize|renderSize)\)\) \{ return; \}/, ); }); + + it('keeps the reviewed production assembly', () => { + expect(fingerprint(source)).toBe(BASELINE_FINGERPRINTS[_name]); + }); + + it('contains only the active baseline bindings and body', () => { + const bindings = [...source.matchAll(/@group\(0\) @binding\((\d+)\)/g)].map((match) => + Number(match[1]), + ); + expect(bindings).toEqual( + Array.from({ length: BASELINE_BINDING_COUNTS[_name] }, (_, index) => index), + ); + expect(source).not.toMatch(/(?:^|\n)\s*override\s+|E00_CANDIDATE|candidate algorithm/i); + }); +}); + +describe.each(Object.entries(CANDIDATE_SHADERS))( + '%s candidate shader', + (name, source) => { + it('assembles as one independent 8x8 pipeline', () => { + expect(source.match(/@compute/g)).toHaveLength(1); + expect(source).toMatch(/@compute @workgroup_size\(8, 8\)\s*\nfn main\(/); + expect(source.match(/struct FsrConstants/g)).toHaveLength(1); + expect(source).toContain( + '@group(0) @binding(0) var C : FsrConstants;', + ); + expect(source).toMatch( + /if \(any\(vec2f\(gid\.xy\) >= C\.(displaySize|renderSize)\)\) \{ return; \}/, + ); + }); + + it('has contiguous candidate-only bindings', () => { + const bindings = [ + ...source.matchAll(/@group\(0\) @binding\((\d+)\)/g), + ].map((match) => Number(match[1])); + expect(bindings).toEqual( + Array.from( + { length: CANDIDATE_BINDING_COUNTS[name] }, + (_, index) => index, + ), + ); + }); + + it('has balanced syntax and unique function names', () => { + const count = (pattern: RegExp) => (source.match(pattern) ?? []).length; + expect(count(/\{/g)).toBe(count(/\}/g)); + expect(count(/\(/g)).toBe(count(/\)/g)); + const names = [...source.matchAll(/\bfn\s+(\w+)\s*\(/g)].map( + (match) => match[1], + ); + expect(new Set(names).size).toBe(names.length); + }); + + it('keeps presentation transforms outside the candidate', () => { + expect(source).not.toMatch(/acesFilm|srgbEncode|displayTransform/); + }); + }, +); + +describe('source candidate bundle structure', () => { + it('keeps filter and resolver history-alpha semantics separate', () => { + expect(ACCUMULATE_SOURCE_FILTER_SHADER).toContain( + 'newCount / C.maxAccumulation', + ); + expect(ACCUMULATE_SOURCE_RESOLVER_SHADER).toContain( + 'vec4f(result, lock)', + ); + expect(ACCUMULATE_SOURCE_RESOLVER_SHADER).not.toContain( + 'newCount / C.maxAccumulation', + ); + }); + + it('authors atomic depth scatter and distinct reactivity channels', () => { + expect(PREPARE_INPUTS_SOURCE_SHADER).toContain( + 'atomicMax(&reconstructedDepth.values[index], encoded);', + ); + expect(PREPARE_INPUTS_SOURCE_SHADER).toContain( + 'override PREPARE_STRUCTURAL_SIGNALS : bool = false;', + ); + expect(DEPTH_CLIP_SOURCE_SHADER).toContain( + 'override DEPTH_CLIP_MOTION_DIVERGENCE : bool = false;', + ); + expect(PREPARE_REACTIVITY_SOURCE_SHADER).toContain( + 'vec4f(softReactive, disocclusion, shading', + ); + expect(PREPARE_REACTIVITY_SOURCE_SHADER).toContain( + 'aggressiveReactive = max(', + ); + expect(PREPARE_REACTIVITY_SOURCE_SHADER).toContain( + 'accumulation = min(accumulation + 1.0 / max(C.maxAccumulation, 1.0), 1.0);', + ); + }); + + it('tracks conditioning and host pre-exposure independently', () => { + expect(EXPOSURE_HISTORY_SOURCE_SHADER).toContain( + 'vec4f(conditioning, averageLuma, host', + ); + expect(ACCUMULATE_SOURCE_FILTER_SHADER).toContain( + 'currentHost / previousHost', + ); + expect(ACCUMULATE_SOURCE_RESOLVER_SHADER).toContain( + 'currentFrameInfo.b / max(previousFrameInfo.b', + ); + }); + + it('provides both SPD chains and persistent four-frame luma state', () => { + expect(LUMA_SPD_SOURCE_SHADER).toContain( + 'var lumaMip2 : texture_storage_2d', + ); + expect(SHADING_CHANGE_SPD_SOURCE_SHADER).toContain( + 'fn signedDifference(', + ); + expect(LUMA_INSTABILITY_SOURCE_SHADER).toContain( + 'history = vec4f(current, history.xyz);', + ); + expect(SHADING_CHANGE_SPD_SOURCE_SHADER).toContain( + 'if (hasFlag(FLAG_RESET)) { return vec2f(0.0); }', + ); + expect(LUMA_INSTABILITY_SOURCE_SHADER).toContain( + 'var history = vec4f(currentHostLuma);', + ); + }); + + it('keeps reconstruction weights independent from clamped border loads', () => { + for (const source of [ + ACCUMULATE_SOURCE_FILTER_SHADER, + ACCUMULATE_SOURCE_RESOLVER_SHADER, + ]) { + expect(source).toContain('let tapCoord = sourceBase + vec2i(x, y);'); + expect(source).toContain('let offset = vec2f(tapCoord) - sourcePosition;'); + } + }); +}); + +describe('FSR 3.1.5 RCAS numeric parity', () => { + it('keeps a separate pipeline body with the source limiter and denoise luma', () => { + expect(RCAS_SHADER).not.toBe(RCAS_LEGACY_SHADER); + expect(RCAS_SHADER).toContain('let lowerLimiterMultiplier = clamp('); + expect(RCAS_SHADER).toContain('let eL = 0.5 * e.r + e.g + 0.5 * e.b;'); + expect(RCAS_SHADER).toContain('let mn = min(min(min(bL, dL), eL), min(fL, hL));'); + expect(RCAS_SHADER).toContain( + 'let hitMin = mn4 / (4.0 * mx4) * lowerLimiterMultiplier;', + ); + expect(RCAS_LEGACY_SHADER).not.toContain('lowerLimiterMultiplier'); + }); +}); + +describe('linear HDR output domain', () => { + it('keeps presentation transforms out of upscaling shaders', () => { + for (const source of [BLIT_SHADER, EASU_SHADER, RCAS_SHADER]) { + expect(source).not.toMatch(/acesFilm|srgbEncode|displayTransform/); + } + }); + + it('writes final and debug output through rgba16float storage', () => { + for (const source of [BLIT_SHADER, RCAS_SHADER, DEBUG_SHADER]) { + expect(source).toContain('texture_storage_2d'); + expect(source).not.toContain('texture_storage_2d'); + } + }); +}); + +describe('E00 benchmark foundation', () => { + it('enforces a single active resolver without a GPU', () => { + let created = 0; + let disposed = 0; + const metadata: BenchmarkVariantMetadata = { + id: 'baseline', + name: 'fake baseline', + supportedRatios: [2], + settings: {}, + resourceGraph: [], + pipeline: { + shaderKey: 'fake', + pipelineKey: 'fake', + assembledChunks: [], + wgslOverrides: {}, + timingPassLabels: ['fake'], + }, + }; + const fakeResolver = { + dispose: () => disposed++, + } as unknown as BenchmarkResolver; + const registry = new SingleVariantRegistry([ + { + metadata, + create: () => { + created++; + return fakeResolver; + }, + }, + ]); + + expect(registry.resolve('baseline', 2)).toBe(metadata); + expect(() => registry.resolve('unknown', 2)).toThrow(/Unknown benchmark variant/); + expect(() => registry.resolve('baseline', 3)).toThrow(/does not support ratio/); + expect(created).toBe(0); + expect(registry.create('baseline', 2, {})).toBe(fakeResolver); + expect(getActiveResolverCount()).toBe(1); + expect(() => registry.create('baseline', 2, {})).toThrow(/already active/); + expect(created).toBe(1); + registry.disposeActive(); + expect(disposed).toBe(1); + expect(getActiveResolverCount()).toBe(0); + }); + + it('registers three distinct cumulative candidate profiles', () => { + const registry = new SingleVariantRegistry(); + const filter = registry.resolve('source-filter-bundle-v1', 2); + const structural = registry.resolve('source-structural-bundle-v1', 2); + const resolver = registry.resolve('source-spd-resolver-bundle-v1', 2); + + expect( + new Set([ + filter.pipeline.pipelineKey, + structural.pipeline.pipelineKey, + resolver.pipeline.pipelineKey, + ]).size, + ).toBe(3); + expect(filter.pipeline.wgslOverrides).toMatchObject({ + prepareStructuralSignals: false, + depthClipMotionDivergence: false, + }); + expect(structural.pipeline.wgslOverrides).toMatchObject({ + prepareStructuralSignals: true, + depthClipMotionDivergence: true, + }); + expect(filter.resourceGraph).toContain('prepare-inputs-atomic-depth'); + expect(filter.resourceGraph).not.toContain( + 'prepare-inputs-atomic-depth-farthest-luma', + ); + expect(structural.resourceGraph).toContain( + 'prepare-inputs-atomic-depth-farthest-luma', + ); + expect(resolver.resourceGraph).toContain('source-resolver-history-lock-alpha'); + }); + + it('uses an integer 60 Hz clock and exact scenario events', () => { + const clock = new BenchmarkClock(); + expect(clock.step()).toBe(0); + expect(clock.frame).toBe(1); + expect(clock.time).toBe(1 / 60); + clock.seek(120); + expect(clock.time).toBe(2); + clock.reset(); + expect(clock.frame).toBe(0); + + expect(getBenchmarkScenario('Q9').frame(60).directionalIntensity).toBe(8); + expect(getBenchmarkScenario('Q9').frame(179).directionalIntensity).toBe(2); + expect(getBenchmarkScenario('Q10').frame(120).resize).toEqual({ + width: 1280, + height: 720, + devicePixelRatio: 1, + }); + expect(getBenchmarkScenario('Q6', 'gtao').unsupported).toBeNull(); + expect(getBenchmarkScenario('Q7').unsupported).toBeNull(); + expect(getBenchmarkScenario('Q8', 'recurrent').unsupported).toBeNull(); + expect(resolveCaptureFrames(['0', 'P-1', 'P', '2*P-1'], 32)).toEqual([ + 0, 31, 32, 63, + ]); + }); + + it('summarizes fresh samples and reports missing frames', () => { + const collector = new BenchmarkCollector([10, 11, 12]); + collector.add([ + { + frameTag: 10, + sequence: 1, + passes: [ + { label: 'a', milliseconds: 1 }, + { label: 'b', milliseconds: 2 }, + ], + }, + { + frameTag: 11, + sequence: 2, + passes: [ + { label: 'a', milliseconds: 3 }, + { label: 'b', milliseconds: 4 }, + ], + }, + ]); + const summary = collector.summarize(); + expect(summary.missingFrameCount).toBe(1); + expect(summary.computeSum.samples).toEqual([3, 7]); + expect(summary.computeSum.median).toBe(5); + expect(summary.passes.find((pass) => pass.label === 'a')?.p95).toBe(2.9); + expect(summary.invalidityCount).toBe(1); + }); + + it('rejects malformed authoritative timing evidence', () => { + const collector = new BenchmarkCollector([20, 21]); + collector.add([ + { + frameTag: 19, + sequence: 0, + passes: [{ label: 'a', milliseconds: 1 }], + }, + { + frameTag: 20, + sequence: 1, + passes: [ + { label: 'a', milliseconds: 1 }, + { label: 'a', milliseconds: -1 }, + ], + }, + { + frameTag: 21, + sequence: 2, + passes: [{ label: 'b', milliseconds: 2 }], + }, + ]); + const summary = collector.summarize(); + expect(summary.unexpectedFrameCount).toBe(1); + expect(summary.duplicatePassLabelCount).toBe(1); + expect(summary.invalidValueCount).toBe(1); + expect(summary.invalidityCount).toBeGreaterThan(0); + expect(summary.computeSum.samples).toEqual([2]); + }); + + it('threads optional pipeline constants and metadata', () => { + let descriptor: GPUComputePipelineDescriptor | null = null; + const pipeline = { getBindGroupLayout: () => ({}) } as unknown as GPUComputePipeline; + const device = { + createShaderModule: () => ({}), + createComputePipeline: (value: GPUComputePipelineDescriptor) => { + descriptor = value; + return pipeline; + }, + } as unknown as GPUDevice; + const pass = new ComputePass(device, 'test', '@compute fn main() {}', { + constants: { SAMPLE_COUNT: 4 }, + shaderKey: 'test-key', + assembledChunks: ['common', 'body'], + }); + + expect(descriptor!.compute.constants).toEqual({ SAMPLE_COUNT: 4 }); + expect(pass.metadata).toEqual({ + shaderKey: 'test-key', + constants: { SAMPLE_COUNT: 4 }, + assembledChunks: ['common', 'body'], + }); + }); }); diff --git a/src/types.ts b/src/types.ts index 1cc7152..f362fce 100644 --- a/src/types.ts +++ b/src/types.ts @@ -135,6 +135,15 @@ export interface DispatchInputs { * it yourself (render your transparents' coverage) or via a future helper. */ reactive?: Texture; + /** + * Optional Transparency & Composition mask at render resolution. This is + * intentionally softer than {@link reactive}: it tightens history + * rectification and reduces lock/history confidence without forcing the + * aggressive current-frame reset used for particles and untracked transparents. + * Consumed by source-style structural resolver candidates; ignored by the + * production fallback. + */ + transparencyAndComposition?: Texture; /** * Opaque-only scene color at render resolution. When provided (and no * explicit {@link reactive} mask is given), the upscaler auto-generates the @@ -154,6 +163,14 @@ export interface DispatchInputs { * brightness. Mirrors FSR3's `exposure` dispatch resource. */ exposureTexture?: Texture; + /** + * Optional host pre-exposure texture (red texel, typically 1×1). Unlike + * {@link exposureTexture}, this factor is part of the caller's color + * domain and is therefore preserved at output. Source-style candidates + * track its previous/current ratio to correct reprojected history. + * Omission is equivalent to `1`. + */ + preExposureTexture?: Texture; /** Drop all history this frame (camera cut, teleport, resize). */ reset?: boolean; /** Seconds since the previous frame. */ From 9cafeaa9d9de690e121537c276d71d4a4ecac3f6 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 11:40:15 +0900 Subject: [PATCH 03/22] docs: move parity working docs to bench/docs, add post-parity work plan PARITY.md stays at root as the consumer-facing rationale; the internal ledgers (PARITY-PROGRESS/CANDIDATES/DECISIONS, THREE-TEMPORAL-COMPARISON) move to bench/docs/. NEXT-STEPS.md captures the four surviving adoption items with per-item validation gates. CLAUDE.md points at the new layout. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 9 ++ bench/docs/NEXT-STEPS.md | 84 +++++++++++++++++++ bench/{ => docs}/PARITY-CANDIDATES.md | 0 bench/{ => docs}/PARITY-DECISIONS.md | 0 bench/{ => docs}/PARITY-PROGRESS.md | 4 +- bench/{ => docs}/THREE-TEMPORAL-COMPARISON.md | 0 bench/results/README.md | 2 +- bench/results/experiments/README.md | 2 +- 8 files changed, 97 insertions(+), 4 deletions(-) create mode 100644 bench/docs/NEXT-STEPS.md rename bench/{ => docs}/PARITY-CANDIDATES.md (100%) rename bench/{ => docs}/PARITY-DECISIONS.md (100%) rename bench/{ => docs}/PARITY-PROGRESS.md (98%) rename bench/{ => docs}/THREE-TEMPORAL-COMPARISON.md (100%) diff --git a/CLAUDE.md b/CLAUDE.md index 201a742..9765bd2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -27,6 +27,15 @@ fidelity/tuning of the temporal path (accumulation, disocclusion thresholds, mot convention) is correct enough to render cleanly but **not tuned** — the "landmines still live" section remains the guide for visual regressions. +**Parity program concluded (2026-07-21):** the three source-style FSR 3.1.5 candidate +graphs were GPU-verified and A/B-benchmarked against production — **+36% / +6.5% / ++76% GPU compute with no visual win**; none adopted. Consumer-facing rationale in +`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-*.md`; the four +surviving adoption items (RCAS input-range fix, host pre-exposure, AMD disocclusion +constant, Phase-5 SPD detector) are planned in `bench/docs/NEXT-STEPS.md`. Candidate +A/B runs: `node scripts/run-benchmark.mjs --smoke --variant --comparison ` +(see `--help`). + If you touch shaders/passes, re-verify on a real GPU. A dependency-free way to do it headlessly (no Playwright): launch Chrome with `--headless=new --enable-unsafe-webgpu --remote-debugging-port=N`, drive it over the DevTools Protocol (Node 22 has a native diff --git a/bench/docs/NEXT-STEPS.md b/bench/docs/NEXT-STEPS.md new file mode 100644 index 0000000..bf77a59 --- /dev/null +++ b/bench/docs/NEXT-STEPS.md @@ -0,0 +1,84 @@ +# Post-parity work plan (2026-07-21) + +Outcome of the parity program: no candidate bundle adopted (see +[PARITY-DECISIONS.md](PARITY-DECISIONS.md) and the consumer-facing +[/PARITY.md](../../PARITY.md)). Four items survive as adoption-worthy. Ordered by +value-per-risk; each is a self-contained session with its own GPU verification. + +Every item follows the same gate: `npm test && npm run typecheck && npm run lint`, +then an A/B timing + capture run +(`node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant --comparison `, +plus `--mode capture --scenarios Q0,Q1,Q3 --reloads 1 --allow-differences --review-all`). +≥5% repeatable = actionable, <3% = noise; any visual regression rejects. + +## 1. RCAS input-range investigation (potential free ~2× RCAS) + +Measured fact: RCAS ran 47% cheaper consuming the resolver candidate's history than +production's (0.105 → 0.056 ms, repeatable, noise ≤1%). Hypothesis: production +accumulate emits ALU-hostile values (denormals/extremes, plausibly from the exposure +divide-back or the invertible-tonemap inversion). + +- Dump/histogram production vs resolver history texels on an identical frame (small + CDP harness or a debug view) to find the range difference. +- If confirmed, add a cheap clamp/flush at the end of `accumulate.ts` (or fix the + divide-back ordering) and A/B `rcas` per-pass time: production vs patched production. +- Files: `src/shaders/accumulate.ts`, possibly `src/shaders/rcas.ts`. +- Risk: low — a clamp cannot regress correctness if bounds are chosen above the + invertible-tonemap output range. + +## 2. Host pre-exposure correction (correctness for HDR apps) + +`DispatchInputs.preExposureTexture` exists but production ignores it. Port the +candidate's `DeltaPreExposure()` semantics: track previous/current host pre-exposure, +ratio-correct reprojected history into the current domain before blending, keep host +pre-exposure in the output (unlike conditioning exposure, which is divided out). + +- Source: the exposure/host handling in `src/shaders/candidateTemporal.ts` (exposure + pass) and its accumulate-side history correction — measured ~free there. +- Wire into production `luminancePyramid.ts` (1×1 frame-info texel already exists) and + `accumulate.ts`; thread the dispatch input through `Upscaler.dispatch`. +- Validate with the Q9 exposure-transition scenario: step + ramp host pre-exposure and + confirm no pumping/trails, and that output brightness is unchanged for + pre-exposure = 1. +- Risk: medium — touches the accumulate blend; keep behind a flag until Q9 passes. + +## 3. AMD disocclusion constant in the fused reconstruct pass + +Replace the guessed `DEPTH_SEPARATION_SCALE` / `DEPTH_SIMILARITY_FLOOR` in +`src/shaders/reconstruct.ts` with AMD's viewport/depth-scaled formulation +(`1.37e-05 * halfViewportWidth * maxDepth` shape — implemented in +`src/shaders/candidateInputs.ts`, depth-clip section). Keep the fused single-pass +structure (measured faster than the source's scatter + separate pass). + +- Validate with Q3 (object-motion disocclusion) captures + the Disocclusion debug + view: thin stable silhouette outlines, no full-frame flashing, accumulation-age + resets confined to trails. +- Risk: low-medium — threshold semantics change; the debug views make regressions + obvious. + +## 4. Phase-5 SPD session: coarse-mip shading-change detector + +The roadmap item in [/CLAUDE.md](../../CLAUDE.md) ("True SPD luminance mip chain + +shading-change coarse mip"). Start from the GPU-proven candidate implementation — +`SHADING_CHANGE_SPD` + 3-mip resolve in `src/shaders/candidateTemporal.ts` — not from +scratch. Extract the detector alone; do **not** bring the surrounding resolver (+76% +measured). + +- Known perf issues to fix on extraction (from the code audit): hoist the per-tap + 1×1 `frameInfo` reloads out of the reduction loops; drop the write-only luma-pyramid + mips unless a consumer lands. +- Wire its output into the existing `FLAG_SHADING_CHANGE` aging path in + `accumulate.ts`, replacing the 3×3-neighborhood mean; keep the lock-suppression + behavior exactly (locks must NOT break on shading change — regression documented in + CLAUDE.md). +- Validate: `DebugView.ShadingChange` black on a still scene, lights up under an + animated light; high-frequency content under heavy motion should show fewer false + positives than production (this is the whole point — capture both). +- Risk: highest of the four — dedicated session with GPU tuning time, per the roadmap. + +## Explicitly not planned (measured against) + +- Lanczos2/bicubic history filtering (+47% accumulate, no visible win). +- Farthest depth / motion divergence signals (+30% prepareInputs, outputs unconsumed). +- Atomic depth scatter as a wholesale replacement for the fused reconstruct pass. +- T&C as a distinct softer channel — revisit only on user demand with real content. diff --git a/bench/PARITY-CANDIDATES.md b/bench/docs/PARITY-CANDIDATES.md similarity index 100% rename from bench/PARITY-CANDIDATES.md rename to bench/docs/PARITY-CANDIDATES.md diff --git a/bench/PARITY-DECISIONS.md b/bench/docs/PARITY-DECISIONS.md similarity index 100% rename from bench/PARITY-DECISIONS.md rename to bench/docs/PARITY-DECISIONS.md diff --git a/bench/PARITY-PROGRESS.md b/bench/docs/PARITY-PROGRESS.md similarity index 98% rename from bench/PARITY-PROGRESS.md rename to bench/docs/PARITY-PROGRESS.md index 7f86701..55063a2 100644 --- a/bench/PARITY-PROGRESS.md +++ b/bench/docs/PARITY-PROGRESS.md @@ -48,7 +48,7 @@ Each experiment allows at most two consolidated fix rounds after its initial imp ## Documentation Ownership - The controller owns this ledger, immutable manifests, state transitions, decisions, and cross-experiment dependencies. -- Concise measured results, recommendations, user votes, and resulting actions are tracked in `bench/PARITY-DECISIONS.md`. +- Concise measured results, recommendations, user votes, and resulting actions are tracked in `bench/docs/PARITY-DECISIONS.md`. - Implementers own only files explicitly listed in their manifest and may not edit manifests or this ledger. - Reviewers and research agents are read-only unless a separate exact allowlist says otherwise. - Implementation agents whose exact allowlists share any path are serialized. The controller must finish or stop the active writer before starting another overlapping writer. @@ -88,7 +88,7 @@ Three cumulative internal candidates are now authored for later A/B work. This i implementation state only: no benchmark, browser GPU validation, visual review, timing claim, or adoption decision has been made. -See `bench/PARITY-CANDIDATES.md` for the candidate hypotheses, cumulative dependencies, +See `bench/docs/PARITY-CANDIDATES.md` for the candidate hypotheses, cumulative dependencies, fallbacks, and the performance-first test matrix required before any adoption decision. - `source-filter-bundle-v1` replaces current/history reconstruction, EASU implementation diff --git a/bench/THREE-TEMPORAL-COMPARISON.md b/bench/docs/THREE-TEMPORAL-COMPARISON.md similarity index 100% rename from bench/THREE-TEMPORAL-COMPARISON.md rename to bench/docs/THREE-TEMPORAL-COMPARISON.md diff --git a/bench/results/README.md b/bench/results/README.md index 22896a7..b006ab3 100644 --- a/bench/results/README.md +++ b/bench/results/README.md @@ -38,7 +38,7 @@ small diagnostic artifacts under a separately authorized manifest. The authored source-style bundles do not have result manifests yet. Their hypotheses, cumulative ordering, and required later evidence are documented in -`bench/PARITY-CANDIDATES.md`; do not store interactive or smoke output as adoption +`bench/docs/PARITY-CANDIDATES.md`; do not store interactive or smoke output as adoption evidence. Q6-Q8 construct reduced-resolution three.js effect graphs for GTAO, SSR, SSGI, diff --git a/bench/results/experiments/README.md b/bench/results/experiments/README.md index 5f7d59f..b229c71 100644 --- a/bench/results/experiments/README.md +++ b/bench/results/experiments/README.md @@ -87,7 +87,7 @@ The controller: 7. Runs or assigns real-device validation and evidence collection. 8. Assigns a fresh read-only task review. 9. Consolidates critical and important findings into no more than two fix rounds. -10. Makes the result decision and updates `bench/PARITY-PROGRESS.md`. +10. Makes the result decision and updates `bench/docs/PARITY-PROGRESS.md`. 11. Starts dependent work only from an adopted, documented integration state. The controller may narrow a task without changing its manifest. Any expansion requires a new manifest revision before work resumes. From df06d47da920a5c3a00c3fa8d04df6057b326fca Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 14:20:53 +0900 Subject: [PATCH 04/22] =?UTF-8?q?feat:=20land=20post-parity=20items=201-3?= =?UTF-8?q?=20=E2=80=94=20RCAS=20conditioned-space=20sharpening,=20DeltaPr?= =?UTF-8?q?eExposure,=20AMD=20disocclusion?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Item 1 (RCAS cost investigation, resolved): the resolver candidate's "47% cheaper RCAS" was per-tap work, not value ranges — production paid a 1x1 exposure load + tonemapInvert division + exposure division per tap behind FLAG_INPUT_REINHARD. Production RCAS now sharpens the accumulate history's conditioned tonemap-space texels (the [0,1) range the limiter math assumes) and inverts the conditioning once on the result: RCAS 0.103 -> 0.068 ms (-34%), total pipeline -5.7% (warm ABBA blocks, ratio 2). Captures on Q0/Q1/Q3 + Q9 HDR stress: RMSE <= 1.8/255, HDR-bulb ROI <= 9/255 — visually identical, no overshoot. The per-tap form is frozen as RCAS_PER_TAP_SHADER under the rcas-fsr315-limiter / rcas-fsr315-numeric bench identities; both isolating timing variants (rcas-hoisted-exposure-v1, rcas-tonemap-space-v1) stay in the registry. Item 2 (host pre-exposure, DeltaPreExposure semantics): the pyramid publishes preExposureTexture's value in the exposure texel's .b and meters auto-exposure host-invariantly (GPU-found: without this the conditioning chases a host step for ~2s and the drift fires the shading-change detector full-screen on flat regions). Accumulate ratio-corrects reprojected history in linear space across a host change (new binding 11 = previous exposure texel). Validated on the new Q11 host-pre-exposure scenario (bench drives the scene MRT color and preExposureTexture together; E00 manifest extended): a 2.5x step + ramp leaves the detector at baseline, never resets accumulation age, and output brightness tracks the drive. Without the input the pass is byte-identical (PNG-compared pre/post). Item 3 (AMD disocclusion constant): the fused reconstruct pass replaces the DEPTH_SEPARATION_SCALE / DEPTH_SIMILARITY_FLOOR guesses with AMD's per-bilinear-tap confidence voting and viewport/depth-scaled tolerance (1.37e-5 * halfViewportWidth * maxDepth, ffx_fsr2_depth_clip.h via the GPU-verified candidate port). Q3: thin stable silhouette outlines, still scenes near-black, age resets confined to trails; finals RMSE <= 1.1/255; pass time unchanged. Fused single-pass structure kept (source's scatter + separate pass measured +30%/+22% for no visual win). GPU verification: all runs on Apple Metal via CDP; examples 01/05/07 smoke clean (no WGSL errors) on the default-constructed Upscaler. Docs updated (PARITY.md, shaders README fidelity table, NEXT-STEPS, CLAUDE.md). Only NEXT-STEPS item 4 (Phase-5 SPD shading-change detector) remains open. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 15 +- PARITY.md | 32 ++-- bench/docs/NEXT-STEPS.md | 99 +++++++------ bench/results/experiments/e00-harness.json | 116 ++++++++++++++- bench/src/BenchPipeline.ts | 60 +++++++- bench/src/BenchScene.ts | 6 + bench/src/benchmark/BenchmarkResolver.ts | 37 ++++- bench/src/benchmark/api.ts | 2 + bench/src/benchmark/config.ts | 4 +- bench/src/benchmark/scenarios.ts | 30 ++++ bench/src/benchmark/variants.ts | 19 ++- bench/src/main.ts | 2 + bench/src/types/benchmark.d.ts | 7 +- src/Upscaler.ts | 6 + src/shaders/README.md | 112 +++++++------- src/shaders/accumulate.ts | 28 +++- src/shaders/luminancePyramid.ts | 18 ++- src/shaders/rcas.ts | 163 +++++++++++++++++++-- src/shaders/reconstruct.ts | 71 +++++---- src/shaders/shaders.test.ts | 54 ++++++- src/types.ts | 13 +- 21 files changed, 704 insertions(+), 190 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 9765bd2..97ffbc3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,9 +30,16 @@ live" section remains the guide for visual regressions. **Parity program concluded (2026-07-21):** the three source-style FSR 3.1.5 candidate graphs were GPU-verified and A/B-benchmarked against production — **+36% / +6.5% / +76% GPU compute with no visual win**; none adopted. Consumer-facing rationale in -`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-*.md`; the four -surviving adoption items (RCAS input-range fix, host pre-exposure, AMD disocclusion -constant, Phase-5 SPD detector) are planned in `bench/docs/NEXT-STEPS.md`. Candidate +`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-*.md`. **Post-parity +items 1–3 landed the same day** (see `bench/docs/NEXT-STEPS.md` for evidence): +(1) RCAS now sharpens in conditioned tonemap space, inverting once — **−34% RCAS, +−5.7% total** with capture-identical output; the old form is frozen as +`RCAS_PER_TAP_SHADER` under the `rcas-fsr315-limiter` bench identity. (2) Host +pre-exposure (`preExposureTexture`) is honored end-to-end — DeltaPreExposure history +correction + host-invariant auto-exposure metering, validated on the new **Q11** +scenario, byte-identical when absent. (3) The reconstruct pass uses AMD's +viewport/depth-scaled disocclusion (per-tap confidence voting) inside the fused +single pass. Only NEXT-STEPS item 4 (Phase-5 SPD detector) remains. Candidate A/B runs: `node scripts/run-benchmark.mjs --smoke --variant --comparison ` (see `--help`). @@ -145,7 +152,7 @@ These were discovered by reading three's source; they're non-obvious and easy to - **Accumulation tuning.** Catmull-Rom history filter, YCoCg variance-clip gamma (`CLIP_GAMMA = 1.0`), disocclusion/clip weight falloffs, **and the luminance-lock constants (`LOCK_*`)** in `accumulate.ts` are sensible defaults, not tuned. Ghosting → tighten (lower `LOCK_CLAMP_RELAX`/`LOCK_HISTORY_BOOST`, raise the peak/contrast thresholds); instability/shimmer or thin features dimming → loosen. Use `DebugView.Locks` to see where locks form. - **Shading-change constants** (`accumulate.ts`: `SHADING_LO/HI/AGE`) are conservative defaults. If stable, steadily-lit surfaces shimmer or fail to converge, `SHADING_LO` is too low (it's aging history that didn't change) — raise it; if a genuine lighting change ghosts its old shading, lower it. `DebugView.ShadingChange` should be black on a still scene — check it before suspecting the accumulate blend. The detector reuses the 3×3 neighborhood mean, not a coarse pyramid mip, so it can false-positive on very high-frequency content under heavy motion; that's the Phase-5 SPD-mip refinement. - **Auto-exposure constants** (`luminancePyramid.ts`: `EXPOSURE_KEY`, `EXPOSURE_MIN/MAX`, `ADAPT_SPEED`) are defaults. Because exposure is divided back out before display, the *visible* effect is subtle (better accumulation stability, not a brightness change). If a scene pulses in brightness, `ADAPT_SPEED` is the suspect; if the image goes flat/washed on a very bright or dark scene, check the min/max clamp. `DebugView.Exposure` should read near mid-grey — verify there before suspecting the accumulate math. -- **Depth separation threshold** in `reconstruct.ts` (`DEPTH_SEPARATION_SCALE`, `DEPTH_SIMILARITY_FLOOR`) is a guess. Too aggressive = history thrown away everywhere (no convergence); too slack = ghost trails behind moving objects. +- ~~**Depth separation threshold** in `reconstruct.ts` is a guess~~ — **resolved 2026-07-21**: replaced by AMD's viewport/depth-scaled formulation (`1.37e-5 · halfViewportWidth · maxDepth`, per-bilinear-tap confidence voting from `ffx_fsr2_depth_clip.h`), GPU-validated on Q3 (thin stable outlines, still scenes quiet, age resets confined to trails). No scene-tuned constants remain in this pass. - **`timestamp-query`** may be absent; `GpuTimer` no-ops gracefully, but confirm the GPU-ms readout actually appears where supported. --- diff --git a/PARITY.md b/PARITY.md index dc8594e..972f3fe 100644 --- a/PARITY.md +++ b/PARITY.md @@ -40,6 +40,21 @@ Reproduce with: - **RCAS numeric math.** The production sharpener uses FSR 3.1.5's lower limiter and the corrected denoise luma/range math, adopted after A/B measurement showed parity was free (E01). Denoise is opt-in pending evidence on representative noisy content. + (One deliberate divergence in the *load domain*: the temporal path sharpens the + accumulated history's conditioned tonemap-space texels and inverts the conditioning + once on the result, instead of upstream's exposed-linear per-tap domain — measured + 34% cheaper on the RCAS pass with capture-verified visually identical output, + including an HDR stress scenario.) +- **Host pre-exposure (`DeltaPreExposure`).** The `preExposureTexture` dispatch input is + honored end-to-end: reprojected history is ratio-corrected across a host pre-exposure + change, auto-exposure meters host-invariantly, and the host factor is preserved in + the output — upstream's contract. Validated with a dedicated step+ramp scenario + (Q11): no history invalidation, no brightness pumping, and byte-identical output when + the input is absent. +- **Viewport/depth-scaled disocclusion.** The disocclusion threshold uses AMD's + formulation (`ffx_fsr2_depth_clip.h`'s per-tap confidence with the + `1.37e-5 · halfViewportWidth · maxDepth` tolerance) instead of the earlier fixed + relative-threshold guess — kept inside our faster fused reconstruction pass. - **Color and exposure domains.** Like upstream, the upscaler applies no tone mapping or output encoding — input and output are the caller's linear/HDR domain, and internal conditioning exposure is divided back out before output. An earlier internal ACES/sRGB @@ -89,15 +104,12 @@ claims can be re-tested; a repeatable ≥5% result is treated as actionable, <3% ## Open items we do intend to converge -- **Host pre-exposure semantics** (`DispatchInputs.preExposureTexture`): correcting - reprojected history across changing host pre-exposure, as upstream's - `DeltaPreExposure()` does — a correctness fix for HDR apps, measured ~free in the - candidate graph. -- **AMD's viewport/depth-scaled disocclusion constant** in place of the current fixed - threshold guess, kept inside the (faster) fused reconstruction pass. +Three of the four post-parity adoption items landed on 2026-07-21 (host +pre-exposure correction, the AMD disocclusion constant, and the RCAS cost +investigation — resolved as the conditioned-space load domain, −34% on the pass). +One remains: + - **A coarse-mip shading-change detector** (the source design, GPU-proven in the resolver candidate) to replace the 3×3-neighborhood heuristic, which can - false-positive on high-frequency content under heavy motion. -- **RCAS input-range investigation:** the source resolver's history made RCAS 47% - cheaper in measurement; if production accumulate emits ALU-hostile value ranges, - clamping them is a free performance win. + false-positive on high-frequency content under heavy motion. Scheduled as its own + session (`bench/docs/NEXT-STEPS.md`, item 4). diff --git a/bench/docs/NEXT-STEPS.md b/bench/docs/NEXT-STEPS.md index bf77a59..d4bc5c4 100644 --- a/bench/docs/NEXT-STEPS.md +++ b/bench/docs/NEXT-STEPS.md @@ -2,8 +2,9 @@ Outcome of the parity program: no candidate bundle adopted (see [PARITY-DECISIONS.md](PARITY-DECISIONS.md) and the consumer-facing -[/PARITY.md](../../PARITY.md)). Four items survive as adoption-worthy. Ordered by -value-per-risk; each is a self-contained session with its own GPU verification. +[/PARITY.md](../../PARITY.md)). Four items survived as adoption-worthy. +**Items 1–3 landed on 2026-07-21** (same-day session; evidence below). Item 4 +remains open as its own dedicated session. Every item follows the same gate: `npm test && npm run typecheck && npm run lint`, then an A/B timing + capture run @@ -11,52 +12,51 @@ then an A/B timing + capture run plus `--mode capture --scenarios Q0,Q1,Q3 --reloads 1 --allow-differences --review-all`). ≥5% repeatable = actionable, <3% = noise; any visual regression rejects. -## 1. RCAS input-range investigation (potential free ~2× RCAS) - -Measured fact: RCAS ran 47% cheaper consuming the resolver candidate's history than -production's (0.105 → 0.056 ms, repeatable, noise ≤1%). Hypothesis: production -accumulate emits ALU-hostile values (denormals/extremes, plausibly from the exposure -divide-back or the invertible-tonemap inversion). - -- Dump/histogram production vs resolver history texels on an identical frame (small - CDP harness or a debug view) to find the range difference. -- If confirmed, add a cheap clamp/flush at the end of `accumulate.ts` (or fix the - divide-back ordering) and A/B `rcas` per-pass time: production vs patched production. -- Files: `src/shaders/accumulate.ts`, possibly `src/shaders/rcas.ts`. -- Risk: low — a clamp cannot regress correctness if bounds are chosen above the - invertible-tonemap output range. - -## 2. Host pre-exposure correction (correctness for HDR apps) - -`DispatchInputs.preExposureTexture` exists but production ignores it. Port the -candidate's `DeltaPreExposure()` semantics: track previous/current host pre-exposure, -ratio-correct reprojected history into the current domain before blending, keep host -pre-exposure in the output (unlike conditioning exposure, which is divided out). - -- Source: the exposure/host handling in `src/shaders/candidateTemporal.ts` (exposure - pass) and its accumulate-side history correction — measured ~free there. -- Wire into production `luminancePyramid.ts` (1×1 frame-info texel already exists) and - `accumulate.ts`; thread the dispatch input through `Upscaler.dispatch`. -- Validate with the Q9 exposure-transition scenario: step + ramp host pre-exposure and - confirm no pumping/trails, and that output brightness is unchanged for - pre-exposure = 1. -- Risk: medium — touches the accumulate blend; keep behind a flag until Q9 passes. - -## 3. AMD disocclusion constant in the fused reconstruct pass - -Replace the guessed `DEPTH_SEPARATION_SCALE` / `DEPTH_SIMILARITY_FLOOR` in -`src/shaders/reconstruct.ts` with AMD's viewport/depth-scaled formulation -(`1.37e-05 * halfViewportWidth * maxDepth` shape — implemented in -`src/shaders/candidateInputs.ts`, depth-clip section). Keep the fused single-pass -structure (measured faster than the source's scatter + separate pass). - -- Validate with Q3 (object-motion disocclusion) captures + the Disocclusion debug - view: thin stable silhouette outlines, no full-frame flashing, accumulation-age - resets confined to trails. -- Risk: low-medium — threshold semantics change; the debug views make regressions - obvious. - -## 4. Phase-5 SPD session: coarse-mip shading-change detector +## 1. RCAS input-range investigation — DONE (adopted: conditioned-space sharpening) + +The measured "resolver history made RCAS 47% cheaper" was **not** a value-range +effect — production history texels are bounded [0,1). Reading the wiring showed the +cost: with `FLAG_INPUT_REINHARD`, production RCAS paid a 1×1 exposure load + a +`tonemapInvert` division + an exposure division **per tap** (5 taps/pixel); the +resolver ran with the flag off (plain loads). + +- Two isolating variants were built and ABBA-timed (warm blocks, ratio 2): + `rcas-hoisted-exposure-v1` (identical math, hoisted exposure) → **−20% RCAS**; + `rcas-tonemap-space-v1` (sharpen the bounded tonemapped texels, invert once) + → **−34% RCAS** (0.103 → 0.068 ms), −5.7% total pipeline compute. +- Captures Q0/Q1/Q3 + Q9 HDR stress: full-frame RMSE ≤ 1.8/255, HDR-bulb ROI + ≤ 9/255 max — visually indistinguishable, no overshoot. +- **Adopted** as production `RCAS_SHADER`. The per-tap form is frozen as + `RCAS_PER_TAP_SHADER` behind the `rcas-fsr315-limiter` / `rcas-fsr315-numeric` + bench identities; the timing variants remain in the registry for re-testing. + +## 2. Host pre-exposure correction — DONE (DeltaPreExposure semantics) + +- Pyramid publishes host pre-exposure in the exposure texel's `.b` (1.0 when no + `preExposureTexture` is supplied) and **meters host-invariantly** — auto-exposure + must not chase a step the app already metered (found on GPU: without this, the + conditioning re-adapts for ~2s after a host step and the drift reads as a + full-screen shading change on flat regions). +- Accumulate ratio-corrects reprojected history in linear space when the host value + changed (binding 11 = previous frame's exposure texel). Self-gating: identity + without the input. +- Validated on the new **Q11 host-pre-exposure scenario** (bench drives the scene + MRT color and `preExposureTexture` together; manifest updated): 2.5× step + ramp + leaves the shading detector at baseline, never resets accumulation age, and output + brightness tracks the drive. No-input captures are **byte-identical** pre/post. + +## 3. AMD disocclusion constant — DONE (in the fused reconstruct pass) + +- `DEPTH_SEPARATION_SCALE`/`DEPTH_SIMILARITY_FLOOR` guesses replaced by AMD's + per-bilinear-tap confidence voting with the viewport/depth-scaled tolerance + (`1.37e-5 · halfViewportWidth · max(depth)`), lifted from the GPU-verified + candidate port. Fused single-pass structure kept (the source's atomic scatter + + separate pass measured +30%/+22% with no visual win). +- Q3 validation: thin stable silhouette outlines, still scenes near-black, age + resets confined to trails; finals shift RMSE ≤ 1.1/255; reconstruct pass time + unchanged (0.035 ms at ratio 2). + +## 4. Phase-5 SPD session: coarse-mip shading-change detector — OPEN The roadmap item in [/CLAUDE.md](../../CLAUDE.md) ("True SPD luminance mip chain + shading-change coarse mip"). Start from the GPU-proven candidate implementation — @@ -82,3 +82,6 @@ measured). - Farthest depth / motion divergence signals (+30% prepareInputs, outputs unconsumed). - Atomic depth scatter as a wholesale replacement for the fused reconstruct pass. - T&C as a distinct softer channel — revisit only on user demand with real content. +- Conditioning-exposure history correction (beyond host pre-exposure): eased + adaptation keeps the per-frame mismatch under the shading detector's threshold; + correcting it changes output for every auto-exposure user. Revisit with evidence. diff --git a/bench/results/experiments/e00-harness.json b/bench/results/experiments/e00-harness.json index 0ce143d..a36b94e 100644 --- a/bench/results/experiments/e00-harness.json +++ b/bench/results/experiments/e00-harness.json @@ -1367,6 +1367,120 @@ ] } } + }, + { + "id": "Q11", + "name": "host-pre-exposure", + "purpose": "Replay an app-baked host pre-exposure step at frame 60, a hold, and a linear ramp home over frames 120 through 179, with the same value fed as the preExposureTexture dispatch input; proves DeltaPreExposure history correction (no shading-change firing, no accumulation reset, output brightness tracks the drive).", + "initial_camera": { + "position": [ + 9, + 6, + 12 + ], + "target": [ + 0, + 1.6, + 0 + ], + "up": [ + 0, + 1, + 0 + ], + "projection": { + "type": "perspective", + "vertical_fov_degrees": 50, + "aspect": 1.7777777777777777, + "near": 0.1, + "far": 200 + } + }, + "frame_range": { + "start": 0, + "end_inclusive": 239, + "duration_seconds": 4 + }, + "camera_formula": "position=(9,6,12), target=(0,1.6,0), up=(0,1,0) for every n.", + "object_formula": "All object transforms are frozen to formula_conventions.baseline_scene_formula evaluated at t=0.", + "light_exposure_formula": "Directional intensity fixed at 3.2 and hemisphere at 0.9 for every n. Host pre-exposure H(n)=1.0 for 0<=n<60; H(n)=2.5 for 60<=n<120; H(n)=2.5-1.5*(n-120)/59 for 120<=n<180; H(n)=1.0 for 180<=n<=239. The scene MRT color is multiplied by H(n) and the same value is bound as the preExposureTexture dispatch input. settings.exposure=1.0 and autoExposure=true for every n; no external exposure texture is bound.", + "events": "Host pre-exposure step 1.0 to 2.5 at n=60; inclusive linear ramp 2.5 to 1.0 over n=120..179; steady at 1.0 from n=180. Reset only before n=0.", + "randomness": { + "seed_uint32": 0, + "prng": "none", + "policy": "No PRNG calls are permitted." + }, + "active_inputs_effects": [ + "bench color scaled by host pre-exposure", + "float depth", + "zero scene velocity except jitter compensation", + "auto exposure", + "host preExposureTexture", + "shading-change detector", + "locks" + ], + "captures": { + "frames": [ + "0", + "23", + "P-1", + "P", + "59", + "60", + "61", + "62", + "64", + "68", + "76", + "83", + "119", + "120", + "121", + "135", + "149", + "164", + "179", + "180", + "181", + "184", + "196", + "203", + "239" + ], + "debug_views": [ + "final", + "accumulation-age", + "locks", + "exposure", + "shading-change" + ], + "ratios": [ + 1, + 1.5, + 2, + 3 + ], + "rois": { + "full": [ + 0, + 0, + 1, + 1 + ], + "lit_knots": [ + 0.18, + 0.16, + 0.64, + 0.38 + ], + "hdr_bulb": [ + 0.43, + 0.08, + 0.14, + 0.2 + ] + } + } } ], "required_debug_views": [ @@ -1761,4 +1875,4 @@ "scope_rule": "If any additional write path or forbidden change appears necessary, stop before modifying it and return SCOPE_BLOCKED.", "pass_rule": "PASS means only that the assigned implementation scope and all available assigned gates passed; it does not authorize adoption." } -} +} \ No newline at end of file diff --git a/bench/src/BenchPipeline.ts b/bench/src/BenchPipeline.ts index bd87e26..8a25f0e 100644 --- a/bench/src/BenchPipeline.ts +++ b/bench/src/BenchPipeline.ts @@ -8,6 +8,7 @@ import { pass, roughness, texture, + uniform, vec4, velocity, } from 'three/tsl'; @@ -144,10 +145,15 @@ export class BenchPipeline { //* Scene Target private _renderTarget: THREE.RenderTarget | null = null; private _reactiveTarget: THREE.RenderTarget | null = null; - private readonly _mrtNode = mrt({ output, velocity }); + private _mrtNode = mrt({ output, velocity }); // Single-output MRT for non-temporal modes — its output count matches the // count:1 render target so color attachment 0 is actually written. - private readonly _mrtOutputOnly = mrt({ output }); + private _mrtOutputOnly = mrt({ output }); + //* Host pre-exposure drive (scenario Q11) — off by default so every other + //* scenario keeps its exact historical scene shader and dispatch inputs. + private readonly _hostPreExposure = uniform(1); + private _hostPreExposureValue: number | null = null; + private readonly _hostPreExposureTextures = new Map(); private _effectScenario: EffectScenario | null = null; private _effectPass: ReturnType | null = null; private _effectSsrNodes: SsrPrivateNode[] = []; @@ -734,6 +740,50 @@ export class BenchPipeline { * @param deltaTime - Fixed or interactive timestep * @param frameTag - Deterministic frame identity for fresh timing */ + /** + * Opts this pipeline into host pre-exposure driving (scenario Q11): the + * scene MRT color is scaled by the driven value — emulating an app that + * bakes its own exposure into the render — and the same value is fed to + * the resolver as `preExposureTexture` every dispatch. + */ + enableHostPreExposureDrive(): void { + this._mrtNode = mrt({ output: output.mul(this._hostPreExposure), velocity }); + this._mrtOutputOnly = mrt({ output: output.mul(this._hostPreExposure) }); + this._hostPreExposureValue = 1; + } + + /** + * Sets the host pre-exposure for the next rendered frame. Requires + * {@link enableHostPreExposureDrive}. + * @param value - Positive exposure factor baked into the scene color + */ + setHostPreExposure(value: number): void { + if (this._hostPreExposureValue === null) + throw new Error('Host pre-exposure drive is not enabled for this pipeline.'); + this._hostPreExposure.value = value; + this._hostPreExposureValue = value; + } + + // The 1×1 texture is created per distinct value: three only re-uploads + // texture data for textures its own render graph consumes, so mutating one + // texture's texels would never reach the GPU. initTexture() forces upload. + private _hostPreExposureTexture(value: number): THREE.DataTexture { + let tex = this._hostPreExposureTextures.get(value); + if (!tex) { + tex = new THREE.DataTexture( + new Float32Array([value, 0, 0, 1]), + 1, + 1, + THREE.RGBAFormat, + THREE.FloatType, + ); + tex.needsUpdate = true; + this._renderer.initTexture(tex); + this._hostPreExposureTextures.set(value, tex); + } + return tex; + } + dispatchResolver( camera: THREE.PerspectiveCamera, deltaTime: number, @@ -759,6 +809,10 @@ export class BenchPipeline { depth, velocity: velocityTexture, reactive, + preExposureTexture: + this._hostPreExposureValue !== null + ? this._hostPreExposureTexture(this._hostPreExposureValue) + : undefined, deltaTime, frameTag, }, @@ -1012,6 +1066,8 @@ export class BenchPipeline { this._quadMaterial.dispose(); this._effectMaterial.dispose(); this._depthOnlyMaterial.dispose(); + this._hostPreExposureTextures.forEach((tex) => tex.dispose()); + this._hostPreExposureTextures.clear(); this.resolver.dispose(); } } diff --git a/bench/src/BenchScene.ts b/bench/src/BenchScene.ts index ba5949d..e3f670d 100644 --- a/bench/src/BenchScene.ts +++ b/bench/src/BenchScene.ts @@ -295,6 +295,12 @@ export function createBenchScene(): BenchScene { particles.visible = frame.particlesVisible; reactiveParticles.visible = frame.particlesVisible; sun.intensity = frame.directionalIntensity; + // The Q11 host pre-exposure multiplier lives in the MRT output node, + // which the background never passes through — scale it here so the + // whole frame is uniformly pre-exposed like a real app's render. + (scene.background as THREE.Color) + .setHex(0x10141a) + .multiplyScalar(frame.hostPreExposure ?? 1); } return { scene, roomScene, reactiveScene, update, applyFrame, resetDeterministicState }; diff --git a/bench/src/benchmark/BenchmarkResolver.ts b/bench/src/benchmark/BenchmarkResolver.ts index 19a6de0..90c53bf 100644 --- a/bench/src/benchmark/BenchmarkResolver.ts +++ b/bench/src/benchmark/BenchmarkResolver.ts @@ -1,7 +1,12 @@ import type * as THREE from 'three/webgpu'; import { Upscaler } from '@pmndrs/upscaler'; -import { RCAS_LEGACY_SHADER, RCAS_SHADER } from '../../../src/shaders/rcas'; +import { + RCAS_HOISTED_EXPOSURE_SHADER, + RCAS_LEGACY_SHADER, + RCAS_PER_TAP_SHADER, + RCAS_TONEMAP_SPACE_SHADER, +} from '../../../src/shaders/rcas'; interface BenchmarkTimerBridge { readonly enabled: boolean; @@ -176,7 +181,9 @@ export function createBaselineResolver( } /** - * Creates the isolated FSR 3.1.5 RCAS numeric candidate. + * Creates the isolated FSR 3.1.5 RCAS numeric candidate. Pinned to the + * per-tap shader (the production form when these identities were measured) so + * their timings stay frozen; current production is `rcas-tonemap-space-v1`. * @param renderer - Initialized three WebGPU renderer * @param metadata - Registry metadata for the candidate identity * @returns One candidate resolver instance @@ -188,7 +195,27 @@ export function createRcasNumericParityResolver( return new BaselineBenchmarkResolver( renderer as THREE.WebGPURenderer, metadata, - RCAS_SHADER, + RCAS_PER_TAP_SHADER, + ); +} + +/** + * Creates an RCAS load-strategy experiment candidate (item 1 of + * bench/docs/NEXT-STEPS.md): production pipeline, only the RCAS shader varies. + * @param renderer - Initialized three WebGPU renderer + * @param metadata - Registry metadata carrying the experiment identity + * @returns One candidate resolver instance + */ +export function createRcasExperimentResolver( + renderer: unknown, + metadata: BenchmarkVariantMetadata, +): BenchmarkResolver { + return new BaselineBenchmarkResolver( + renderer as THREE.WebGPURenderer, + metadata, + metadata.id === 'rcas-hoisted-exposure-v1' + ? RCAS_HOISTED_EXPOSURE_SHADER + : RCAS_TONEMAP_SPACE_SHADER, ); } @@ -205,7 +232,9 @@ export function createSourceBundleResolver( return new BaselineBenchmarkResolver( renderer as THREE.WebGPURenderer, metadata, - RCAS_SHADER, + // The bundles run RCAS without FLAG_INPUT_REINHARD, where the per-tap + // and conditioned forms are identical — keep the measured identity. + RCAS_PER_TAP_SHADER, metadata.id, ); } diff --git a/bench/src/benchmark/api.ts b/bench/src/benchmark/api.ts index 6e8abcc..2ec3fbd 100644 --- a/bench/src/benchmark/api.ts +++ b/bench/src/benchmark/api.ts @@ -197,6 +197,8 @@ class BrowserBenchmarkApi implements UpscalerBenchmarkApi { const scenarioFrame = scenario.frame(Math.min(frame, scenario.endFrame)); if (scenarioFrame.resize) this._context.resize(scenarioFrame.resize); this._applyFrameState(scenarioFrame); + if (scenarioFrame.hostPreExposure !== undefined) + pipeline.setHostPreExposure(scenarioFrame.hostPreExposure); if (scenarioFrame.resetHistory) pipeline.reset(bench.scene, camera, frame); await pipeline.prepareTiming(); diff --git a/bench/src/benchmark/config.ts b/bench/src/benchmark/config.ts index 1978e2c..c0a4f43 100644 --- a/bench/src/benchmark/config.ts +++ b/bench/src/benchmark/config.ts @@ -5,11 +5,13 @@ const VARIANTS = [ 'local-baseline-through-e00-harness', 'rcas-fsr315-limiter', 'rcas-fsr315-numeric', + 'rcas-hoisted-exposure-v1', + 'rcas-tonemap-space-v1', 'source-filter-bundle-v1', 'source-structural-bundle-v1', 'source-spd-resolver-bundle-v1', ] as const; -const SCENARIOS = ['Q0', 'Q1', 'Q2', 'Q3', 'Q4', 'Q5', 'Q6', 'Q7', 'Q8', 'Q9', 'Q10'] as const; +const SCENARIOS = ['Q0', 'Q1', 'Q2', 'Q3', 'Q4', 'Q5', 'Q6', 'Q7', 'Q8', 'Q9', 'Q10', 'Q11'] as const; function numberParam(params: URLSearchParams, name: string, fallback: number): number { const raw = params.get(name); diff --git a/bench/src/benchmark/scenarios.ts b/bench/src/benchmark/scenarios.ts index 7ef47f3..e126e86 100644 --- a/bench/src/benchmark/scenarios.ts +++ b/bench/src/benchmark/scenarios.ts @@ -81,6 +81,17 @@ function q9(frame: number): BenchmarkFrameState { return { ...state(frame), directionalIntensity }; } +function q11(frame: number): BenchmarkFrameState { + // Host pre-exposure transition on an otherwise static scene: identity, + // 2.5× step at 60, hold, ramp back to 1 over 120–179. With DeltaPreExposure + // correction the shading-change view stays black through all of it and + // accumulation age never resets; output brightness simply tracks the drive. + let hostPreExposure = 1; + if (frame >= 60 && frame < 120) hostPreExposure = 2.5; + else if (frame >= 120 && frame < 180) hostPreExposure = 2.5 - (1.5 * (frame - 120)) / 59; + return { ...state(frame), hostPreExposure }; +} + function q10(frame: number): BenchmarkFrameState { const afterCut = frame >= 60; return { @@ -275,6 +286,25 @@ const SCENARIOS: Record = { unsupported: null, frame: q10, }, + Q11: { + id: 'Q11', + name: 'host-pre-exposure', + endFrame: 239, + captures: [ + '0', '23', 'P-1', 'P', '59', '60', '61', '62', '64', '68', '76', '83', + '119', '120', '121', '135', '149', '164', '179', '180', '181', '184', + '196', '203', '239', + ], + debugViews: ['final', 'accumulation-age', 'locks', 'exposure', 'shading-change'], + rois: { + full: [0, 0, 1, 1], + lit_knots: [0.18, 0.16, 0.64, 0.38], + hdr_bulb: [0.43, 0.08, 0.14, 0.2], + }, + subruns: [], + unsupported: null, + frame: q11, + }, }; /** diff --git a/bench/src/benchmark/variants.ts b/bench/src/benchmark/variants.ts index e76cee2..9877caa 100644 --- a/bench/src/benchmark/variants.ts +++ b/bench/src/benchmark/variants.ts @@ -1,5 +1,6 @@ import { createBaselineResolver, + createRcasExperimentResolver, createRcasNumericParityResolver, createSourceBundleResolver, } from './BenchmarkResolver'; @@ -31,8 +32,10 @@ function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { const structural = id === 'source-structural-bundle-v1' || id === 'source-spd-resolver-bundle-v1'; const spdResolver = id === 'source-spd-resolver-bundle-v1'; + const rcasExperiment = + id === 'rcas-hoisted-exposure-v1' || id === 'rcas-tonemap-space-v1'; const rcasLimiterParity = id === 'rcas-fsr315-limiter'; - const rcasNumericParity = rcasLimiterParity || id === 'rcas-fsr315-numeric'; + const rcasNumericParity = rcasLimiterParity || id === 'rcas-fsr315-numeric' || rcasExperiment; const rcasDenoise = id === 'rcas-fsr315-numeric'; const sourceResourceGraph = spdResolver ? [ @@ -97,6 +100,10 @@ function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { ? 'Source structural inputs/reactivity bundle v1' : id === 'source-filter-bundle-v1' ? 'Source reconstruction/filter bundle v1' + : id === 'rcas-hoisted-exposure-v1' + ? 'RCAS with hoisted exposure load' + : id === 'rcas-tonemap-space-v1' + ? 'RCAS sharpening in tonemap space' : rcasDenoise ? 'FSR 3.1.5 RCAS limiter + denoise' : rcasLimiterParity @@ -117,7 +124,7 @@ function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { }, resourceGraph: sourceBundle ? sourceResourceGraph : RESOURCE_GRAPH, pipeline: { - shaderKey: sourceBundle + shaderKey: sourceBundle || rcasExperiment ? id : rcasNumericParity ? rcasDenoise @@ -172,6 +179,14 @@ DEFAULT_DEFINITIONS.push({ metadata: metadata('rcas-fsr315-numeric'), create: createRcasNumericParityResolver, }); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-hoisted-exposure-v1'), + create: createRcasExperimentResolver, +}); +DEFAULT_DEFINITIONS.push({ + metadata: metadata('rcas-tonemap-space-v1'), + create: createRcasExperimentResolver, +}); for (const id of [ 'source-filter-bundle-v1', 'source-structural-bundle-v1', diff --git a/bench/src/main.ts b/bench/src/main.ts index 519ce96..781f658 100644 --- a/bench/src/main.ts +++ b/bench/src/main.ts @@ -118,6 +118,8 @@ const pipeline = scenario.unsupported ); if (pipeline && (scenario.id === 'Q6' || scenario.id === 'Q7' || scenario.id === 'Q8')) pipeline.configureEffectScenario(scenario.id, config.subrun, bench.roomScene, camera); +// Q11 drives an app-baked pre-exposure through the scene color + resolver. +if (pipeline && scenario.id === 'Q11') pipeline.enableHostPreExposureDrive(); function reconfigure(): void { if (!pipeline) return; diff --git a/bench/src/types/benchmark.d.ts b/bench/src/types/benchmark.d.ts index c2de2a6..c723b2d 100644 --- a/bench/src/types/benchmark.d.ts +++ b/bench/src/types/benchmark.d.ts @@ -5,6 +5,8 @@ declare type BenchmarkVariantId = | 'local-baseline-through-e00-harness' | 'rcas-fsr315-limiter' | 'rcas-fsr315-numeric' + | 'rcas-hoisted-exposure-v1' + | 'rcas-tonemap-space-v1' | 'source-filter-bundle-v1' | 'source-structural-bundle-v1' | 'source-spd-resolver-bundle-v1'; @@ -19,7 +21,8 @@ declare type BenchmarkScenarioId = | 'Q7' | 'Q8' | 'Q9' - | 'Q10'; + | 'Q10' + | 'Q11'; declare type BenchmarkDebugView = | 'final' | 'motion-vectors' @@ -201,6 +204,8 @@ declare interface BenchmarkFrameState { resetHistory: boolean; resize: BenchmarkDimensions | null; particlesVisible: boolean; + /** App-baked exposure factor driven into the scene color + resolver (Q11). */ + hostPreExposure?: number; } declare interface BenchmarkCaptureRequest { diff --git a/src/Upscaler.ts b/src/Upscaler.ts index 4f3aa8c..a528f86 100644 --- a/src/Upscaler.ts +++ b/src/Upscaler.ts @@ -685,6 +685,10 @@ export class Upscaler { const externalExposureView = inputs.exposureTexture ? getGPUTexture(this._renderer, inputs.exposureTexture).createView() : this._reactiveDummy!.createView(); + // Host pre-exposure input — the zero dummy publishes as 1.0 (inert). + const hostPreExposureView = inputs.preExposureTexture + ? getGPUTexture(this._renderer, inputs.preExposureTexture).createView() + : this._reactiveDummy!.createView(); const exposureBindGroup = this._exposurePass.createBindGroup([ { buffer: this._constants.buffer }, colorGPU.createView(), @@ -692,6 +696,7 @@ export class Upscaler { exposurePrev.createView(), exposureCur.createView(), externalExposureView, + hostPreExposureView, ]); const exposurePass = encoder.beginComputePass({ label: 'upscale-exposure', @@ -737,6 +742,7 @@ export class Upscaler { locksOut.createView(), exposureCur.createView(), reactiveView, + exposurePrev.createView(), ]); const accumulatePass = encoder.beginComputePass({ label: 'upscale-accumulate', diff --git a/src/shaders/README.md b/src/shaders/README.md index 6b826d0..025353c 100644 --- a/src/shaders/README.md +++ b/src/shaders/README.md @@ -159,6 +159,22 @@ conversion on representative noisy content. Separately benchmark native versus approximate-medium final normalization and 8×8 per-pixel dispatch versus quad-remapped coverage only if RCAS performance becomes material. +#### RCAS load domain (temporal path) + +- **Current status:** Deliberate divergence, measured (2026-07-21, NEXT-STEPS item 1). +- **Local implementation:** Sharpens the accumulate history's conditioned tonemap-space + texels directly (bounded [0,1), the range the limiter math assumes) and inverts the + conditioning once on the result. +- **FSR 3.1.5 behavior:** Sharpens in exposed linear space (applies `Exposure()` per + load, reverses it after). +- **Why it differs / evidence — Measured:** The per-tap inversion form (five + `tonemapInvert` divisions + five 1×1 exposure loads per pixel) measured RCAS at + 0.103 ms; the conditioned-space form measures 0.068 ms (**−34%**, repeatable, warm + ABBA blocks). Captures on Q0/Q1/Q3/Q9 including the HDR-bulb ROI show RMSE ≤ 1.8/255 + full-frame and ≤ 9/255 max on the brightest content — visually indistinguishable, no + overshoot. The prior per-tap shader is kept as `RCAS_PER_TAP_SHADER` for the frozen + `rcas-fsr315-limiter` benchmark identity. + #### RCAS color domain - **Current status:** Source-aligned output/color domain. @@ -177,35 +193,23 @@ when drawing to the screen; library users may instead continue linear post-proce #### Reconstruction and disocclusion (`reconstruct.ts`) -- **Current status:** Custom replacement. -- **Local implementation:** Ping-pongs bilinearly sampled linear dilated depth and applies - fixed relative thresholds in one fused render-resolution pass. -- **FSR 3.1.5 behavior:** Scatters nearest current depth into previous-frame positions with - atomics, then evaluates reconstructed samples with viewport- and depth-scaled thresholds. -- **Why it differs / evidence confidence — Rationale unclear; structural effect - verified:** The local fusion demonstrably avoids one dispatch boundary, intermediate - resource traffic, and source scatter atomics. No evidence establishes that performance - motivated the original divergence or that the reduced work is faster on target devices. - It does not preserve scatter coverage semantics; those writes require synchronization - or atomics. - -**Keep the local path** - -- **Pros:** Avoids a dispatch and intermediate traffic. Avoids atomic scatter requirements - that may be costly or awkward on some WebGPU devices. -- **Cons:** Cannot reconstruct previous-depth coverage around motion in the same way as the - source. Fixed thresholds may classify disocclusion differently by depth and resolution. - -**Adopt FSR parity** - -- **Pros:** Restores source coverage and threshold scaling, which may improve history - rejection around moving silhouettes and depth discontinuities. -- **Cons:** Adds synchronized scatter work, resources, and a pass boundary. The actual GPU - cost and quality gain are unmeasured locally. - -**Next action:** **Benchmark.** Build a distinct synchronized reconstruction variant, -capture disocclusion and accumulation-age results on controlled motion, and report its -per-pass distributions and upscaler compute-pass sum against the fused pass. +- **Current status:** Hybrid — fused pass structure kept (measured faster), AMD's + threshold formulation adopted (2026-07-21, NEXT-STEPS item 3). +- **Local implementation:** One fused render-resolution pass: nearest-depth dilation, + then per-bilinear-tap disocclusion voting against last frame's dilated depth using + AMD's viewport/depth-scaled tolerance + (`1.37e-5 · halfViewportWidth · max(depth)` — `ffx_fsr2_depth_clip.h` + `ComputeDepthClip`, taken from the GPU-verified candidate port). +- **FSR 3.1.5 behavior:** Scatters nearest current depth into previous-frame positions + with atomics, then evaluates reconstructed samples with the same viewport- and + depth-scaled thresholds in a separate pass. +- **Evidence — Measured:** The atomic scatter + separate depth-clip pass was benchmarked + in the structural candidate: +30% prepareInputs and +22% depthClip with no visible + win on Q3 (see PARITY-DECISIONS). The fused pass with AMD's thresholds shows the + expected debug signature (thin stable silhouette outlines, near-black still scenes, + age resets confined to trails) and shifts finals by RMSE ≤ 1.1/255 versus the old + fixed-threshold guess. Reconstruct pass time unchanged (0.035 ms at ratio 2). + The remaining divergence from source is scatter coverage semantics only. #### Farthest depth and motion divergence @@ -436,34 +440,28 @@ resolver variant, with debug visualization for every signal that scales the box. #### Exposure history -- **Current status:** Diverges and has a domain-consistency gap. -- **Local implementation:** Stores locally exposed, invertible-tonemapped history without - correcting reprojected history when auto, fixed, or external conditioning exposure - changes. `exposureTexture` is a conditioning override, not host `preExposure`. -- **FSR 3.1.5 behavior:** Applies `DeltaPreExposure()` and `Exposure()` while moving history - into the current working domain, then removes only `Exposure()` before output and - preserves host `preExposure`. -- **Why it differs / evidence confidence — Unclear:** The local representation lacks - previous/current exposure metadata. This is not a demonstrated optimization; it is a - domain mismatch that can compare current and history samples under different effective - exposures. - -**Keep the local path** - -- **Pros:** Requires no new exposure state and preserves the current API behavior. -- **Cons:** Changing exposure can cause pumping or trails. Treating `exposureTexture` as - host pre-exposure also divides out a domain the caller may expect preserved. - -**Adopt FSR parity** - -- **Pros:** Makes history comparisons domain-consistent and supports a proper host - pre-exposure contract. -- **Cons:** Requires explicit previous/current exposure state and careful migration of the - existing conditioning-exposure API. - -**Next action:** **Target parity.** First add deterministic tests/captures for step and -ramped exposure changes. Then track the host pre-exposure ratio separately, correct -reprojected history, and remove only internal/app conditioning exposure on output. +- **Current status:** Host pre-exposure parity adopted (2026-07-21, NEXT-STEPS item 2); + conditioning-exposure drift remains uncorrected by design. +- **Local implementation:** The pyramid publishes the app's `preExposureTexture` value + in the 1×1 exposure texel's `.b` (1.0 when absent) and meters auto-exposure + host-invariantly (divides host out of the measured average, as FSR2 divides + pre-exposure out of every input load). Accumulate reads the previous frame's texel and + ratio-corrects reprojected history in linear space across a host change — FSR3's + `DeltaPreExposure()`. Host pre-exposure stays in the output (caller domain); + conditioning exposure is still divided out. +- **FSR 3.1.5 behavior:** Same contract: `DeltaPreExposure()` corrects history, + `Exposure()` is removed before output, host `preExposure` is preserved. +- **Evidence — Measured (scenario Q11, host-pre-exposure step + ramp):** With the + correction, a 2.5× host step leaves the shading-change detector at baseline + (mean 1.19 vs quiet 1.44), never resets accumulation age (165.5 vs a 131.4 reset + without it), and output brightness tracks the drive in a single frame. Without a + `preExposureTexture` both texels publish 1.0 and captures are **byte-identical** to + the pre-change build — the correction is free for existing users. +- **Still divergent:** Conditioning (auto/fixed/external) exposure changes are not + ratio-corrected; adaptation is eased slowly (`ADAPT_SPEED`) so the per-frame mismatch + stays below the shading detector's threshold. Correcting it would change output for + all auto-exposure users and needs its own captures — revisit only with evidence of + pumping on real content. #### Locks diff --git a/src/shaders/accumulate.ts b/src/shaders/accumulate.ts index 2b1d3d3..2478610 100644 --- a/src/shaders/accumulate.ts +++ b/src/shaders/accumulate.ts @@ -30,7 +30,11 @@ import { assembleShader } from './wgsl'; * - 6: history out (rgba16float storage, display size) * - 7: locks in, display size (rgba16float; r = lifetime, g = locked luma) * - 8: locks out (rgba16float storage, display size) - * - 9: exposure, 1×1 (rgba16float; r = pre-exposure for this frame) + * - 9: exposure, 1×1 (rgba16float; r = conditioning pre-exposure for this + * frame, b = host pre-exposure) + * - 10: reactive mask, render size (r = reactivity; 1×1 dummy when absent) + * - 11: previous frame's exposure, 1×1 (b = last frame's host pre-exposure, + * for FSR3-style DeltaPreExposure history correction) */ export const ACCUMULATE_SHADER = assembleShader( WGSL_CONSTANTS, @@ -47,6 +51,7 @@ export const ACCUMULATE_SHADER = assembleShader( @group(0) @binding(8) var locksOut : texture_storage_2d; @group(0) @binding(9) var exposureTex : texture_2d; @group(0) @binding(10) var reactiveMask : texture_2d; +@group(0) @binding(11) var exposurePrevTex : texture_2d; const PI : f32 = 3.14159265358979; // How hard a fully-reactive pixel snaps to the current frame in the blend. @@ -137,7 +142,9 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { let motion = textureLoad(dilatedMotion, renderCoord, 0).xy; let disocclusion = textureLoad(masks, renderCoord, 0).r; // Pre-exposure for this frame (auto or manual) — divided back out at output. - let exposure = textureLoad(exposureTex, vec2i(0), 0).r; + // .b carries the app's host pre-exposure (1.0 when none is supplied). + let frameInfo = textureLoad(exposureTex, vec2i(0), 0); + let exposure = frameInfo.r; // Reactive mask: pixels the caller flags (particles, transparents) should // lean on the current frame instead of ghosting through history. var reactivity = 0.0; @@ -198,7 +205,22 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { return; } - let history = sampleHistoryCatmullRom(prevUV); + var history = sampleHistoryCatmullRom(prevUV); + + //* Host Pre-Exposure Delta (FSR3's DeltaPreExposure) + // If the app changed the pre-exposure baked into its render since last + // frame, the reprojected history is in the old brightness domain and would + // read as a full-screen shading change — ratio-correct it in linear space. + // Without a preExposureTexture input both texels publish 1.0 and this is + // skipped, leaving the pass bit-identical. Conditioning exposure is + // deliberately not corrected here: it adapts smoothly by design. + let hostPrev = textureLoad(exposurePrevTex, vec2i(0), 0).b; + var hostRatio = 1.0; + if (hostPrev > 1.0e-4 && frameInfo.b > 1.0e-4) { hostRatio = frameInfo.b / hostPrev; } + if (abs(hostRatio - 1.0) > 1.0e-3) { + history = vec4f(tonemapInvertible(tonemapInvert(history.rgb) * hostRatio), history.a); + } + var sampleCount = history.a * C.maxAccumulation; //* Neighborhood Statistics (YCoCg) diff --git a/src/shaders/luminancePyramid.ts b/src/shaders/luminancePyramid.ts index 7526c31..08457f6 100644 --- a/src/shaders/luminancePyramid.ts +++ b/src/shaders/luminancePyramid.ts @@ -28,6 +28,10 @@ import { assembleShader } from './wgsl'; * - 4: exposure out (rgba16float storage, 1×1) * - 5: external exposure (app-supplied; r = exposure). Read only when * FLAG_EXTERNAL_EXPOSURE is set — else a 1×1 dummy is bound and ignored. + * - 6: host pre-exposure (app-supplied; r = the pre-exposure baked into this + * frame's input color). Published in the output's .b so accumulate can + * ratio-correct history across a change (FSR3's DeltaPreExposure). A zero + * dummy (no input) publishes 1.0, keeping the correction inert. */ export const LUMINANCE_PYRAMID_SHADER = assembleShader( WGSL_CONSTANTS, @@ -38,6 +42,7 @@ export const LUMINANCE_PYRAMID_SHADER = assembleShader( @group(0) @binding(3) var prevExposure : texture_2d; @group(0) @binding(4) var exposureOut : texture_storage_2d; @group(0) @binding(5) var externalExposure : texture_2d; +@group(0) @binding(6) var hostPreExposure : texture_2d; // 32×32 = 1024 bilinear taps across the whole frame — a coarse but stable // average for exposure (each tap already averages 4 texels). @@ -67,7 +72,14 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { logSum = logSum + log2(max(luma(c), 1.0e-4)); } } - let avgLum = exp2(logSum / f32(EXPOSURE_TAPS * EXPOSURE_TAPS)); + // Meter host-invariantly (FSR2 divides pre-exposure out of every input + // load): the app already metered what it baked in, so a host pre-exposure + // step must not send auto-exposure re-adapting — that multi-frame + // conditioning drift would desynchronize history from the current frame + // and read as a full-screen shading change. + let hostRaw = textureLoad(hostPreExposure, vec2i(0), 0).r; + let host = select(1.0, hostRaw, hostRaw > 0.0); + let avgLum = exp2(logSum / f32(EXPOSURE_TAPS * EXPOSURE_TAPS)) / host; let targetExposure = clamp(EXPOSURE_KEY / max(avgLum, 1.0e-4), EXPOSURE_MIN, EXPOSURE_MAX); @@ -91,7 +103,9 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { let ext = textureLoad(externalExposure, vec2i(0), 0).r; exposure = select(exposure, ext, hasFlag(FLAG_EXTERNAL_EXPOSURE)); - textureStore(exposureOut, vec2i(0), vec4f(exposure, avgLum, 0.0, 0.0)); + // Host pre-exposure rides along in .b: 0 (the dummy) means "not supplied" + // and publishes as 1.0 so the accumulate-side ratio correction is inert. + textureStore(exposureOut, vec2i(0), vec4f(exposure, avgLum, host, 0.0)); } `, ); diff --git a/src/shaders/rcas.ts b/src/shaders/rcas.ts index da48d34..8b3f56a 100644 --- a/src/shaders/rcas.ts +++ b/src/shaders/rcas.ts @@ -1,7 +1,20 @@ import { WGSL_CONSTANTS, WGSL_TONEMAP } from './common'; import { assembleShader } from './wgsl'; -function createRcasShader(fsr315NumericParity: boolean): string { +/** + * Builds the RCAS compute shader. + * + * `conditionedInput` selects where the temporal path's tonemap/pre-exposure + * conditioning is undone. `false` reproduces the historical form: every tap + * inverts the conditioning, so the sharpening math runs in the caller's + * linear/HDR domain. `true` sharpens the accumulate history's conditioned + * [0,1) texels directly — the range the limiter math assumes — and inverts + * the conditioning once on the result. Measured ~35% cheaper on GPU with + * visually equivalent output (Q0/Q1/Q3/Q9 captures, 2026-07-21); this is the + * production form. The spatial (EASU) path is identical in both: without + * `FLAG_INPUT_REINHARD` no conditioning exists to undo. + */ +function createRcasShader(fsr315NumericParity: boolean, conditionedInput = false): string { const luma = fsr315NumericParity ? /* wgsl */ ` // FSR's inexpensive luma is scaled by two; the scale cancels in ratios. @@ -37,6 +50,37 @@ function createRcasShader(fsr315NumericParity: boolean): string { nz = clamp(abs(nz) / max(mx - mn, 1.0e-4), 0.0, 1.0); lobe *= 1.0 - 0.5 * nz; `; + const load = conditionedInput + ? /* wgsl */ ` +// Loads a display-resolution texel as-is: conditioned tonemap-space history on +// the temporal path, the caller's linear domain on the spatial path. +fn rcasLoad(p : vec2i) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + return textureLoad(inputColor, clamped, 0).rgb; +}` + : /* wgsl */ ` +// Loads a display-resolution texel in the caller's linear/HDR domain. +fn rcasLoad(p : vec2i) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + let c = textureLoad(inputColor, clamped, 0).rgb; + if (hasFlag(FLAG_INPUT_REINHARD)) { + // Undo the pre-exposure the accumulate pass baked in before tonemapping. + let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + return tonemapInvert(c) / exposure; + } + return c; +}`; + const resolve = conditionedInput + ? /* wgsl */ ` + var pix = (lobe * b + lobe * d + lobe * h + lobe * f + e) * rcpL; + if (hasFlag(FLAG_INPUT_REINHARD)) { + // Undo the accumulate conditioning once on the sharpened result: invert + // the tonemap, then divide out the baked-in pre-exposure. + let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + pix = tonemapInvert(max(pix, vec3f(0.0))) / exposure; + }` + : /* wgsl */ ` + let pix = (lobe * b + lobe * d + lobe * h + lobe * f + e) * rcpL;`; return assembleShader( WGSL_CONSTANTS, @@ -49,18 +93,7 @@ function createRcasShader(fsr315NumericParity: boolean): string { // Maximum sharpening lobe magnitude — set so a single tap cannot exceed the // local contrast ring (0.25 - 1/16 in the reference). const RCAS_LIMIT : f32 = 0.25 - (1.0 / 16.0); - -// Loads a display-resolution texel in the caller's linear/HDR domain. -fn rcasLoad(p : vec2i) -> vec3f { - let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); - let c = textureLoad(inputColor, clamped, 0).rgb; - if (hasFlag(FLAG_INPUT_REINHARD)) { - // Undo the pre-exposure the accumulate pass baked in before tonemapping. - let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); - return tonemapInvert(c) / exposure; - } - return c; -} +${load} @compute @workgroup_size(8, 8) fn main(@builtin(global_invocation_id) gid : vec3u) { @@ -101,7 +134,7 @@ ${denoise} //* Resolve let rcpL = 1.0 / (4.0 * lobe + 1.0); - let pix = (lobe * b + lobe * d + lobe * h + lobe * f + e) * rcpL; +${resolve} textureStore(outputColor, gid.xy, vec4f(pix, 1.0)); } @@ -109,12 +142,110 @@ ${denoise} ); } +function createRcasExperimentShader(loadStrategy: 'hoisted' | 'tonemap-space'): string { + const base = createRcasShader(true); + if (loadStrategy === 'hoisted') { + // Same math as production, but the uniform 1×1 exposure load is hoisted + // out of the tap function and the per-tap division folded to a multiply. + return base + .replace( + `fn rcasLoad(p : vec2i) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + let c = textureLoad(inputColor, clamped, 0).rgb; + if (hasFlag(FLAG_INPUT_REINHARD)) { + // Undo the pre-exposure the accumulate pass baked in before tonemapping. + let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + return tonemapInvert(c) / exposure; + } + return c; +}`, + `fn rcasLoad(p : vec2i, rcpExposure : f32) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + let c = textureLoad(inputColor, clamped, 0).rgb; + if (hasFlag(FLAG_INPUT_REINHARD)) { + return tonemapInvert(c) * rcpExposure; + } + return c; +}`, + ) + .replace( + ` let sp = vec2i(gid.xy);`, + ` let sp = vec2i(gid.xy); + // Undo the pre-exposure the accumulate pass baked in — once, not per tap. + var rcpExposure = 1.0; + if (hasFlag(FLAG_INPUT_REINHARD)) { + rcpExposure = 1.0 / max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + }`, + ) + .replace(/rcasLoad\(sp( \+ vec2i\(-?\d, -?\d\))?\)/g, (m) => + m.replace(/\)$/, ', rcpExposure)'), + ); + } + // tonemap-space: sharpen the raw tonemapped history texels (bounded [0,1), + // the range RCAS's limiter math assumes) and invert once on the result. + return base + .replace( + `fn rcasLoad(p : vec2i) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + let c = textureLoad(inputColor, clamped, 0).rgb; + if (hasFlag(FLAG_INPUT_REINHARD)) { + // Undo the pre-exposure the accumulate pass baked in before tonemapping. + let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + return tonemapInvert(c) / exposure; + } + return c; +}`, + `fn rcasLoad(p : vec2i) -> vec3f { + let clamped = clamp(p, vec2i(0), vec2i(C.displaySize) - 1); + return textureLoad(inputColor, clamped, 0).rgb; +}`, + ) + .replace( + ` let pix = (lobe * b + lobe * d + lobe * h + lobe * f + e) * rcpL; + + textureStore(outputColor, gid.xy, vec4f(pix, 1.0));`, + ` var pix = (lobe * b + lobe * d + lobe * h + lobe * f + e) * rcpL; + if (hasFlag(FLAG_INPUT_REINHARD)) { + // Undo the conditioning once on the sharpened result instead of per tap. + let exposure = max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4); + pix = tonemapInvert(max(pix, vec3f(0.0))) / exposure; + } + + textureStore(outputColor, gid.xy, vec4f(pix, 1.0));`, + ); +} + /** * Legacy RCAS shader retained only for benchmark comparisons. */ export const RCAS_LEGACY_SHADER = createRcasShader(false); /** - * Production RCAS shader with FSR 3.1.5 lower-limiter and denoise parity. + * The pre-2026-07-21 production shader (FSR 3.1.5 numeric parity, per-tap + * conditioning inversion), retained so the `rcas-fsr315-limiter` / + * `rcas-fsr315-numeric` benchmark identities stay frozen and reproducible. + */ +export const RCAS_PER_TAP_SHADER = createRcasShader(true); + +/** + * Production RCAS shader: FSR 3.1.5 lower-limiter and denoise parity, + * sharpening in conditioned tonemap space with a single inversion on the + * result (~35% cheaper than the per-tap form, visually equivalent — see + * bench/docs/NEXT-STEPS.md item 1). + */ +export const RCAS_SHADER = createRcasShader(true, true); + +/** + * Benchmark candidate: production math with the exposure load hoisted out of + * the tap function (one 1×1 load + reciprocal per pixel instead of five + * loads + divisions). Output is visually identical to {@link RCAS_SHADER}. + */ +export const RCAS_HOISTED_EXPOSURE_SHADER = createRcasExperimentShader('hoisted'); + +/** + * Benchmark candidate: sharpens in invertible-tonemap space (plain per-tap + * loads, like the source resolver's RCAS) and inverts conditioning once on + * the result. Output differs subtly from {@link RCAS_SHADER} — needs capture + * validation before any adoption. */ -export const RCAS_SHADER = createRcasShader(true); +export const RCAS_TONEMAP_SPACE_SHADER = createRcasExperimentShader('tonemap-space'); diff --git a/src/shaders/reconstruct.ts b/src/shaders/reconstruct.ts index f5dd758..b722f45 100644 --- a/src/shaders/reconstruct.ts +++ b/src/shaders/reconstruct.ts @@ -35,27 +35,12 @@ export const RECONSTRUCT_SHADER = assembleShader( @group(0) @binding(5) var dilatedMotion : texture_storage_2d; @group(0) @binding(6) var maskOutput : texture_storage_2d; -// Relative separation (fraction of current view depth) treated as a full -// disocclusion. Depth differences below ~1.5% are considered the same surface, -// absorbing depth-buffer quantization and dilation error. -const DEPTH_SEPARATION_SCALE : f32 = 0.066; -const DEPTH_SIMILARITY_FLOOR : f32 = 0.015; - -// Manual bilinear fetch — r32float is not filterable, but linear view-space -// depth interpolates correctly by hand. -fn samplePreviousDepth(uv : vec2f) -> f32 { - let pos = uv * C.renderSize - 0.5; - let base = floor(pos); - let frac = pos - base; - let maxCoord = vec2i(C.renderSize) - 1; - let p00 = clamp(vec2i(base), vec2i(0), maxCoord); - let p11 = clamp(vec2i(base) + 1, vec2i(0), maxCoord); - let d00 = textureLoad(previousDepth, p00, 0).r; - let d10 = textureLoad(previousDepth, vec2i(p11.x, p00.y), 0).r; - let d01 = textureLoad(previousDepth, vec2i(p00.x, p11.y), 0).r; - let d11 = textureLoad(previousDepth, p11, 0).r; - return mix(mix(d00, d10, frac.x), mix(d01, d11, frac.x), frac.y); -} +// AMD's separation tolerance (ffx_fsr2_depth_clip.h): the minimum view-depth +// gap that reads as a different surface scales with viewport resolution and +// scene depth, absorbing depth-buffer quantization without a scene-tuned guess. +const DEPTH_SEPARATION_CONSTANT : f32 = 1.37e-5; +// Bilinear taps lighter than this cannot vote (matches the reference). +const DEPTH_TAP_WEIGHT_FLOOR : f32 = 6.1e-4; @compute @workgroup_size(8, 8) fn main(@builtin(global_invocation_id) gid : vec3u) { @@ -89,18 +74,50 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { textureStore(dilatedMotion, gid.xy, vec4f(uvDelta, 0.0, 0.0)); //* Depth Clip — disocclusion from the just-dilated depth + motion. + // AMD's formulation (ffx_fsr2_depth_clip.h ComputeDepthClip, via the + // GPU-verified candidate port): each bilinear tap of last frame's dilated + // depth votes a confidence that its separation from the current depth is + // within the viewport/depth-scaled tolerance; disocclusion is the weighted + // complement, and only positive separations (current surface was occluded) + // count at all. let uv = (vec2f(gid.xy) + 0.5) * C.renderSizeInv; let prevUV = uv - uvDelta; if (any(prevUV < vec2f(0.0)) || any(prevUV > vec2f(1.0))) { textureStore(maskOutput, gid.xy, vec4f(1.0, 0.0, 0.0, 1.0)); return; } - // History is invalid when the previous surface was meaningfully nearer than - // the current one (i.e. the current surface was occluded). - let prevDepth = samplePreviousDepth(prevUV); - let separation = max(0.0, curDepth - prevDepth); - let relative = max(0.0, separation / max(curDepth, 1.0e-4) - DEPTH_SIMILARITY_FLOOR); - let disocclusion = clamp(relative / DEPTH_SEPARATION_SCALE, 0.0, 1.0); + let samplePosition = prevUV * C.renderSize - 0.5; + let base = vec2i(floor(samplePosition)); + let fraction = fract(samplePosition); + let offsets = array(vec2i(0, 0), vec2i(1, 0), vec2i(0, 1), vec2i(1, 1)); + let weights = vec4f( + (1.0 - fraction.x) * (1.0 - fraction.y), + fraction.x * (1.0 - fraction.y), + (1.0 - fraction.x) * fraction.y, + fraction.x * fraction.y + ); + let halfViewportWidth = length(C.renderSize * 0.5); + var separationConfidence = 0.0; + var weightSum = 0.0; + var potentialDisocclusion = true; + for (var index = 0; index < 4; index++) { + let weight = weights[index]; + if (weight <= DEPTH_TAP_WEIGHT_FLOOR) { continue; } + let p = clamp(base + offsets[index], vec2i(0), maxCoord); + let prevDepth = textureLoad(previousDepth, p, 0).r; + let difference = curDepth - prevDepth; + potentialDisocclusion = potentialDisocclusion && difference > 1.175e-38; + if (potentialDisocclusion) { + let required = DEPTH_SEPARATION_CONSTANT * halfViewportWidth * max(curDepth, prevDepth); + separationConfidence += clamp(required / max(difference, 1.0e-7), 0.0, 1.0) * weight; + weightSum += weight; + } + } + let disocclusion = select( + 0.0, + clamp(1.0 - separationConfidence / max(weightSum, 1.0e-6), 0.0, 1.0), + potentialDisocclusion && weightSum > 0.0, + ); textureStore(maskOutput, gid.xy, vec4f(disocclusion, 0.0, 0.0, 1.0)); } `, diff --git a/src/shaders/shaders.test.ts b/src/shaders/shaders.test.ts index 23de986..0da7a74 100644 --- a/src/shaders/shaders.test.ts +++ b/src/shaders/shaders.test.ts @@ -41,7 +41,12 @@ import { DEBUG_SHADER } from './debug'; import { EASU_SHADER } from './easu'; import { GENERATE_REACTIVE_SHADER } from './generateReactive'; import { LUMINANCE_PYRAMID_SHADER } from './luminancePyramid'; -import { RCAS_LEGACY_SHADER, RCAS_SHADER } from './rcas'; +import { + RCAS_HOISTED_EXPOSURE_SHADER, + RCAS_LEGACY_SHADER, + RCAS_SHADER, + RCAS_TONEMAP_SPACE_SHADER, +} from './rcas'; import { RECONSTRUCT_SHADER } from './reconstruct'; import { assembleShader } from './wgsl'; @@ -61,8 +66,8 @@ const BASELINE_BINDING_COUNTS: Record = { easu: 3, rcas: 4, reconstruct: 7, - accumulate: 11, - luminancePyramid: 6, + accumulate: 12, + luminancePyramid: 7, generateReactive: 4, debug: 10, }; @@ -70,10 +75,13 @@ const BASELINE_BINDING_COUNTS: Record = { const BASELINE_FINGERPRINTS: Record = { blit: '673108e1', easu: '11632358', - rcas: 'c803572b', - reconstruct: '1ced83aa', - accumulate: 'd0973222', - luminancePyramid: '7a806c41', + // Updated 2026-07-21: conditioned-space sharpening adopted (NEXT-STEPS item 1). + rcas: '51bd6d54', + // Updated 2026-07-21: AMD viewport/depth-scaled disocclusion (NEXT-STEPS item 3). + reconstruct: '19104db7', + // Updated 2026-07-21: DeltaPreExposure history correction (NEXT-STEPS item 2). + accumulate: 'df540efa', + luminancePyramid: 'e4b7a644', generateReactive: '6ed4b549', debug: 'e30ebd6c', }; @@ -314,6 +322,38 @@ describe('FSR 3.1.5 RCAS numeric parity', () => { }); }); +describe('RCAS load-strategy experiment shaders', () => { + // These are built by string transforms over the production shader — a + // silently missed replacement would produce WGSL that only fails on a real + // device. Assert every transform actually landed. + it('hoisted-exposure moves the exposure load out of the tap function', () => { + expect(RCAS_HOISTED_EXPOSURE_SHADER).toContain( + 'fn rcasLoad(p : vec2i, rcpExposure : f32) -> vec3f {', + ); + expect(RCAS_HOISTED_EXPOSURE_SHADER).toContain( + 'rcpExposure = 1.0 / max(textureLoad(exposureTex, vec2i(0), 0).r, 1.0e-4);', + ); + expect(RCAS_HOISTED_EXPOSURE_SHADER).toContain('rcasLoad(sp, rcpExposure)'); + expect(RCAS_HOISTED_EXPOSURE_SHADER).toContain( + 'rcasLoad(sp + vec2i(0, -1), rcpExposure)', + ); + expect(RCAS_HOISTED_EXPOSURE_SHADER).not.toContain('tonemapInvert(c) / exposure'); + // Same sharpening math as production. + expect(RCAS_HOISTED_EXPOSURE_SHADER).toContain('lowerLimiterMultiplier'); + }); + + it('tonemap-space uses plain tap loads and inverts once on the result', () => { + expect(RCAS_TONEMAP_SPACE_SHADER).toContain( + 'return textureLoad(inputColor, clamped, 0).rgb;\n}', + ); + expect(RCAS_TONEMAP_SPACE_SHADER).not.toContain('tonemapInvert(c)'); + expect(RCAS_TONEMAP_SPACE_SHADER).toContain( + 'pix = tonemapInvert(max(pix, vec3f(0.0))) / exposure;', + ); + expect(RCAS_TONEMAP_SPACE_SHADER).toContain('lowerLimiterMultiplier'); + }); +}); + describe('linear HDR output domain', () => { it('keeps presentation transforms out of upscaling shaders', () => { for (const source of [BLIT_SHADER, EASU_SHADER, RCAS_SHADER]) { diff --git a/src/types.ts b/src/types.ts index f362fce..36e8c97 100644 --- a/src/types.ts +++ b/src/types.ts @@ -164,11 +164,14 @@ export interface DispatchInputs { */ exposureTexture?: Texture; /** - * Optional host pre-exposure texture (red texel, typically 1×1). Unlike - * {@link exposureTexture}, this factor is part of the caller's color - * domain and is therefore preserved at output. Source-style candidates - * track its previous/current ratio to correct reprojected history. - * Omission is equivalent to `1`. + * Optional host pre-exposure texture (red texel, typically 1×1): the + * exposure factor the app has already baked into this frame's input color. + * Unlike {@link exposureTexture}, this factor is part of the caller's + * color domain and is therefore preserved at output. The temporal path + * tracks its previous/current ratio and corrects reprojected history + * across a change (FSR3's `DeltaPreExposure`), so stepping or ramping the + * host exposure does not read as a full-screen shading change. Omission is + * equivalent to `1`. */ preExposureTexture?: Texture; /** Drop all history this frame (camera cut, teleport, resize). */ From 1ec104ee0b10fa987b3a6a6bcfd9227afeb95b25 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 17:26:43 +0900 Subject: [PATCH 05/22] feat: multi-scale shading-change detector (Phase-5 SPD item, NEXT-STEPS 4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces accumulate's inline 3x3-neighborhood shading heuristic with src/shaders/shadingChange.ts: one fused half-resolution dispatch whose 8x8 workgroup covers a 16x16 render tile, making the 4x4 and 8x8 block reductions workgroup-local — each luma comparison is evaluated once with hoisted frame-info loads, no mip textures, no separate resolve pass. The pass maintains a 1-frame luma history, reprojects it jitter-delta-aligned (bilinear), neutralizes disoccluded texels, and compares block-MEAN luma per scale gated by a base + contrast-scaled (coefficient of variation) noise floor. The strongest gated scale feeds accumulate's existing SHADING_AGE aging path (new binding 12; zero dummy when settings.detectShadingChanges is off — the pass simply isn't dispatched). Locks keep their self-referential break, untouched. Five GPU tuning iterations (evidence in bench/results/raw/E00/pre-spd-* and post-spd-v*): the source-style mean-of-per-texel-signed-ratios floors at ~0.10 still-scene response — the ratio metric weights the darker side of alias residue, a coherent bias signed averaging cannot cancel — and neither jitter alignment nor bilinear reprojection alone fixes it. Averaging luma BEFORE the ratio, adaptive floors, and dropping the flicker-dominated 2x2 scale hit the full acceptance matrix: - Q1 still scene: mean ~2/255 (old detector ~1.1) — near-black - Q4 camera orbit over high-frequency content: worst 3.6 vs old 4.9 — measurably FEWER false positives (the item's whole point) - Q9 light steps: clean single-frame spikes at 137/113 per 255 (old: 84 with a ~20-frame decay tail); slow ramps deliberately quiet (1-frame comparison; blend tracks ramps — Q9 ramp finals show no lag, RMSE 0.78) - Q11 host pre-exposure step: quiet (hostRatio correction inside the pass) - Finals within 1.6/255 RMSE of the old detector everywhere Cost: 0.044 ms at ratio 2 — 5x cheaper than the candidate's two-pass form (0.231 ms measured). Examples 01/07/09 GPU-smoke clean. Bench gains the 'shadingChange' timing label. SHADING_LO/HI are gone (the floors live in shadingChange.ts); SHADING_AGE stays in accumulate.ts. Docs updated: CLAUDE.md (roadmap item closed, landmine + debug-protocol entries rewritten), shaders README (adopted section with the rejected designs and why), NEXT-STEPS item 4, PARITY.md. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 17 +-- PARITY.md | 16 ++- bench/docs/NEXT-STEPS.md | 44 +++++--- bench/src/benchmark/variants.ts | 3 +- src/Upscaler.ts | 58 ++++++++++ src/shaders/README.md | 66 ++++++------ src/shaders/accumulate.ts | 28 +++-- src/shaders/shaders.test.ts | 9 +- src/shaders/shadingChange.ts | 185 ++++++++++++++++++++++++++++++++ 9 files changed, 342 insertions(+), 84 deletions(-) create mode 100644 src/shaders/shadingChange.ts diff --git a/CLAUDE.md b/CLAUDE.md index 97ffbc3..a8f317e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,7 +39,12 @@ pre-exposure (`preExposureTexture`) is honored end-to-end — DeltaPreExposure h correction + host-invariant auto-exposure metering, validated on the new **Q11** scenario, byte-identical when absent. (3) The reconstruct pass uses AMD's viewport/depth-scaled disocclusion (per-tap confidence voting) inside the fused -single pass. Only NEXT-STEPS item 4 (Phase-5 SPD detector) remains. Candidate +single pass. **Item 4 (the Phase-5 SPD detector) landed the same session**: the +3×3-neighborhood shading heuristic is replaced by `shadingChange.ts` — a fused +multi-scale block-mean detector (0.044 ms, 5× cheaper than the candidate's +two-pass form, measurably fewer false positives under motion; five GPU tuning +iterations documented in NEXT-STEPS). Nothing from the parity program remains +open. Candidate A/B runs: `node scripts/run-benchmark.mjs --smoke --variant --comparison ` (see `--help`). @@ -92,7 +97,7 @@ src/ shaders/ common.ts — shared WGSL chunks: FsrConstants UBO, color/depth/tonemap helpers, FLAG_* bits wgsl.ts — assembleShader() dedup concatenator (WGSL has no #include) - blit / easu / rcas / reconstruct / accumulate / luminancePyramid / generateReactive / debug .ts — the passes + blit / easu / rcas / reconstruct / shadingChange / accumulate / luminancePyramid / generateReactive / debug .ts — the passes README.md — per-pass fidelity vs FidelityFX reference + debugging guide internal/ threeWebGPU.ts — getDevice() / getGPUTexture(): the three-internals bridge @@ -150,7 +155,7 @@ These were discovered by reading three's source; they're non-obvious and easy to - **Motion vector sign/scale.** `motionScale = (0.5, -0.5)` converts the `velocity` node's NDC delta to a UV delta, and reprojection is `prevUV = uv - motion`. **Corroborated against three's own `TAAUNode`** (2026-07-08), which uses the identical `velocity.xy * vec2(0.5, -0.5)` scale and `historyUV = uv - offset` reprojection, plus the same `setViewOffset` jitter in render-pixel units — so this is no longer a guess. Still the first thing to re-check if history smears or tears under camera motion; **Motion Vectors debug view is your friend.** - **Accumulation tuning.** Catmull-Rom history filter, YCoCg variance-clip gamma (`CLIP_GAMMA = 1.0`), disocclusion/clip weight falloffs, **and the luminance-lock constants (`LOCK_*`)** in `accumulate.ts` are sensible defaults, not tuned. Ghosting → tighten (lower `LOCK_CLAMP_RELAX`/`LOCK_HISTORY_BOOST`, raise the peak/contrast thresholds); instability/shimmer or thin features dimming → loosen. Use `DebugView.Locks` to see where locks form. -- **Shading-change constants** (`accumulate.ts`: `SHADING_LO/HI/AGE`) are conservative defaults. If stable, steadily-lit surfaces shimmer or fail to converge, `SHADING_LO` is too low (it's aging history that didn't change) — raise it; if a genuine lighting change ghosts its old shading, lower it. `DebugView.ShadingChange` should be black on a still scene — check it before suspecting the accumulate blend. The detector reuses the 3×3 neighborhood mean, not a coarse pyramid mip, so it can false-positive on very high-frequency content under heavy motion; that's the Phase-5 SPD-mip refinement. +- **Shading-change tuning** (`shadingChange.ts`: `SHADING_FLOOR_MID/COARSE/CV`; `accumulate.ts`: `SHADING_AGE`) — GPU-tuned on Q1/Q4/Q9/Q11 (2026-07-21) but still constants, not laws. Ghosting after a genuine lighting change → lower the floors; flat steadily-lit surfaces shimmering → raise them (raise `SHADING_FLOOR_CV` if the noise sits on textured regions). `DebugView.ShadingChange` should be near-black on a still scene — check it before suspecting the accumulate blend. Slow ramps deliberately don't fire (1-frame comparison; the blend tracks ramps — verified no lag on Q9). - **Auto-exposure constants** (`luminancePyramid.ts`: `EXPOSURE_KEY`, `EXPOSURE_MIN/MAX`, `ADAPT_SPEED`) are defaults. Because exposure is divided back out before display, the *visible* effect is subtle (better accumulation stability, not a brightness change). If a scene pulses in brightness, `ADAPT_SPEED` is the suspect; if the image goes flat/washed on a very bright or dark scene, check the min/max clamp. `DebugView.Exposure` should read near mid-grey — verify there before suspecting the accumulate math. - ~~**Depth separation threshold** in `reconstruct.ts` is a guess~~ — **resolved 2026-07-21**: replaced by AMD's viewport/depth-scaled formulation (`1.37e-5 · halfViewportWidth · maxDepth`, per-bilinear-tap confidence voting from `ffx_fsr2_depth_clip.h`), GPU-validated on Q3 (thin stable outlines, still scenes quiet, age resets confined to trails). No scene-tuned constants remain in this pass. - **`timestamp-query`** may be absent; `GpuTimer` no-ops gracefully, but confirm the GPU-ms readout actually appears where supported. @@ -166,7 +171,7 @@ These were discovered by reading three's source; they're non-obvious and easy to 3. **Accumulation age** — should saturate to white within ~1s when still, reset along disocclusion trails. Never whitening ⇒ history not persisting (ping-pong or reset logic). 4. **Locks** — lights up on thin high-contrast features (grid lines, wire/fence edges, specular silhouettes), black on flat surfaces. All-black ⇒ thresholds too high (thin features dim); lit everywhere ⇒ too low (ghosting). 5. **Exposure** — exposed scene luma should read near mid-grey everywhere. All-black/all-white ⇒ exposure pinned at its min/max clamp. -6. **Shading change** — black on a static steadily-lit scene; lights up on surfaces whose shading actually changes (moving specular, animating light). Lit everywhere while still ⇒ `SHADING_LO` too low. +6. **Shading change** — black on a static steadily-lit scene; fires as clean single-frame spikes on light steps (moving specular, animating light). Lit everywhere while still ⇒ `SHADING_FLOOR_*` too low (see `shadingChange.ts`). 7. **Reactivity** — the caller's reactive mask (white on flagged transparents/particles, black on opaque). Empty/misaligned ⇒ the mask isn't authored/passed right. Full guide in `src/shaders/README.md`. @@ -185,7 +190,7 @@ variant on `06-screenspace-gi`, and expose new toggles in `02-fsr1-vs-fsr3`. Rem - ~~**Luminance-stability locks**~~ — **done & GPU-verified.** Persistent display-res lock buffer in `accumulate.ts` (r = lifetime, g = locked luma), reprojected through motion; detects thin luminance outliers, grows a lock while present, breaks on disocclusion/shading change, then widens the rectification AABB + boosts history for locked pixels. Toggle `settings.lockThinFeatures` (`FLAG_LOCKS`); inspect via `DebugView.Locks`. Tuning constants (top of `accumulate.ts`) are defaults, not final — tighten if thin features ghost, loosen if they still dim. - ~~**Luminance pyramid + auto-exposure**~~ — **done.** `luminancePyramid.ts` reduces the scene to a single log-average luminance (one-workgroup 32×32-tap reduction, not yet an SPD mip chain) → a pre-exposure eased over time (eye-adaptation). `accumulate.ts` pre-exposes the input before the invertible tonemap; `rcas.ts`/`blit.ts` divide it back out before display — so HDR scenes of very different brightness accumulate in the same well-conditioned range **without changing final brightness**. This also fixed a latent bug: the temporal path previously baked in `settings.exposure` and never divided it out. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); inspect via `DebugView.Exposure`. Constants (top of `luminancePyramid.ts`: key/min/max/adapt-speed) are defaults. The SPD mip chain is deferred until the shading-change detector needs the intermediate mips. - **External exposure input** — **done & GPU-verified.** An app that meters its own exposure feeds it via `dispatch({ exposureTexture })` (value in the red texel, any float format); it overrides both auto and fixed exposure and is still divided back out before display (conditions accumulation, not brightness). Wired as binding 5 of the pyramid pass behind `FLAG_EXTERNAL_EXPOSURE` (the 1024 bit) and funnelled through the same single `select`, so downstream passes are untouched and `avgLum` stays our own measurement for the shading detector. Bound to the reactive dummy as a placeholder when absent. Also on the composable node as `options.exposureTexture` (mirrors FSR3's `exposure` dispatch resource). - - ~~**Shading-change detector**~~ — **done & GPU-verified.** `accumulate.ts` compares the reprojected history luma to the current 3×3 neighborhood mean, normalized by the neighborhood's own variance; a coherent disagreement the variance can't explain reads as a genuine shading change (light turning on, animated material) and ages **non-locked** history so it re-converges. A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with the background-dominated neighborhood mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`); inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). **Simplification:** uses the neighborhood mean, not a dedicated coarse pyramid mip — a true SPD mip (steadier on high-frequency content) is deferred to Phase 5. Constants `SHADING_LO/HI/AGE` (top of `accumulate.ts`) are conservative defaults. + - ~~**Shading-change detector**~~ — **done & GPU-verified; upgraded to the multi-scale form 2026-07-21.** `shadingChange.ts` (one fused half-res dispatch) compares jitter-aligned block-mean luma at 4×4/8×8 render scales against a 1-frame luma history, with contrast-adaptive noise floors and disocclusion neutralization; the response ages **non-locked** history via accumulate's `SHADING_AGE` path so changed surfaces re-converge. A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with its block mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`) — the pass isn't dispatched when off; inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). - ~~**Reactive-mask input**~~ — **done & GPU-verified.** Optional `dispatch({ reactive })` render-res mask (red = reactivity); flagged pixels suppress locks, keep near-zero accumulation, and snap to the current frame (`REACTIVE_STRENGTH` in `accumulate.ts`). No mask → a 1×1 zero texture is bound and `FLAG_REACTIVE` stays off (zero cost). `UpscalePresenter.setReactiveMask()` threads it through; `examples/05-transparency` authors one by rendering the transparents' coverage and is the acceptance demo. Inspect via `DebugView.Reactivity`. **Authoring helpers** (auto-generating the mask) — **done**, see the next bullet. **Composable-node parity** — **done.** `upscale()` / `UpscalerNode` now take `options.reactive` and `options.reactiveOpaqueColor` texture nodes (registered as graph deps so the opaque buffer renders in-pipeline, jittered, aligned with color), closing an asymmetry with the raw `Upscaler`/`UpscalePass`. No dedicated node demo yet — it reuses the GPU-verified reactive dispatch (example 05, imperative) + the proven graph-dep mechanism (examples 07/09), so worst case on a plumbing miss is a silent no-op, not a crash. - **Phase 4 — API & ecosystem:** - ~~**RCAS denoise variant**~~ — **done & GPU-verified.** `rcas.ts` gains FSR1's `FSR_RCAS_DENOISE` path (attenuate the sharpening lobe on lone luma outliers so grain from noisy inputs isn't amplified), gated by `settings.rcasDenoise` (`FLAG_RCAS_DENOISE`, off by default). Pairs with an upstream spatial denoiser; `examples/06-screenspace-gi` toggles it on for the reduced-res SSR/GI. One of Dennis's original asks. @@ -202,7 +207,7 @@ variant on `06-screenspace-gi`, and expose new toggles in `02-fsr1-vs-fsr3`. Rem - **Resolved** (was an open question): the composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. - ~~MSAA-input support~~ — **dropped by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). - **Phase 5 — performance & the one remaining quality item:** ~~merge dilate+depth-clip into one pass~~ **done** (`reconstruct.ts` — one render-res dispatch, one fewer intermediate round-trip; GPU-verified disocclusion unchanged). Remaining, all deferred as of 2026-07-10 (the rename + node-parity + exposure work above was the safe, high-value batch; these were held back so a core-path change wouldn't ride onto the rename merge): - - **True SPD luminance mip chain + shading-change coarse mip** — *the one remaining quality gain.* A real Single-Pass-Downsample mip chain would let the shading-change detector sample a coarse mip instead of the current 3×3 neighborhood mean, which steadies it on high-frequency content under heavy motion (fewer false positives → less needless history aging). But it's a **core-accumulate refactor with genuine tuning risk** (allocate the mip chain, bind the coarse mip, rewire + re-tune the detector, GPU-A/B it) — the kind of thing where an untuned constant silently regresses. Deserves its own focused session with heavy GPU tuning, **not** a tail-of-session add. This is the next thing to pick up. + - ~~**True SPD luminance mip chain + shading-change coarse mip**~~ — **done 2026-07-21** (`shadingChange.ts`, NEXT-STEPS item 4). The coarse-mip comparison exists as a fused workgroup-local reduction rather than a literal mip chain (one dispatch, no mip textures, 0.044 ms); it measured *fewer* false positives than the 3×3 heuristic on high-frequency content under camera motion (Q4) with clean single-frame response to light steps (Q9). The predicted tuning risk was real — five GPU iterations, all documented in `bench/docs/NEXT-STEPS.md`. - **Perf-only (no quality gain, correctness risk):** `textureGather` tap packing (EASU/RCAS currently use per-tap `textureLoad` — but these are the AMD-faithful ports; changing their sampling risks subtle artifacts); f16 arithmetic (`shader-f16`, needs feature detection + fallback, precision risk); bind-group caching (rebuilt per dispatch — "fine but wasteful," but caching adds stale-view-on-resize risk to the core); half-res luma analysis. None gain image quality; each adds risk to a core path the project deliberately protects — do them only when perf is the actual bottleneck. - **Future project (deferred) — fused GI/denoise temporal path.** Denoising very noisy screen-space inputs (SSGI especially) can't be solved by stacking a *separate* temporal denoiser before FSR3: any second temporal resolver reprojects by velocity, which is jitter-free, so it can't see FSR3's sub-pixel jitter — it rejects the misaligned history (noise survives) and cancels the jitter variance FSR3 needs (aliasing returns). Verified 2026-07-10 with three r185's `recurrentDenoise`/`temporalReproject` in `examples/10-ssgi-denoise` (kept as **experimental documentation**, not a library feature). Spatial-only denoise + FSR3-owns-temporal avoids the conflict but inherits the third-party à-trous kernel's halos/step-lines/update-cadence skipping. The real fix is to **fuse GI history into FSR3's own accumulation** — reprojected with *our* motion vectors, sampled at *our* jitter, with GI-appropriate variance handling (not the AA-tuned clip). This is genuine R&D that touches the core accumulate pass, so it's a deliberate future effort, not a quick add. Priority remains the FSR3 upscaler itself; don't spend core complexity bending upstream nodes to it. diff --git a/PARITY.md b/PARITY.md index 972f3fe..424aed8 100644 --- a/PARITY.md +++ b/PARITY.md @@ -104,12 +104,10 @@ claims can be re-tested; a repeatable ≥5% result is treated as actionable, <3% ## Open items we do intend to converge -Three of the four post-parity adoption items landed on 2026-07-21 (host -pre-exposure correction, the AMD disocclusion constant, and the RCAS cost -investigation — resolved as the conditioned-space load domain, −34% on the pass). -One remains: - -- **A coarse-mip shading-change detector** (the source design, GPU-proven in the - resolver candidate) to replace the 3×3-neighborhood heuristic, which can - false-positive on high-frequency content under heavy motion. Scheduled as its own - session (`bench/docs/NEXT-STEPS.md`, item 4). +All four post-parity adoption items landed on 2026-07-21: the RCAS cost +investigation (resolved as the conditioned-space load domain, −34% on the pass), +host pre-exposure correction, the AMD disocclusion constant, and the multi-scale +shading-change detector (the source's coarse-mip design, re-derived as a single +fused pass — 0.044 ms vs the source-style candidate's 0.231 ms, with measurably +fewer false positives than the old 3×3 heuristic on high-frequency content under +motion). Details and evidence: `bench/docs/NEXT-STEPS.md`. diff --git a/bench/docs/NEXT-STEPS.md b/bench/docs/NEXT-STEPS.md index d4bc5c4..bee00d3 100644 --- a/bench/docs/NEXT-STEPS.md +++ b/bench/docs/NEXT-STEPS.md @@ -56,25 +56,35 @@ resolver ran with the flag off (plain loads). resets confined to trails; finals shift RMSE ≤ 1.1/255; reconstruct pass time unchanged (0.035 ms at ratio 2). -## 4. Phase-5 SPD session: coarse-mip shading-change detector — OPEN +## 4. Phase-5 SPD session: coarse-mip shading-change detector — DONE -The roadmap item in [/CLAUDE.md](../../CLAUDE.md) ("True SPD luminance mip chain + -shading-change coarse mip"). Start from the GPU-proven candidate implementation — -`SHADING_CHANGE_SPD` + 3-mip resolve in `src/shaders/candidateTemporal.ts` — not from -scratch. Extract the detector alone; do **not** bring the surrounding resolver (+76% -measured). +The roadmap item landed as `src/shaders/shadingChange.ts`: one fused half-resolution +dispatch (an 8×8 workgroup covers a 16×16 render tile, so the 4×4/8×8 reductions are +workgroup-local) that maintains a 1-frame luma history, compares jitter-aligned +block-mean luma per scale with base + contrast-scaled noise floors, neutralizes +disoccluded texels, and feeds accumulate's `FLAG_SHADING_CHANGE` aging path (binding +12). Locks kept their self-referential break — untouched, per the documented trap. -- Known perf issues to fix on extraction (from the code audit): hoist the per-tap - 1×1 `frameInfo` reloads out of the reduction loops; drop the write-only luma-pyramid - mips unless a consumer lands. -- Wire its output into the existing `FLAG_SHADING_CHANGE` aging path in - `accumulate.ts`, replacing the 3×3-neighborhood mean; keep the lock-suppression - behavior exactly (locks must NOT break on shading change — regression documented in - CLAUDE.md). -- Validate: `DebugView.ShadingChange` black on a still scene, lights up under an - animated light; high-frequency content under heavy motion should show fewer false - positives than production (this is the whole point — capture both). -- Risk: highest of the four — dedicated session with GPU tuning time, per the roadmap. +Five GPU tuning iterations were needed (all evidence in +`bench/results/raw/E00/pre-spd-reference` + `post-spd-v*`): +1. The candidate's mean-of-per-texel-signed-ratios floored at ~0.10 still-scene + response — the relative-difference metric weights the darker side of alias + residue, a coherent bias signed averaging cannot cancel. +2. Jitter-delta-aligned bilinear reprojection helped but did not fix it. +3. Ratio-of-block-means (average first) collapsed the floor. +4. Disocclusion neutralization + coefficient-of-variation-scaled floors fixed + moving-silhouette false fires. +5. Dropping the 2×2 scale (thin features flicker at that scale regardless) hit the + full acceptance matrix: still scene at the old detector's baseline (Q1 ≈ 2 vs + 1.1), **fewer** false positives under camera motion on high-frequency content + (Q4 worst 3.6 vs old 4.9), light steps fire as clean single-frame spikes (Q9: + 137/255 vs old 84 with a 20-frame decay tail), host pre-exposure steps quiet + (Q11), finals within 1.6/255 RMSE of the old detector. + +Cost: **0.044 ms** at ratio 2 (the candidate's two-pass form measured 0.231 ms; +5× cheaper), zero when `settings.detectShadingChanges` is off. Slow ramps +deliberately do not fire (the 1-frame comparison sees only the per-frame delta; +blend + variance clip track ramps — verified no lag/ghosting on Q9 ramp finals). ## Explicitly not planned (measured against) diff --git a/bench/src/benchmark/variants.ts b/bench/src/benchmark/variants.ts index 9877caa..6ee2e95 100644 --- a/bench/src/benchmark/variants.ts +++ b/bench/src/benchmark/variants.ts @@ -10,6 +10,7 @@ const RESOURCE_GRAPH = [ 'scene-color-depth-velocity', 'exposure', 'reconstruct', + 'shading-change-pyramid', 'accumulate-history-locks', 'rcas-or-blit', 'debug-or-output', @@ -158,7 +159,7 @@ function metadata(id: BenchmarkVariantId): BenchmarkVariantMetadata { : {}, timingPassLabels: sourceBundle ? sourceTimingLabels - : ['exposure', 'reconstruct', 'accumulate', 'rcas'], + : ['exposure', 'reconstruct', 'shadingChange', 'accumulate', 'rcas'], }, }; } diff --git a/src/Upscaler.ts b/src/Upscaler.ts index a528f86..e50af82 100644 --- a/src/Upscaler.ts +++ b/src/Upscaler.ts @@ -57,6 +57,7 @@ import { GENERATE_REACTIVE_SHADER } from './shaders/generateReactive'; import { LUMINANCE_PYRAMID_SHADER } from './shaders/luminancePyramid'; import { RCAS_SHADER } from './shaders/rcas'; import { RECONSTRUCT_SHADER } from './shaders/reconstruct'; +import { SHADING_CHANGE_SHADER } from './shaders/shadingChange'; import { DebugView, QualityMode, @@ -137,6 +138,7 @@ export class Upscaler { private _accumulatePass!: ComputePass; private _exposurePass!: ComputePass; private _generateReactivePass!: ComputePass; + private _shadingChangePass!: ComputePass; private _debugPass!: ComputePass; private _depthClipPass: ComputePass | null = null; private _prepareReactivityPass: ComputePass | null = null; @@ -191,6 +193,10 @@ export class Upscaler { private _shadingChange: GPUTexture | null = null; private _lumaHistory: [GPUTexture, GPUTexture] | null = null; private _lumaInstability: GPUTexture | null = null; + // Production shading-change detector state (distinct from the candidate + // resolver's _lumaHistory/_shadingChange above). + private _shadingLumaHistory: [GPUTexture, GPUTexture] | null = null; + private _shadingSignal: GPUTexture | null = null; constructor(options: { renderer: WebGPURenderer }); constructor(options: UpscalerInternalOptions) { @@ -304,6 +310,7 @@ export class Upscaler { } : {}, ); + this._shadingChangePass = new ComputePass(device, 'shading-change', SHADING_CHANGE_SHADER); this._generateReactivePass = new ComputePass( device, 'gen-reactive', @@ -729,6 +736,39 @@ export class Upscaler { ); reconstructPass.end(); + //* Shading Change — signed luma-difference pyramid (skipped entirely + //* when the detector is off; accumulate then reads a zero dummy). + let shadingSignalView = this._reactiveDummy!.createView(); + if (this.settings.detectShadingChanges) { + const shadingLumaIn = this._shadingLumaHistory![this._historyIndex]; + const shadingLumaOut = this._shadingLumaHistory![1 - this._historyIndex]; + const shadingBindGroup = this._shadingChangePass.createBindGroup([ + { buffer: this._constants.buffer }, + colorGPU.createView(), + shadingLumaIn.createView(), + this._dilatedMotion!.createView(), + exposureCur.createView(), + exposurePrev.createView(), + shadingLumaOut.createView(), + this._shadingSignal!.createView(), + this._masks!.createView(), + ]); + const shadingPass = encoder.beginComputePass({ + label: 'upscale-shading-change', + timestampWrites: this._timer.passDescriptor('shadingChange'), + }); + // Half-resolution grid: one thread per 2×2 render block (the pass + // covers a 16×16 render tile per workgroup — see shadingChange.ts). + this._shadingChangePass.dispatch( + shadingPass, + shadingBindGroup, + Math.max(1, Math.ceil(this._renderWidth / 2)), + Math.max(1, Math.ceil(this._renderHeight / 2)), + ); + shadingPass.end(); + shadingSignalView = this._shadingSignal!.createView(); + } + //* Accumulate — jittered upsample + history reprojection/rectification const accumulateBindGroup = this._accumulatePass.createBindGroup([ { buffer: this._constants.buffer }, @@ -743,6 +783,7 @@ export class Upscaler { exposureCur.createView(), reactiveView, exposurePrev.createView(), + shadingSignalView, ]); const accumulatePass = encoder.beginComputePass({ label: 'upscale-accumulate', @@ -1313,6 +1354,17 @@ export class Upscaler { this._candidateBundle ? 'rgba16float' : 'rgba8unorm', ); this._reactiveGenerated = this._createTexture('reactive-gen', rw, rh, 'rgba8unorm'); + //* Shading-change detector state (shadingChange.ts) + this._shadingLumaHistory = [ + this._createTexture('shading-luma-0', rw, rh, 'r32float'), + this._createTexture('shading-luma-1', rw, rh, 'r32float'), + ]; + this._shadingSignal = this._createTexture( + 'shading-signal', + Math.max(1, Math.ceil(rw / 2)), + Math.max(1, Math.ceil(rh / 2)), + 'r32float', + ); if (this._candidateBundle) { this._reconstructedDepth = this._device.createBuffer({ @@ -1435,5 +1487,11 @@ export class Upscaler { } this._lumaInstability?.destroy(); this._lumaInstability = null; + if (this._shadingLumaHistory) { + this._shadingLumaHistory.forEach((texture) => texture.destroy()); + this._shadingLumaHistory = null; + } + this._shadingSignal?.destroy(); + this._shadingSignal = null; } } diff --git a/src/shaders/README.md b/src/shaders/README.md index 025353c..fbdab20 100644 --- a/src/shaders/README.md +++ b/src/shaders/README.md @@ -494,32 +494,29 @@ and replace them only inside the coordinated parity resolver tested on #### Shading change -- **Current status:** Custom replacement. -- **Local implementation:** Compares reprojected history luma with a current 3×3 - neighborhood mean and variance. -- **FSR 3.1.5 behavior:** Builds a dedicated signed-difference SPD from corrected - current/previous luma and evaluates multiple mips. -- **Why it differs / evidence confidence — Likely:** The local heuristic avoids a pyramid, - its resources, and extra history inputs. That architectural saving is visible, but its - GPU benefit and quality tradeoff have not been measured. - -**Keep the local path** - -- **Pros:** Avoids the dedicated SPD resources/work and is already observable through a - debug view. -- **Cons:** Lacks scale separation and can confuse high-frequency motion or aliasing with a - real shading change. - -**Adopt FSR parity** - -- **Pros:** Provides source multi-scale detection and corrected history/current luma - comparison. -- **Cons:** Adds a dedicated pyramid and dependencies. It may cost more GPU time; the - amount is unknown. - -**Next action:** **Benchmark.** Add the signed-difference SPD as a structural variant and -compare false positives, response to controlled lighting changes, per-pass distributions, -and upscaler compute-pass sum. Do not reuse nonexistent local exposure mips. +- **Current status:** Multi-scale detector adopted (2026-07-21, NEXT-STEPS item 4 / + the Phase-5 roadmap item), in a fused form. +- **Local implementation:** `shadingChange.ts` — one half-resolution dispatch whose + 8×8 workgroup covers a 16×16 render tile, so 4×4 and 8×8 block reductions are + workgroup-local. Current and jitter-aligned reprojected previous luma are averaged + per block and the *means* are compared per scale, gated by a base + contrast-scaled + (coefficient-of-variation) noise floor; disoccluded texels are neutralized. The + strongest gated scale is the response consumed by accumulate's `SHADING_AGE` path. + The pass also maintains the 1-frame luma history it compares against. +- **FSR 3.1.5 behavior:** Builds a signed-difference SPD from corrected + current/previous luma over multiple mips, in two passes with dedicated mip + resources. +- **Evidence — Measured (five GPU tuning iterations, 2026-07-21):** The source-style + mean-of-per-texel-signed-ratios was implemented first and floored at ~0.10 mean + response on a still jittered scene (the ratio metric weights the darker side of any + alias residue — a coherent bias signed averaging cannot cancel); block-mean ratios + with adaptive floors and coarse-scales-only resolve grade cleanly: still scene at + the old detector's baseline, camera-orbit false positives *below* the old 3×3 + heuristic (Q4 worst 3.6 vs 4.9), light steps fire as clean single-frame spikes + (137/255 vs the old 84 with a 20-frame decay tail), and a host pre-exposure step + stays quiet. Finals differ ≤ 1.6/255 RMSE. Cost: 0.044 ms at ratio 2 (the + candidate's two-pass form measured 0.231 ms), zero when + `settings.detectShadingChanges` is off — the pass is simply not dispatched. #### Luma instability @@ -774,17 +771,18 @@ declare AMD-style host `preExposure`; using it as one will divide that factor ba during output and will not provide `DeltaPreExposure()` history correction. `DebugView.Exposure` visualizes clamped exposed luma, not the selected exposure scalar. -### Custom shading-change heuristic +### Shading-change detector -The local heuristic compares reprojected history luma with the current 3×3 neighborhood -mean, normalized by neighborhood variance. When it responds, non-locked history is aged by -`SHADING_AGE`; locked pixels suppress this aging. This can identify some lighting/material -changes, but it can also respond to high-frequency motion or aliasing. It is not FSR -3.1.5's signed-difference SPD and multiple-mip analysis. +`shadingChange.ts` compares block-mean luma (4×4 and 8×8 render blocks) against the +previous frame's jitter-aligned reprojected luma; where a scale's mean moved beyond its +noise floor, non-locked history is aged by `SHADING_AGE` in `accumulate.ts`; locked +pixels suppress this aging (locks must never break on shading change — see CLAUDE.md). +Ghosting after a lighting change → lower the `SHADING_FLOOR_*` constants (top of +`shadingChange.ts`); flat steadily-lit surfaces shimmering → raise them, or raise +`SHADING_FLOOR_CV` if the noise sits on textured regions. Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`) and inspect -`DebugView.ShadingChange`. `SHADING_LO`, `SHADING_HI`, and `SHADING_AGE` are heuristic -tuning controls, not source constants or guarantees. +`DebugView.ShadingChange`. When off, the pass is not dispatched at all. ### Reactive masks diff --git a/src/shaders/accumulate.ts b/src/shaders/accumulate.ts index 2478610..65133cd 100644 --- a/src/shaders/accumulate.ts +++ b/src/shaders/accumulate.ts @@ -35,6 +35,8 @@ import { assembleShader } from './wgsl'; * - 10: reactive mask, render size (r = reactivity; 1×1 dummy when absent) * - 11: previous frame's exposure, 1×1 (b = last frame's host pre-exposure, * for FSR3-style DeltaPreExposure history correction) + * - 12: shading-change response, ceil(render/2) (r32float from + * shadingChange.ts; 1×1 zero dummy when the detector is off) */ export const ACCUMULATE_SHADER = assembleShader( WGSL_CONSTANTS, @@ -52,6 +54,7 @@ export const ACCUMULATE_SHADER = assembleShader( @group(0) @binding(9) var exposureTex : texture_2d; @group(0) @binding(10) var reactiveMask : texture_2d; @group(0) @binding(11) var exposurePrevTex : texture_2d; +@group(0) @binding(12) var shadingChangeTex : texture_2d; const PI : f32 = 3.14159265358979; // How hard a fully-reactive pixel snaps to the current frame in the blend. @@ -74,15 +77,11 @@ const LOCK_PEAK_HI : f32 = 2.0; const LOCK_CLAMP_RELAX : f32 = 12.0; // how much a full lock widens the variance AABB const LOCK_HISTORY_BOOST : f32 = 0.7; // how much a full lock favors history in the blend -//* Shading-change detection (FSR2/3's "luminance instability"). -// Distinguishes a genuine shading change (a light turning on, an animated -// material) from mere motion by comparing the reprojected history's luma to the -// current neighborhood's averaged luma — a coherent disagreement the local -// variance can't explain. Measured on averaged luma so sub-pixel aliasing on -// thin edges doesn't read as a change. Where it fires, history is aged so the +//* Shading-change aging (FSR2/3's "luminance instability"). +// The detection itself lives in the dedicated signed-difference pyramid pass +// (shadingChange.ts) — a coarse-mip signal that fires on coherent regional luma +// changes and cancels alias flicker. Where it fires, history is aged so the // surface re-converges to its new shading instead of ghosting the old. -const SHADING_LO : f32 = 1.5; // luma disagreement (in neighborhood std-devs) to start reacting -const SHADING_HI : f32 = 4.0; // ...and to treat as a full shading change const SHADING_AGE : f32 = 0.75; // max fraction of accumulation dropped on a full change // Lanczos2 kernel (the same window FSR2 uses for its upsample taps). @@ -227,17 +226,16 @@ fn main(@builtin(global_invocation_id) gid : vec3u) { let mean = m1 / 9.0; let variance = max(m2 / 9.0 - mean * mean, vec3f(0.0)); let curY = rgbToYCoCg(current).x; - let histY = rgbToYCoCg(history.rgb).x; let contrast = boxMax.x - boxMin.x; - //* Shading-Change Detection - // Coherent luma disagreement between reprojected history and the current - // neighborhood, normalized by how much the neighborhood itself varies — - // large only when the surface's shading changed rather than just moved. + //* Shading Change (from the signed-difference pyramid pass) + // Half-render-resolution response in [0,1]; a zero dummy is bound when the + // detector is disabled. var shadingChange = 0.0; if (hasFlag(FLAG_SHADING_CHANGE)) { - let lumaSpread = sqrt(variance.x) + 0.5 * contrast + 1.0e-3; - shadingChange = smoothstep(SHADING_LO, SHADING_HI, abs(mean.x - histY) / lumaSpread); + let scDim = vec2i(textureDimensions(shadingChangeTex)); + let scCoord = clamp(renderCoord / 2, vec2i(0), scDim - 1); + shadingChange = clamp(textureLoad(shadingChangeTex, scCoord, 0).r, 0.0, 1.0); } //* Luminance-Stability Lock diff --git a/src/shaders/shaders.test.ts b/src/shaders/shaders.test.ts index 0da7a74..04601ec 100644 --- a/src/shaders/shaders.test.ts +++ b/src/shaders/shaders.test.ts @@ -48,6 +48,7 @@ import { RCAS_TONEMAP_SPACE_SHADER, } from './rcas'; import { RECONSTRUCT_SHADER } from './reconstruct'; +import { SHADING_CHANGE_SHADER } from './shadingChange'; import { assembleShader } from './wgsl'; const ALL_SHADERS: Record = { @@ -55,6 +56,7 @@ const ALL_SHADERS: Record = { easu: EASU_SHADER, rcas: RCAS_SHADER, reconstruct: RECONSTRUCT_SHADER, + shadingChange: SHADING_CHANGE_SHADER, accumulate: ACCUMULATE_SHADER, luminancePyramid: LUMINANCE_PYRAMID_SHADER, generateReactive: GENERATE_REACTIVE_SHADER, @@ -66,7 +68,8 @@ const BASELINE_BINDING_COUNTS: Record = { easu: 3, rcas: 4, reconstruct: 7, - accumulate: 12, + shadingChange: 9, + accumulate: 13, luminancePyramid: 7, generateReactive: 4, debug: 10, @@ -79,8 +82,10 @@ const BASELINE_FINGERPRINTS: Record = { rcas: '51bd6d54', // Updated 2026-07-21: AMD viewport/depth-scaled disocclusion (NEXT-STEPS item 3). reconstruct: '19104db7', + // Added 2026-07-21: Phase-5 signed-difference pyramid detector (NEXT-STEPS item 4). + shadingChange: '41ed97fa', // Updated 2026-07-21: DeltaPreExposure history correction (NEXT-STEPS item 2). - accumulate: 'df540efa', + accumulate: 'a2a4ec79', luminancePyramid: 'e4b7a644', generateReactive: '6ed4b549', debug: 'e30ebd6c', diff --git a/src/shaders/shadingChange.ts b/src/shaders/shadingChange.ts new file mode 100644 index 0000000..632c060 --- /dev/null +++ b/src/shaders/shadingChange.ts @@ -0,0 +1,185 @@ +import { WGSL_COLOR, WGSL_CONSTANTS } from './common'; +import { assembleShader } from './wgsl'; + +/** + * Shading-change detection — FSR3's signed luma-difference pyramid (the + * "coarse mip" detector from the source resolver, Phase 5), fused into one + * render-resolution dispatch. + * + * The pass averages current luma and last frame's reprojected luma over + * 2×2 / 4×4 / 8×8 render blocks and compares the *means* per scale, each gated + * by a scale-matched noise floor. Averaging before the ratio is the point: + * block-mean luma is stable under sub-pixel jitter and aliasing, so genuine + * shading changes (a light turning on, an animated material) move the means + * while alias flicker does not. (Two rejected designs, measured on GPU: the + * source-style mean-of-per-texel-signed-ratios floors at ~0.10 on a still + * jittered scene because the ratio metric weights the darker side of any + * residual more — a coherent bias averaging cannot cancel; point-sampled + * reprojection without the jitter-delta alignment is worse still.) The + * strongest gated scale is the response — replacing the 3×3-neighborhood + * variance heuristic, which false-positived on high-frequency content under + * heavy motion. + * + * Structure (why this diverges from the source's SPD + resolve pass pair): one + * 8×8 workgroup with a 2×2 render block per thread covers 16×16 render pixels — + * exactly one 8×8-block (mip2) tile — so every reduction scale is + * workgroup-local. Each signed difference is evaluated once into workgroup + * memory and the mip means + resolve happen in-register; the source-style + * candidate re-evaluated differences per mip with per-tap 1×1 frame-info + * reloads and measured 0.225 ms against this pass's target of a fraction of + * that. The pass also writes this frame's luma history (host-domain) for the + * next frame's comparison. + * + * Bindings: + * - 1: scene color, render size (linear HDR, host domain) + * - 2: luma history in, render size (r32float; last frame's host-domain luma) + * - 3: dilated motion, render size (UV delta in .xy) + * - 4: exposure, 1×1 (r = conditioning, b = host pre-exposure) + * - 5: previous frame's exposure, 1×1 + * - 6: luma history out (r32float storage, render size) + * - 7: shading-change response out (r32float storage, ceil(render/2)) + */ +export const SHADING_CHANGE_SHADER = assembleShader( + WGSL_CONSTANTS, + WGSL_COLOR, + /* wgsl */ ` +@group(0) @binding(1) var inputColor : texture_2d; +@group(0) @binding(2) var lumaHistoryIn : texture_2d; +@group(0) @binding(3) var dilatedMotion : texture_2d; +@group(0) @binding(4) var exposureTex : texture_2d; +@group(0) @binding(5) var exposurePrevTex : texture_2d; +@group(0) @binding(6) var lumaHistoryOut : texture_storage_2d; +@group(0) @binding(7) var shadingChangeOut : texture_storage_2d; +@group(0) @binding(8) var masks : texture_2d; + +// Per-thread block state: x = current-luma sum, y = reprojected previous-luma +// sum, z = current-luma² sum (for the block's coefficient of variation). +// Luma is averaged BEFORE taking any ratio: per-texel relative differences are +// asymmetric (the darker side of any jitter/alias residue always yields the +// larger ratio), so their signed mean carries a coherent bias on +// high-frequency content — measured as a 0.07–0.10 still-scene floor. +var tileSums : array; + +// Per-scale base noise floors for the relative difference of block means. +// Only the 4×4 and 8×8 scales contribute to the response — this is the +// "coarse mip" of the roadmap item: 2×2 means of a thin feature still swing +// under sub-pixel jitter no matter the floor (measured as per-block speckle on +// grid intersections and silhouettes), while a genuinely changing small +// feature still moves its containing 4×4 mean. +const SHADING_FLOOR_MID : f32 = 0.08; // 4×4 render-texel means +const SHADING_FLOOR_COARSE : f32 = 0.04; // 8×8 +// Adaptive part: jitter/alias flicker of a block mean scales with the block's +// own luma contrast, so the floor grows with its coefficient of variation. +// Flat regions (cv ≈ 0) stay maximally sensitive; a checkerboard block +// (cv ≈ 1) is inherently ambiguous and defers to the variance-clip path. +const SHADING_FLOOR_CV : f32 = 0.35; + +// Sums this texel's current luma, the previous frame's luma reprojected to the +// same world position (jitter-delta compensated, bilinear — r32float is not +// filterable), and current luma². Reset/offscreen texels contribute neutrally +// (prev = cur), and disoccluded texels are neutralized toward it — their +// previous luma belongs to another surface, and disocclusion already discards +// that history downstream. +fn lumaPair(coord : vec2i, currentLuma : f32, hostRatio : f32, conditioning : f32) -> vec3f { + let neutral = vec3f(currentLuma, currentLuma, currentLuma * currentLuma); + if (hasFlag(FLAG_RESET)) { return neutral; } + let uv = (vec2f(coord) + 0.5) * C.renderSizeInv; + let motion = textureLoad(dilatedMotion, coord, 0).xy; + // Texel i samples the scene at i + jitter, so the previous frame's + // equivalent position shifts by the jitter delta. + let previousUv = uv - motion + (C.jitter - C.jitterPrev) * C.renderSizeInv; + if (any(previousUv < vec2f(0.0)) || any(previousUv > vec2f(1.0))) { + return neutral; + } + let pos = previousUv * C.renderSize - 0.5; + let base = floor(pos); + let fraction = pos - base; + let maxCoord = vec2i(C.renderSize) - 1; + let p00 = clamp(vec2i(base), vec2i(0), maxCoord); + let p11 = clamp(vec2i(base) + 1, vec2i(0), maxCoord); + let l00 = textureLoad(lumaHistoryIn, p00, 0).r; + let l10 = textureLoad(lumaHistoryIn, vec2i(p11.x, p00.y), 0).r; + let l01 = textureLoad(lumaHistoryIn, vec2i(p00.x, p11.y), 0).r; + let l11 = textureLoad(lumaHistoryIn, p11, 0).r; + let previousHostLuma = mix(mix(l00, l10, fraction.x), mix(l01, l11, fraction.x), fraction.y); + var previousLuma = previousHostLuma * hostRatio * conditioning; + let disocclusion = textureLoad(masks, coord, 0).r; + previousLuma = mix(previousLuma, currentLuma, clamp(disocclusion, 0.0, 1.0)); + return vec3f(currentLuma, previousLuma, currentLuma * currentLuma); +} + +// Relative difference of two block means, gated by the scale's base floor +// plus the block's own contrast-scaled flicker allowance. +fn scaleResponse(sums : vec3f, count : f32, floorBase : f32) -> f32 { + let maximum = max(sums.x, sums.y); + if (maximum <= 1.0e-5) { return 0.0; } + let mean = sums.x / count; + let variance = max(sums.z / count - mean * mean, 0.0); + let cv = sqrt(variance) / max(mean, 1.0e-4); + let floorValue = floorBase + SHADING_FLOOR_CV * cv; + let relative = 1.0 - min(sums.x, sums.y) / maximum; + return smoothstep(floorValue, floorValue * 3.0, relative); +} + +@compute @workgroup_size(8, 8) +fn main( + @builtin(global_invocation_id) gid : vec3u, + @builtin(local_invocation_id) lid : vec3u, + @builtin(local_invocation_index) lidx : u32, +) { + // Hoisted once per invocation — the source-style candidate reloaded these + // 1×1 texels inside every reduction tap, which dominated its cost. + let frameInfo = textureLoad(exposureTex, vec2i(0), 0); + let hostPrev = textureLoad(exposurePrevTex, vec2i(0), 0).b; + let hostRatio = select(1.0, frameInfo.b / hostPrev, hostPrev > 1.0e-4 && frameInfo.b > 1.0e-4); + let conditioning = max(frameInfo.r, 1.0e-4); + + //* Fine Sums + Luma History (2×2 render block per thread) + let origin = vec2i(gid.xy) * 2; + let maxCoord = vec2i(C.renderSize) - 1; + var sums0 = vec3f(0.0); + for (var y = 0; y < 2; y++) { + for (var x = 0; x < 2; x++) { + let coord = clamp(origin + vec2i(x, y), vec2i(0), maxCoord); + // History stays in the caller's host domain; host + conditioning + // are applied to both sides of the comparison only. + let hostLuma = luma(textureLoad(inputColor, coord, 0).rgb); + textureStore(lumaHistoryOut, origin + vec2i(x, y), vec4f(hostLuma, 0.0, 0.0, 0.0)); + sums0 += lumaPair(coord, hostLuma * conditioning, hostRatio, conditioning); + } + } + tileSums[lidx] = sums0; + workgroupBarrier(); + + // Grid guards sit AFTER the barrier: every invocation must reach it + // (uniform control flow), and out-of-range texture writes above are no-ops. + if (any(vec2f(gid.xy) >= C.renderSize)) { return; } + let outputSize = vec2u(textureDimensions(shadingChangeOut)); + if (any(gid.xy >= outputSize)) { return; } + + //* Coarse Sums (workgroup-local: 4×4 render per mid, 8×8 per coarse) + let base1 = (lid.xy / 2u) * 2u; + var sums1 = vec3f(0.0); + for (var y = 0u; y < 2u; y++) { + for (var x = 0u; x < 2u; x++) { + sums1 += tileSums[(base1.y + y) * 8u + base1.x + x]; + } + } + + let base2 = (lid.xy / 4u) * 4u; + var sums2 = vec3f(0.0); + for (var y = 0u; y < 4u; y++) { + for (var x = 0u; x < 4u; x++) { + sums2 += tileSums[(base2.y + y) * 8u + base2.x + x]; + } + } + + //* Resolve — strongest floor-gated mean-ratio across the coarse scales. + let response = max( + scaleResponse(sums1, 16.0, SHADING_FLOOR_MID), + scaleResponse(sums2, 64.0, SHADING_FLOOR_COARSE), + ); + textureStore(shadingChangeOut, vec2i(gid.xy), vec4f(response, 0.0, 0.0, 1.0)); +} +`, +); From 005de6d24bbe7bc7c430e67bf3ed7e5eb901e223 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 17:48:04 +0900 Subject: [PATCH 06/22] =?UTF-8?q?docs:=20post-parity=20cleanup=20=E2=80=94?= =?UTF-8?q?=20PARITY.md=20technical=20writeup,=20current-state=20status=20?= =?UTF-8?q?everywhere?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - PARITY.md: expanded into the shareable design-notes document with a new 'What we changed — enhancements beyond a port' section (fused reconstruction, conditioned-space RCAS, the multi-scale shading-change detector incl. the block-mean-ratio bias finding, host-invariant exposure), each with upstream behavior, our form, and measurements. Stale 'open items' section closed. - README: 'Phases' table replaced by a current-state Status section; PARITY.md now linked from How-it-works and Status. - bench/docs: PARITY-PROGRESS.md (concluded program's process scaffolding) deleted, its load-bearing facts folded into PARITY-DECISIONS.md; NEXT-STEPS.md retitled as a closed adoption record (header wrongly said item 4 still open); PARITY-CANDIDATES.md gained the program-conclusion note; DENOISING-DIRECTION.md moved into bench/docs/. - CLAUDE.md: phase-ledger roadmap restructured into 'Feature status' + 'Deferred / out of scope', keeping every mechanism note and trap. - Code comments: stale phase references removed; accumulate.ts and luminancePyramid.ts headers corrected (they described shipped features as future work). No WGSL strings touched — shader fingerprints unchanged. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 78 ++++++------ PARITY.md | 159 +++++++++++++++++++----- README.md | 19 ++- bench/README.md | 2 +- bench/{ => docs}/DENOISING-DIRECTION.md | 0 bench/docs/NEXT-STEPS.md | 15 +-- bench/docs/PARITY-CANDIDATES.md | 6 + bench/docs/PARITY-DECISIONS.md | 15 ++- bench/docs/PARITY-PROGRESS.md | 116 ----------------- src/shaders/README.md | 4 +- src/shaders/accumulate.ts | 5 +- src/shaders/easu.ts | 3 +- src/shaders/luminancePyramid.ts | 7 +- src/shaders/reconstruct.ts | 2 +- src/shaders/shaders.test.ts | 2 +- src/shaders/shadingChange.ts | 4 +- src/types.ts | 4 +- 17 files changed, 218 insertions(+), 223 deletions(-) rename bench/{ => docs}/DENOISING-DIRECTION.md (100%) delete mode 100644 bench/docs/PARITY-PROGRESS.md diff --git a/CLAUDE.md b/CLAUDE.md index a8f317e..d00d3f9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,7 +30,8 @@ live" section remains the guide for visual regressions. **Parity program concluded (2026-07-21):** the three source-style FSR 3.1.5 candidate graphs were GPU-verified and A/B-benchmarked against production — **+36% / +6.5% / +76% GPU compute with no visual win**; none adopted. Consumer-facing rationale in -`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-*.md`. **Post-parity +`PARITY.md` (root); evidence + decisions in `bench/docs/PARITY-DECISIONS.md` / +`PARITY-CANDIDATES.md`. **Post-parity items 1–3 landed the same day** (see `bench/docs/NEXT-STEPS.md` for evidence): (1) RCAS now sharpens in conditioned tonemap space, inverting once — **−34% RCAS, −5.7% total** with capture-identical output; the old form is frozen as @@ -39,7 +40,7 @@ pre-exposure (`preExposureTexture`) is honored end-to-end — DeltaPreExposure h correction + host-invariant auto-exposure metering, validated on the new **Q11** scenario, byte-identical when absent. (3) The reconstruct pass uses AMD's viewport/depth-scaled disocclusion (per-tap confidence voting) inside the fused -single pass. **Item 4 (the Phase-5 SPD detector) landed the same session**: the +single pass. **Item 4 (the multi-scale shading-change detector) landed the same session**: the 3×3-neighborhood shading heuristic is replaced by `shadingChange.ts` — a fused multi-scale block-mean detector (0.044 ms, 5× cheaper than the candidate's two-pass form, measurably fewer false positives under motion; five GPU tuning @@ -178,41 +179,42 @@ Full guide in `src/shaders/README.md`. --- -## Roadmap (what "finished" looks like) - -Phases 0–2 are **written and GPU-verified** (bench renders all paths correctly; six -standalone examples ship in `examples/`). The examples double as the validation harness -for the remaining phases: prove luma-stability locks on `04-aliasing-torture`, the -reactive mask on `05-transparency` (its explicit acceptance test), the RCAS-denoise -variant on `06-screenspace-gi`, and expose new toggles in `02-fsr1-vs-fsr3`. Remaining: - -- **Phase 3 — fidelity to real FSR3:** - - ~~**Luminance-stability locks**~~ — **done & GPU-verified.** Persistent display-res lock buffer in `accumulate.ts` (r = lifetime, g = locked luma), reprojected through motion; detects thin luminance outliers, grows a lock while present, breaks on disocclusion/shading change, then widens the rectification AABB + boosts history for locked pixels. Toggle `settings.lockThinFeatures` (`FLAG_LOCKS`); inspect via `DebugView.Locks`. Tuning constants (top of `accumulate.ts`) are defaults, not final — tighten if thin features ghost, loosen if they still dim. - - ~~**Luminance pyramid + auto-exposure**~~ — **done.** `luminancePyramid.ts` reduces the scene to a single log-average luminance (one-workgroup 32×32-tap reduction, not yet an SPD mip chain) → a pre-exposure eased over time (eye-adaptation). `accumulate.ts` pre-exposes the input before the invertible tonemap; `rcas.ts`/`blit.ts` divide it back out before display — so HDR scenes of very different brightness accumulate in the same well-conditioned range **without changing final brightness**. This also fixed a latent bug: the temporal path previously baked in `settings.exposure` and never divided it out. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); inspect via `DebugView.Exposure`. Constants (top of `luminancePyramid.ts`: key/min/max/adapt-speed) are defaults. The SPD mip chain is deferred until the shading-change detector needs the intermediate mips. - - **External exposure input** — **done & GPU-verified.** An app that meters its own exposure feeds it via `dispatch({ exposureTexture })` (value in the red texel, any float format); it overrides both auto and fixed exposure and is still divided back out before display (conditions accumulation, not brightness). Wired as binding 5 of the pyramid pass behind `FLAG_EXTERNAL_EXPOSURE` (the 1024 bit) and funnelled through the same single `select`, so downstream passes are untouched and `avgLum` stays our own measurement for the shading detector. Bound to the reactive dummy as a placeholder when absent. Also on the composable node as `options.exposureTexture` (mirrors FSR3's `exposure` dispatch resource). - - ~~**Shading-change detector**~~ — **done & GPU-verified; upgraded to the multi-scale form 2026-07-21.** `shadingChange.ts` (one fused half-res dispatch) compares jitter-aligned block-mean luma at 4×4/8×8 render scales against a 1-frame luma history, with contrast-adaptive noise floors and disocclusion neutralization; the response ages **non-locked** history via accumulate's `SHADING_AGE` path so changed surfaces re-converge. A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with its block mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`) — the pass isn't dispatched when off; inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). - - ~~**Reactive-mask input**~~ — **done & GPU-verified.** Optional `dispatch({ reactive })` render-res mask (red = reactivity); flagged pixels suppress locks, keep near-zero accumulation, and snap to the current frame (`REACTIVE_STRENGTH` in `accumulate.ts`). No mask → a 1×1 zero texture is bound and `FLAG_REACTIVE` stays off (zero cost). `UpscalePresenter.setReactiveMask()` threads it through; `examples/05-transparency` authors one by rendering the transparents' coverage and is the acceptance demo. Inspect via `DebugView.Reactivity`. **Authoring helpers** (auto-generating the mask) — **done**, see the next bullet. **Composable-node parity** — **done.** `upscale()` / `UpscalerNode` now take `options.reactive` and `options.reactiveOpaqueColor` texture nodes (registered as graph deps so the opaque buffer renders in-pipeline, jittered, aligned with color), closing an asymmetry with the raw `Upscaler`/`UpscalePass`. No dedicated node demo yet — it reuses the GPU-verified reactive dispatch (example 05, imperative) + the proven graph-dep mechanism (examples 07/09), so worst case on a plumbing miss is a silent no-op, not a crash. -- **Phase 4 — API & ecosystem:** - - ~~**RCAS denoise variant**~~ — **done & GPU-verified.** `rcas.ts` gains FSR1's `FSR_RCAS_DENOISE` path (attenuate the sharpening lobe on lone luma outliers so grain from noisy inputs isn't amplified), gated by `settings.rcasDenoise` (`FLAG_RCAS_DENOISE`, off by default). Pairs with an upstream spatial denoiser; `examples/06-screenspace-gi` toggles it on for the reduced-res SSR/GI. One of Dennis's original asks. - - ~~**Reactive-mask authoring helper**~~ — **done & GPU-verified.** `dispatch({ reactiveOpaqueColor })` auto-generates the mask from the opaque-vs-final color diff (`generateReactive.ts`, FSR2's `GenerateReactiveMask`); no explicit `reactive` mask needed. `UpscalePresenter.setReactiveOpaqueColor()` threads it; `examples/05-transparency` offers manual-coverage vs auto-diff. Caveat: jitter the opaque pass like the final or high-contrast edges leave faint reactivity (sub-pixel misalignment). - - **Transparency & Composition (T&C) mask** — **deliberately deferred** (assessed 2026-07-10). FSR2/3 takes a second render-res mask alongside reactive, but it is *not* a clean parallel: in FSR2 the T&C mask has a distinct-but-overlapping effect (a softer history-distrust than reactive, it widens the rectification AABB and interacts with locks) that is genuinely tuned. Adding it means new tuning constants and touching the accumulate blend/lock path — exactly the "if it messes us up, delay" risk — and our reactive mask (+ auto-generate) already covers the common three.js transparency case (example 05's whole point). Shipping it as a second channel that behaves identically to reactive would be misleading; shipping the *real* distinct behavior needs core tuning we shouldn't ride onto other work. Revisit if a user actually authors T&C masks and the reactive path proves insufficient. - - ~~**Drop-in driver**~~ — **done.** Two public surfaces, both GPU-verified: - - **`UpscalePass`** (`src/UpscalePass.ts`) — the imperative drop-in (graduated from `UpscalePresenter`, which is now a re-export shim). Bakes in the MRT/jitter/velocity/present recipe. Covers renderer-agnostic / non-graph use. - - **TSL nodes** (`src/UpscalerNode.ts`, native TSL, no pmndrs dep) — "the future" surface. **One node, one code path** — modelled on three's own `FSR1Node` / `TAAUNode`: - - **`upscale(color, depth, velocity, camera, options)`** (`UpscalerNode`) — the composable node, consumes reduced-res texture nodes and outputs the upscaled result. GPU-verified. **The key mechanism** (this is what a first attempt got wrong and rendered black): the inputs must be *graph dependencies* so three renders them in-graph, in dependency order, before this node's `updateBefore`. three discovers child nodes by walking the node's **own non-`_`-prefixed** properties (`Node._getChildren`) — our fields are `_`-prefixed, so `setup()` registers the inputs explicitly into `builder.getNodeProperties(this)` (exactly as `FSR1Node` does with `properties.textureNode`). With that, jitter (applied via the `onBeforeRenderPipeline` hook, like `TAAUNode`) lands because the inputs render *inside* the post render. The factory `convertToTexture`s the color (a no-op for texture/pass nodes, which is why reduced-res pass outputs keep their size — the caller controls input resolution). `UpscalerConfig` gained `renderWidth`/`renderHeight` so the node matches an externally-sized input exactly. - - **`upscaleScene(scene, camera, options)`** — a thin convenience, **not** a separate class: it builds `pass(scene, camera)` with a `{ output, velocity }` MRT at `1/ratio` and hands the texture nodes to `upscale(...)` — the same shape as three's `taau(pass.getTextureNode('output'), …)`. So the scene renders in-graph as an FSR3 input, jitter and all. `post.outputNode = upscaleScene(scene, camera)`. Examples `07-tsl-node`, `08-tsl-compose` (`.mul(vignette)`); the full SSGI/SSR stack is `09-kitchen-sink`. - - **`upscaleSpatial(color, options)`** — color-only **spatial** (FSR1/EASU) node for inputs with no motion data: no depth/velocity/camera, no history, no reconstruction. A thin facade over `path: 'spatial'` with a stand-in camera (EASU reprojects nothing; `_writeConstants` still stages `near`/`far`, which the shader ignores, so any finite values keep NaN out of the UBO). It exists so the "I only have a color texture" case has a clean door instead of `upscale(color, null, null, camera, { path: 'spatial' })` — and so defaulting the *temporal* node to jitter-on isn't a trap. - - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity now `console.warn`s once (was a silent no-op). `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. - - Color path (both): the node emits linear/HDR color. When it is the final graph node, three's RenderPipeline applies the renderer's configured tone mapping and output color space; otherwise it can feed later linear post-processing. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). - - **Resolved** (was an open question): the composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. - - ~~MSAA-input support~~ — **dropped by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). -- **Phase 5 — performance & the one remaining quality item:** ~~merge dilate+depth-clip into one pass~~ **done** (`reconstruct.ts` — one render-res dispatch, one fewer intermediate round-trip; GPU-verified disocclusion unchanged). Remaining, all deferred as of 2026-07-10 (the rename + node-parity + exposure work above was the safe, high-value batch; these were held back so a core-path change wouldn't ride onto the rename merge): - - ~~**True SPD luminance mip chain + shading-change coarse mip**~~ — **done 2026-07-21** (`shadingChange.ts`, NEXT-STEPS item 4). The coarse-mip comparison exists as a fused workgroup-local reduction rather than a literal mip chain (one dispatch, no mip textures, 0.044 ms); it measured *fewer* false positives than the 3×3 heuristic on high-frequency content under camera motion (Q4) with clean single-frame response to light steps (Q9). The predicted tuning risk was real — five GPU iterations, all documented in `bench/docs/NEXT-STEPS.md`. - - **Perf-only (no quality gain, correctness risk):** `textureGather` tap packing (EASU/RCAS currently use per-tap `textureLoad` — but these are the AMD-faithful ports; changing their sampling risks subtle artifacts); f16 arithmetic (`shader-f16`, needs feature detection + fallback, precision risk); bind-group caching (rebuilt per dispatch — "fine but wasteful," but caching adds stale-view-on-resize risk to the core); half-res luma analysis. None gain image quality; each adds risk to a core path the project deliberately protects — do them only when perf is the actual bottleneck. - -- **Future project (deferred) — fused GI/denoise temporal path.** Denoising very noisy screen-space inputs (SSGI especially) can't be solved by stacking a *separate* temporal denoiser before FSR3: any second temporal resolver reprojects by velocity, which is jitter-free, so it can't see FSR3's sub-pixel jitter — it rejects the misaligned history (noise survives) and cancels the jitter variance FSR3 needs (aliasing returns). Verified 2026-07-10 with three r185's `recurrentDenoise`/`temporalReproject` in `examples/10-ssgi-denoise` (kept as **experimental documentation**, not a library feature). Spatial-only denoise + FSR3-owns-temporal avoids the conflict but inherits the third-party à-trous kernel's halos/step-lines/update-cadence skipping. The real fix is to **fuse GI history into FSR3's own accumulation** — reprojected with *our* motion vectors, sampled at *our* jitter, with GI-appropriate variance handling (not the AA-tuned clip). This is genuine R&D that touches the core accumulate pass, so it's a deliberate future effort, not a quick add. Priority remains the FSR3 upscaler itself; don't spend core complexity bending upstream nodes to it. - -Frame generation (the other half of "FSR3") is **out of scope** — it needs swapchain frame pacing browsers don't expose. +## Feature status (all shipped & GPU-verified) + +The pipeline is feature-complete. This section records each feature's mechanism and its +traps — the "why" a future change must not break. The examples double as the validation +harness: locks on `04-aliasing-torture`, the reactive mask on `05-transparency` (its +explicit acceptance test), RCAS denoise on `06-screenspace-gi`. + +**Temporal fidelity:** +- **Luminance-stability locks.** Persistent display-res lock buffer in `accumulate.ts` (r = lifetime, g = locked luma), reprojected through motion; detects thin luminance outliers, grows a lock while present, breaks on disocclusion/shading change, then widens the rectification AABB + boosts history for locked pixels. Toggle `settings.lockThinFeatures` (`FLAG_LOCKS`); inspect via `DebugView.Locks`. Tuning constants (top of `accumulate.ts`) are defaults, not final — tighten if thin features ghost, loosen if they still dim. +- **Auto-exposure.** `luminancePyramid.ts` reduces the scene to a single log-average luminance (one-workgroup 32×32-tap reduction; no mip chain — nothing consumes intermediate mips) → a pre-exposure eased over time (eye-adaptation). `accumulate.ts` pre-exposes the input before the invertible tonemap; `rcas.ts`/`blit.ts` divide it back out before display — so HDR scenes of very different brightness accumulate in the same well-conditioned range **without changing final brightness**. Toggle `settings.autoExposure` (`FLAG_AUTO_EXPOSURE`); inspect via `DebugView.Exposure`. Constants (top of `luminancePyramid.ts`: key/min/max/adapt-speed) are defaults. + - **External exposure input.** An app that meters its own exposure feeds it via `dispatch({ exposureTexture })` (value in the red texel, any float format); it overrides both auto and fixed exposure and is still divided back out before display (conditions accumulation, not brightness). Wired as binding 5 of the pyramid pass behind `FLAG_EXTERNAL_EXPOSURE` (the 1024 bit) and funnelled through the same single `select`, so downstream passes are untouched and `avgLum` stays our own measurement for the shading detector. Bound to the reactive dummy as a placeholder when absent. Also on the composable node as `options.exposureTexture` (mirrors FSR3's `exposure` dispatch resource). + - **Host pre-exposure (`preExposureTexture`).** DeltaPreExposure history correction + host-invariant auto-exposure metering (auto-exposure must not chase a step the app already metered — skipping this reads as a ~2s full-screen false shading change). Validated on the Q11 bench scenario; byte-identical output when the input is absent. +- **Shading-change detector** (multi-scale form, 2026-07-21). `shadingChange.ts` (one fused half-res dispatch) compares jitter-aligned block-mean luma at 4×4/8×8 render scales against a 1-frame luma history, with contrast-adaptive noise floors and disocclusion neutralization; the response ages **non-locked** history via accumulate's `SHADING_AGE` path so changed surfaces re-converge. Costs 0.044 ms at ratio 2; zero when off (pass not dispatched). A lock fully suppresses the aging — and note the detector must NOT drive lock-breaking: a thin bright feature's history always disagrees with its block mean, so feeding `shadingChange` into the lock-break would break every lock (regression caught in GPU verification 2026-07-08; locks keep their own self-referential break term). Toggle `settings.detectShadingChanges` (`FLAG_SHADING_CHANGE`); inspect via `DebugView.ShadingChange` (packed into the locks buffer's `.b`). +- **Reactive-mask input.** Optional `dispatch({ reactive })` render-res mask (red = reactivity); flagged pixels suppress locks, keep near-zero accumulation, and snap to the current frame (`REACTIVE_STRENGTH` in `accumulate.ts`). No mask → a 1×1 zero texture is bound and `FLAG_REACTIVE` stays off (zero cost). `UpscalePresenter.setReactiveMask()` threads it through; `examples/05-transparency` authors one by rendering the transparents' coverage and is the acceptance demo. Inspect via `DebugView.Reactivity`. **Node parity:** `upscale()` / `UpscalerNode` take `options.reactive` and `options.reactiveOpaqueColor` texture nodes (registered as graph deps so the opaque buffer renders in-pipeline, jittered, aligned with color). No dedicated node demo — it reuses the GPU-verified reactive dispatch (example 05, imperative) + the proven graph-dep mechanism (examples 07/09), so worst case on a plumbing miss is a silent no-op, not a crash. +- **Reactive-mask authoring helper.** `dispatch({ reactiveOpaqueColor })` auto-generates the mask from the opaque-vs-final color diff (`generateReactive.ts`, FSR2's `GenerateReactiveMask`); no explicit `reactive` mask needed. `UpscalePresenter.setReactiveOpaqueColor()` threads it; `examples/05-transparency` offers manual-coverage vs auto-diff. Caveat: jitter the opaque pass like the final or high-contrast edges leave faint reactivity (sub-pixel misalignment). +- **RCAS denoise variant.** `rcas.ts` has FSR1's `FSR_RCAS_DENOISE` path (attenuate the sharpening lobe on lone luma outliers so grain from noisy inputs isn't amplified), gated by `settings.rcasDenoise` (`FLAG_RCAS_DENOISE`, off by default). Pairs with an upstream spatial denoiser; `examples/06-screenspace-gi` toggles it on for the reduced-res SSR/GI. + +**Public surfaces** (both GPU-verified): +- **`UpscalePass`** (`src/UpscalePass.ts`) — the imperative drop-in (graduated from `UpscalePresenter`, which is now a re-export shim). Bakes in the MRT/jitter/velocity/present recipe. Covers renderer-agnostic / non-graph use. +- **TSL nodes** (`src/UpscalerNode.ts`, native TSL, no pmndrs dep) — "the future" surface. **One node, one code path** — modelled on three's own `FSR1Node` / `TAAUNode`: + - **`upscale(color, depth, velocity, camera, options)`** (`UpscalerNode`) — the composable node, consumes reduced-res texture nodes and outputs the upscaled result. **The key mechanism** (this is what a first attempt got wrong and rendered black): the inputs must be *graph dependencies* so three renders them in-graph, in dependency order, before this node's `updateBefore`. three discovers child nodes by walking the node's **own non-`_`-prefixed** properties (`Node._getChildren`) — our fields are `_`-prefixed, so `setup()` registers the inputs explicitly into `builder.getNodeProperties(this)` (exactly as `FSR1Node` does with `properties.textureNode`). With that, jitter (applied via the `onBeforeRenderPipeline` hook, like `TAAUNode`) lands because the inputs render *inside* the post render. The factory `convertToTexture`s the color (a no-op for texture/pass nodes, which is why reduced-res pass outputs keep their size — the caller controls input resolution). `UpscalerConfig` has `renderWidth`/`renderHeight` so the node matches an externally-sized input exactly. + - **`upscaleScene(scene, camera, options)`** — a thin convenience, **not** a separate class: it builds `pass(scene, camera)` with a `{ output, velocity }` MRT at `1/ratio` and hands the texture nodes to `upscale(...)` — the same shape as three's `taau(pass.getTextureNode('output'), …)`. So the scene renders in-graph as an FSR3 input, jitter and all. `post.outputNode = upscaleScene(scene, camera)`. Examples `07-tsl-node`, `08-tsl-compose` (`.mul(vignette)`); the full SSGI/SSR stack is `09-kitchen-sink`. + - **`upscaleSpatial(color, options)`** — color-only **spatial** (FSR1/EASU) node for inputs with no motion data: no depth/velocity/camera, no history, no reconstruction. A thin facade over `path: 'spatial'` with a stand-in camera (EASU reprojects nothing; `_writeConstants` still stages `near`/`far`, which the shader ignores, so any finite values keep NaN out of the UBO). It exists so the "I only have a color texture" case has a clean door instead of `upscale(color, null, null, camera, { path: 'spatial' })` — and so defaulting the *temporal* node to jitter-on isn't a trap. + - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity `console.warn`s once. `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. + - Color path (both): the node emits linear/HDR color. When it is the final graph node, three's RenderPipeline applies the renderer's configured tone mapping and output color space; otherwise it can feed later linear post-processing. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). + - The composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. +- **MSAA input — rejected by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). + +**Performance structure:** dilate + depth-clip are fused into the single `reconstruct.ts` dispatch (GPU-verified disocclusion unchanged); the shading detector is one fused workgroup-local reduction instead of the source's SPD mip chain + resolve pair. The measured story of these divergences from FSR 3.1.5 — and the four upstream behaviors adopted in re-derived form — is `PARITY.md` (root) with evidence in `bench/docs/NEXT-STEPS.md`. + +## Deferred / out of scope + +- **Transparency & Composition (T&C) mask** — deliberately deferred (assessed 2026-07-10). FSR2/3 takes a second render-res mask alongside reactive, but it is *not* a clean parallel: in FSR2 the T&C mask has a distinct-but-overlapping effect (a softer history-distrust than reactive, it widens the rectification AABB and interacts with locks) that is genuinely tuned. Adding it means new tuning constants and touching the accumulate blend/lock path, and our reactive mask (+ auto-generate) already covers the common three.js transparency case (example 05's whole point). Shipping it as a second channel that behaves identically to reactive would be misleading; shipping the *real* distinct behavior needs core tuning we shouldn't ride onto other work. Revisit if a user actually authors T&C masks and the reactive path proves insufficient. +- **Perf-only micro-optimizations (no quality gain, correctness risk):** `textureGather` tap packing (EASU/RCAS currently use per-tap `textureLoad` — but these are the AMD-faithful ports; changing their sampling risks subtle artifacts); f16 arithmetic (`shader-f16`, needs feature detection + fallback, precision risk); bind-group caching (rebuilt per dispatch — "fine but wasteful," but caching adds stale-view-on-resize risk to the core); half-res luma analysis. None gain image quality; each adds risk to a core path the project deliberately protects — do them only when perf is the actual bottleneck. +- **Future project — fused GI/denoise temporal path.** Denoising very noisy screen-space inputs (SSGI especially) can't be solved by stacking a *separate* temporal denoiser before FSR3: any second temporal resolver reprojects by velocity, which is jitter-free, so it can't see FSR3's sub-pixel jitter — it rejects the misaligned history (noise survives) and cancels the jitter variance FSR3 needs (aliasing returns). Verified 2026-07-10 with three r185's `recurrentDenoise`/`temporalReproject` in `examples/10-ssgi-denoise` (kept as **experimental documentation**, not a library feature). Spatial-only denoise + FSR3-owns-temporal avoids the conflict but inherits the third-party à-trous kernel's halos/step-lines/update-cadence skipping. The real fix is to **fuse GI history into FSR3's own accumulation** — reprojected with *our* motion vectors, sampled at *our* jitter, with GI-appropriate variance handling (not the AA-tuned clip). This is genuine R&D that touches the core accumulate pass, so it's a deliberate future effort, not a quick add. Priority remains the FSR3 upscaler itself; don't spend core complexity bending upstream nodes to it. +- **Frame generation** (the other half of "FSR3") — needs swapchain frame pacing browsers don't expose. --- @@ -226,4 +228,4 @@ Follow the existing style: ## Provenance / license -MIT (`LICENSE`). The EASU/RCAS WGSL derives from AMD's MIT-licensed FidelityFX (`ffx_fsr1.h`); AMD's copyright notice is in `LICENSE`. Preserve it. When porting more FidelityFX stages (Phase 3), keep the "faithful port vs. simplification" table in `src/shaders/README.md` honest. +MIT (`LICENSE`). The EASU/RCAS WGSL derives from AMD's MIT-licensed FidelityFX (`ffx_fsr1.h`); AMD's copyright notice is in `LICENSE`. Preserve it. If more FidelityFX stages are ever ported, keep the "faithful port vs. simplification" table in `src/shaders/README.md` honest. diff --git a/PARITY.md b/PARITY.md index 424aed8..461ff96 100644 --- a/PARITY.md +++ b/PARITY.md @@ -1,14 +1,15 @@ -# Why @pmndrs/upscaler is not a line-for-line FSR 3.1.5 port +# @pmndrs/upscaler vs FSR 3.1.5 — design notes `@pmndrs/upscaler` derives from AMD FidelityFX FSR — FSR1's EASU/RCAS directly, and an FSR2/3-style temporal resolver written for WebGPU compute. It is **not** byte-for-byte FSR 3.1.5, and that is a deliberate, *measured* position, not an unfinished port. This -document explains where we match upstream, where we diverge, and the evidence behind -each choice. +document explains where we match upstream, where we diverge, what we changed outright, +and the evidence behind each choice. Reference: FidelityFX SDK commit `60f4ea8` (FSR Upscaler 3.1.5). The full per-pass audit lives in [`src/shaders/README.md`](src/shaders/README.md); raw benchmark evidence in -`bench/results/`. +`bench/results/`; the adoption record in +[`bench/docs/NEXT-STEPS.md`](bench/docs/NEXT-STEPS.md). ## The short version @@ -35,26 +36,121 @@ Every delta was repeatable across four interleaved A/B blocks with noise floors Reproduce with: `node scripts/run-benchmark.mjs --smoke --ratios 2 --blocks 4 --warmup 240 --samples 300 --variant rcas-fsr315-limiter --comparison `. +The program then fed back: four upstream behaviors *were* worth having, and each was +adopted — but re-derived into cheaper forms rather than transplanted (next section). + +## What we changed — enhancements beyond a port + +These are the places where this implementation deliberately does something *different* +from FSR 3.1.5 and has measurements showing the difference is an improvement on this +platform. Each was validated on GPU with the deterministic capture harness (byte-level +RMSE gates) in addition to timing. + +### 1. Fused single-pass depth reconstruction & disocclusion + +**Upstream:** three stages — "reconstruct previous depth" (an atomic floating-point +scatter into a previous-depth buffer), "dilate depth & motion", and "depth clip" +(disocclusion), with intermediate textures between them. Core WebGPU has no +floating-point storage atomics, so the scatter must be emulated through `u32` +storage-buffer atomics. + +**Ours:** one render-resolution dispatch (`reconstruct.ts`). The key observation: depth +clip only ever reads the current pixel's own dilated depth and motion — both already +in-register after the dilate step — plus last frame's dilated depth, which a gather +(read the previous frame's output) provides without any scatter. We kept AMD's +*math* — the viewport/depth-scaled disocclusion tolerance +(`1.37e-5 · halfViewportWidth · maxDepth`) with per-bilinear-tap confidence voting from +`ffx_fsr2_depth_clip.h` — inside our fused structure. + +**Measured:** the source-style scatter + separate passes cost +30% (prepareInputs) and ++22% (depthClip) in the structural candidate with no visual difference; the fused pass +runs in 0.035 ms at ratio 2 with disocclusion output validated identical in behavior +(thin stable silhouette outlines, quiet still scenes, age resets confined to +disocclusion trails). + +### 2. RCAS in conditioned tonemap space + +**Upstream:** RCAS sharpens exposed linear texels; each of the 5 taps is loaded in the +working color domain. + +**Ours:** the temporal path's history is already stored in invertible-tonemap space +(`c/(1+max(c))`, FSR2's own conditioning). Production RCAS sharpens those bounded +conditioned texels directly and inverts the conditioning + exposure **once on the +result** instead of per tap. Tonemap inversion and exposure division are exactly the +per-tap ALU that made the flag-on form expensive; hoisting them out of the tap loop is +free because RCAS's ratio-based limiter is scale-invariant enough that the sharpening +decision is unchanged in practice. + +**Measured:** −34% on the RCAS pass (0.103 → 0.068 ms), −5.7% total pipeline compute; +captures across Q0/Q1/Q3 plus an HDR-bulb stress scenario show full-frame RMSE +≤ 1.8/255 and HDR ROI maxima ≤ 9/255 — visually identical, no overshoot. The per-tap +form is frozen in the bench registry (`rcas-fsr315-limiter`) so the comparison stays +reproducible. + +### 3. Fused multi-scale shading-change detector + +The most substantial re-derivation, and the one with a genuinely new result. + +**Upstream:** a two-pass design — an SPD (single-pass downsampler) builds a +signed luma-difference mip chain, then the resolve reads multiple mips to detect +shading changes. The per-texel metric is a relative difference +(`1 − min/max`) averaged over each mip footprint. + +**Ours:** one fused half-resolution dispatch (`shadingChange.ts`). An 8×8 workgroup with +a 2×2 render block per thread covers exactly one 16×16 render tile, so every reduction +scale (4×4, 8×8) is workgroup-shared-memory-local — no mip textures, no second pass, no +per-tap re-loads of the 1×1 frame-info texels. + +Two findings from GPU tuning (five documented iterations): + +- **Per-texel relative differences carry a coherent bias under jitter.** On + high-frequency content, sub-pixel jitter leaves alias residue between the current + frame and the reprojected previous frame. The relative-difference metric is + asymmetric — the darker side of any residue always yields the larger ratio — so the + *mean of per-texel ratios* floors at ~0.10 on a completely still scene and no amount + of averaging cancels it. Averaging the *luma first* and taking the ratio of block + means is unbiased: block-mean luma is stable under jitter, so genuine shading changes + move the means while alias flicker does not. +- **The finest (2×2) scale is unrescuable.** 2×2 means of a thin feature still swing + under sub-pixel jitter regardless of the noise floor; a genuinely changing small + feature still moves its containing 4×4 mean. The response therefore uses only the + coarse scales, each gated by a base + contrast-adaptive floor (scaled by the block's + coefficient of variation), with disoccluded texels neutralized. + +**Measured:** 0.044 ms at ratio 2 vs 0.231 ms for the source-style two-pass candidate +(5× cheaper; zero when disabled — the pass isn't dispatched). Quality beats both +alternatives: still-scene response at the old inline heuristic's baseline, *fewer* +false positives than that heuristic under camera motion on high-frequency content +(worst-case 3.6 vs 4.9 on the torture scene), and light steps register as clean +single-frame spikes (137/255) where the old detector produced a weaker response (84) +with a ~20-frame decay tail. + +### 4. Single-workgroup exposure reduction with host-invariant metering + +**Upstream:** auto-exposure reads the coarsest mip of the SPD luminance pyramid. + +**Ours:** no consumer needs the intermediate mips (the shading-change detector above +does its own fused reduction), so exposure is a single 8×8-workgroup log-average +reduction (`luminancePyramid.ts`) — one tiny dispatch instead of a device-wide +pyramid. Two upstream behaviors are preserved exactly: `DeltaPreExposure` history +correction (reprojected history is ratio-corrected across a host pre-exposure change), +and host-invariant metering — auto-exposure divides the host's pre-exposure out of the +scene luma before adapting, so it never chases a step the application already metered. +Skipping that second part reads as a full-screen false shading change for ~2 s after a +host exposure step; we found this on GPU and it is now covered by a dedicated +step+ramp scenario (Q11), with byte-identical output when no pre-exposure input is +supplied. + ## Where we match upstream (adopted parity) - **RCAS numeric math.** The production sharpener uses FSR 3.1.5's lower limiter and the corrected denoise luma/range math, adopted after A/B measurement showed parity was free (E01). Denoise is opt-in pending evidence on representative noisy content. - (One deliberate divergence in the *load domain*: the temporal path sharpens the - accumulated history's conditioned tonemap-space texels and inverts the conditioning - once on the result, instead of upstream's exposed-linear per-tap domain — measured - 34% cheaper on the RCAS pass with capture-verified visually identical output, - including an HDR stress scenario.) + (The load domain diverges — see enhancement 2 above.) - **Host pre-exposure (`DeltaPreExposure`).** The `preExposureTexture` dispatch input is - honored end-to-end: reprojected history is ratio-corrected across a host pre-exposure - change, auto-exposure meters host-invariantly, and the host factor is preserved in - the output — upstream's contract. Validated with a dedicated step+ramp scenario - (Q11): no history invalidation, no brightness pumping, and byte-identical output when - the input is absent. -- **Viewport/depth-scaled disocclusion.** The disocclusion threshold uses AMD's - formulation (`ffx_fsr2_depth_clip.h`'s per-tap confidence with the - `1.37e-5 · halfViewportWidth · maxDepth` tolerance) instead of the earlier fixed - relative-threshold guess — kept inside our faster fused reconstruction pass. + honored end-to-end with upstream's contract (see enhancement 4). +- **Viewport/depth-scaled disocclusion.** AMD's threshold formulation, kept inside our + fused reconstruction pass (see enhancement 1). - **Color and exposure domains.** Like upstream, the upscaler applies no tone mapping or output encoding — input and output are the caller's linear/HDR domain, and internal conditioning exposure is divided back out before output. An earlier internal ACES/sRGB @@ -78,13 +174,12 @@ swapchain pacing control (which rules out FSR3 frame generation entirely). The c graphs prove these can be *emulated* — storage-buffer atomics, direct mip re-reads — but the emulations are part of why the source graphs measure slower here. -**2. Measured cost without measured benefit.** The fused single-pass depth -reconstruction/disocclusion, the 3×3-neighborhood shading detector, and the compact -accumulate pass are simplifications that survived because the source alternatives cost -+36–76% GPU time and the deterministic quality scenarios (static convergence, camera -motion, object-motion disocclusion) could not distinguish them visually. A divergence -is kept only while that remains true — the candidates stay in-repo precisely so this -can be re-tested as scenes, devices, or the library change. +**2. Measured cost without measured benefit.** The compact accumulate pass (bilinear-free +Lanczos2 upsample + Catmull-Rom history, no deringed bicubic) survived because the +source alternative cost +47% on accumulate and the deterministic quality scenarios +(static convergence, camera motion, object-motion disocclusion) could not distinguish +them visually. A divergence is kept only while that remains true — the candidates stay +in-repo precisely so this can be re-tested as scenes, devices, or the library change. **3. Scope decisions.** Frame generation is out of scope (browser swapchain limits). MSAA input is rejected by design — FSR's temporal path *is* the anti-aliaser. The @@ -102,12 +197,10 @@ scenes, noisy GI inputs, extreme motion — that the decisive runs did not exerc benchmark harness (`npm run bench`, `scripts/run-benchmark.mjs`) exists so any of these claims can be re-tested; a repeatable ≥5% result is treated as actionable, <3% as noise. -## Open items we do intend to converge +## Status -All four post-parity adoption items landed on 2026-07-21: the RCAS cost -investigation (resolved as the conditioned-space load domain, −34% on the pass), -host pre-exposure correction, the AMD disocclusion constant, and the multi-scale -shading-change detector (the source's coarse-mip design, re-derived as a single -fused pass — 0.044 ms vs the source-style candidate's 0.231 ms, with measurably -fewer false positives than the old 3×3 heuristic on high-frequency content under -motion). Details and evidence: `bench/docs/NEXT-STEPS.md`. +The parity program is concluded. Every adoption-worthy behavior it identified landed on +2026-07-21 — the four items above. Nothing from the program remains open; what remains +deferred (perf-only micro-optimizations, the distinct T&C channel, a fused GI/denoise +temporal path) is listed with rationale in the project README and +`bench/docs/NEXT-STEPS.md`. diff --git a/README.md b/README.md index 40cc7a0..bd6c740 100644 --- a/README.md +++ b/README.md @@ -135,24 +135,21 @@ Each frame the projection is offset by a sub-pixel **jitter** (Halton(2,3) seque The **spatial path** (`path: 'spatial'`) is a faithful FSR1 port: EASU's edge-direction-rotated, anisotropically-stretched 12-tap Lanczos kernel, then RCAS. No history, no motion vectors — also the fallback story for content that can't produce velocity. -Full per-pass details and deviations from the FidelityFX reference: [`src/shaders/README.md`](./src/shaders/README.md). +Full per-pass details and deviations from the FidelityFX reference: [`src/shaders/README.md`](./src/shaders/README.md). For the measured story of how this implementation relates to real FSR 3.1.5 — what matches, what was re-derived into cheaper forms, and the benchmark evidence — see [`PARITY.md`](./PARITY.md). ### Integration approach Three doesn't expose its WebGPU internals publicly, so the upscaler grabs `renderer.backend.device` and the `GPUTexture` handles behind render-target attachments (`internal/threeWebGPU.ts` documents exactly which internals we touch and throws loudly if a three upgrade changes them). Compute passes are encoded on our own `GPUCommandEncoder` and submitted between three's scene render and the presentation draw — queue order guarantees correctness with zero synchronization code. The final image lands in a three `StorageTexture` so presenting it is ordinary three code. -## Phases +## Status -| Phase | Scope | Status | -| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| **0** | Package scaffold, raw-WebGPU pass infra on three's device, bench app, unit tests | ✅ | -| **1** | Spatial baseline: faithful FSR1 EASU + RCAS ports | ✅ | -| **2** | Temporal pipeline: Halton jitter, unjittered velocity, dilation, depth-clip disocclusion, Lanczos2 accumulate w/ YCoCg variance clipping, per-pass GPU timings | ✅ | -| **3** | Fidelity: luminance-stability **locks**, luminance-average auto-exposure (+ external exposure input), shading-change detection, reactive mask input for transparents/particles | ✅ | -| **4** | API & ecosystem: reactive-mask authoring helpers, imperative `UpscalePass` + composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`), RCAS denoise toggle (MSAA input intentionally excluded — FSR's temporal path _is_ the anti-aliaser) | ✅ | -| **5** | Performance: merged dilate+clip pass ✅; remaining — `textureGather` tap packing, f16 (`shader-f16`), bind-group caching, a true SPD luminance mip chain (steadier shading-change), half-res luma analysis | 🚧 | +The pipeline is **feature-complete and GPU-verified**: spatial (FSR1) and temporal paths, luminance-stability locks, auto-exposure (+ external and host pre-exposure inputs), multi-scale shading-change detection, reactive masks (explicit + auto-generated), RCAS with opt-in denoise, imperative `UpscalePass`, and the composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`). A benchmarking program A/B-compared this implementation against source-style FSR 3.1.5 pass graphs on-GPU; the adopted results and remaining divergences — with measurements — are written up in [`PARITY.md`](./PARITY.md). -Frame generation (the other half of "FSR3") needs swapchain-level frame pacing that browsers don't expose; if we ever want it, the realistic shape is interpolating between presented frames ourselves — out of scope here. +Deliberately **not** planned: + +- **Frame generation** (the other half of "FSR3") — needs swapchain-level frame pacing browsers don't expose. +- **MSAA input** — FSR's temporal path _is_ the anti-aliaser; a multisampled input is redundant and can't bind to the compute passes. +- **Perf-only micro-optimizations** (`textureGather` tap packing, f16 arithmetic, bind-group caching) — each adds correctness risk to a core path with no image-quality gain; deferred until performance is an actual bottleneck on real content. ## Package layout diff --git a/bench/README.md b/bench/README.md index 2e43908..d351f33 100644 --- a/bench/README.md +++ b/bench/README.md @@ -19,7 +19,7 @@ Suggested tour: 1. Start on **FSR3 temporal / Performance (2×)** with auto-orbit on — the scene renders 1/4 the pixels of native. Pause the orbit and watch thin lines converge. 2. Flip to **Bilinear** at the same quality to see what the temporal pass is reconstructing. -3. Try **Ultra Performance (3×)** — 1/9 the pixels; edges hold up, fine texture detail softens (this is where FSR3's locks, Phase 3, would help). +3. Try **Ultra Performance (3×)** — 1/9 the pixels; watch the luminance-stability locks hold thin features that would otherwise dim (**Debug ▸ Locks** shows where they form). 4. Set quality to **Native AA (1.0×)** on the temporal path — that's pure TAA mode, the fair comparison against `Native`'s shimmer. 5. Open **Debug ▸ Motion vectors / Disocclusion** while the spheres orbit to sanity-check the inputs. diff --git a/bench/DENOISING-DIRECTION.md b/bench/docs/DENOISING-DIRECTION.md similarity index 100% rename from bench/DENOISING-DIRECTION.md rename to bench/docs/DENOISING-DIRECTION.md diff --git a/bench/docs/NEXT-STEPS.md b/bench/docs/NEXT-STEPS.md index bee00d3..4e65d4e 100644 --- a/bench/docs/NEXT-STEPS.md +++ b/bench/docs/NEXT-STEPS.md @@ -1,10 +1,10 @@ -# Post-parity work plan (2026-07-21) +# Post-parity adoption record (2026-07-21) — all items landed -Outcome of the parity program: no candidate bundle adopted (see +Outcome of the parity program: no candidate bundle adopted wholesale (see [PARITY-DECISIONS.md](PARITY-DECISIONS.md) and the consumer-facing -[/PARITY.md](../../PARITY.md)). Four items survived as adoption-worthy. -**Items 1–3 landed on 2026-07-21** (same-day session; evidence below). Item 4 -remains open as its own dedicated session. +[/PARITY.md](../../PARITY.md)). Four items survived as adoption-worthy, and +**all four landed on 2026-07-21** — this document is the evidence record for +each. Nothing from the parity program remains open. Every item follows the same gate: `npm test && npm run typecheck && npm run lint`, then an A/B timing + capture run @@ -56,9 +56,10 @@ resolver ran with the flag off (plain loads). resets confined to trails; finals shift RMSE ≤ 1.1/255; reconstruct pass time unchanged (0.035 ms at ratio 2). -## 4. Phase-5 SPD session: coarse-mip shading-change detector — DONE +## 4. Multi-scale shading-change detector — DONE -The roadmap item landed as `src/shaders/shadingChange.ts`: one fused half-resolution +The long-standing roadmap item (the source's SPD coarse-mip detector concept) +landed as `src/shaders/shadingChange.ts`: one fused half-resolution dispatch (an 8×8 workgroup covers a 16×16 render tile, so the 4×4/8×8 reductions are workgroup-local) that maintains a 1-frame luma history, compares jitter-aligned block-mean luma per scale with base + contrast-scaled noise floors, neutralizes diff --git a/bench/docs/PARITY-CANDIDATES.md b/bench/docs/PARITY-CANDIDATES.md index a9e7c65..683a890 100644 --- a/bench/docs/PARITY-CANDIDATES.md +++ b/bench/docs/PARITY-CANDIDATES.md @@ -14,6 +14,12 @@ The production fallback is unchanged and remains the default. No bundle is adopt all three cost more compute than production (+36%, +43%, +76% cumulative) without a demonstrated visual win on the deterministic scenarios. +**Program conclusion (2026-07-21):** four individual behaviors identified through +these bundles were extracted and landed in production in re-derived, cheaper forms — +conditioned-space RCAS, host pre-exposure correction, AMD's disocclusion threshold, +and the multi-scale shading-change detector. Evidence: [NEXT-STEPS.md](NEXT-STEPS.md). +The bundles themselves stay registered as bench variants for future re-testing. + ## What is already production behavior These changes are adopted; they are not part of the unmeasured candidate decision: diff --git a/bench/docs/PARITY-DECISIONS.md b/bench/docs/PARITY-DECISIONS.md index aef2ff2..913f5b6 100644 --- a/bench/docs/PARITY-DECISIONS.md +++ b/bench/docs/PARITY-DECISIONS.md @@ -1,7 +1,16 @@ # FSR 3.1.5 Parity Decisions -This is the concise decision record for parity experiments. Raw evidence remains -under `bench/results/raw/`; `PARITY-PROGRESS.md` retains detailed program state. +This is the concise decision record for the (concluded) parity experiment program. +Raw evidence remains under `bench/results/raw/`; the post-parity adoptions that came +out of these decisions are recorded with evidence in [NEXT-STEPS.md](NEXT-STEPS.md). + +Program facts: pinned FidelityFX SDK source `60f4ea81909200d8542eca14dccb2628b763a9a3` +(FSR Upscaler 3.1.5); local baseline commit `5d6a65e` on `feat-match-fsr3`. The E00 +benchmark harness is adopted as a first-pass engineering tool: repeatable changes +≥5% are actionable, <3% is treated as noise, 3–5% is uncertain, and visual +regressions reject a candidate regardless of speed. (Publication-grade fine-margin +acceptance was deferred — long ABBA sequences showed monotonic timing drift on the +test machine that prevents claims near 1.5–2.5% noise limits.) | Experiment | Candidate | Measured result | Recommendation | User decision | Action | | --- | --- | --- | --- | --- | --- | @@ -11,7 +20,7 @@ under `bench/results/raw/`; `PARITY-PROGRESS.md` retains detailed program state. | E03 | Remove internal ACES/sRGB presentation | All output paths now write caller-domain `rgba16float`; temporal, spatial, bilinear, and depth-debug paths compiled and rendered on WebGPU. | Adopt unconditionally: FSR must not own tone mapping or output encoding. | Must remove — 2026-07-17. | Integrated. Bench/examples apply renderer presentation after the upscaler. | | E05–E07, E15 | `source-filter-bundle-v1` | GPU-validated + measured 2026-07-18 (after 2 compile fixes): **+35–36% total compute** vs production at ratio 2 (accumulate +47%; exposure/RCAS tied; noise floor ≤1.7%). Q0/Q1/Q3 captures artifact-free; steady-state diff vs production grows to RMSE ~10/255. | Do not adopt wholesale — the source reconstruction/history filtering costs ~⅓ more compute with no demonstrated visual win on the bench scenes. Cherry-pick pieces (exposure-domain correction, AMD depth-separation constant) individually. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | | E04, E08–E10 | `source-structural-bundle-v1` | GPU-validated + measured 2026-07-18: **+6.2–6.9% over the filter bundle** (prepareInputs +30%, depthClip +22%, accumulate/RCAS tied; noise floor ≤2%). Output near-identical to filter bundle (its prepared accumulation/newLocks outputs are currently unconsumed). | Cheap increment, but mostly inert until a consumer exists; evaluate only together with the resolver or after wiring its prepared state into an accumulate path. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | -| E11–E14 | `source-spd-resolver-bundle-v1` | GPU-validated + measured 2026-07-18 (after 3 compile/validation fixes + a `/0.5`→`/20.0` velocity-normalization correction): **+75–78% total compute** vs production (accumulate +38%; RCAS **−47%**, see note). Q0/Q1/Q3 artifact-free, no disocclusion trails; steady-state diff vs production RMSE ~4.6/255 — closer than the filter bundle's ~10/255. | Too expensive to adopt as-is. Two leads worth extracting: (1) why its history makes RCAS 47% cheaper (production history may contain value ranges that are ALU-hostile); (2) its coarse-mip shading detector is the Phase-5 SPD design, now GPU-proven. | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | +| E11–E14 | `source-spd-resolver-bundle-v1` | GPU-validated + measured 2026-07-18 (after 3 compile/validation fixes + a `/0.5`→`/20.0` velocity-normalization correction): **+75–78% total compute** vs production (accumulate +38%; RCAS **−47%**, see note). Q0/Q1/Q3 artifact-free, no disocclusion trails; steady-state diff vs production RMSE ~4.6/255 — closer than the filter bundle's ~10/255. | Too expensive to adopt as-is. Two leads worth extracting: (1) why its history makes RCAS 47% cheaper (production history may contain value ranges that are ALU-hostile); (2) its coarse-mip shading detector is the long-planned SPD design, now GPU-proven (since landed in re-derived fused form — NEXT-STEPS item 4). | Pending. | Candidate GPU-verified and benchmarked; production unchanged. | ## E01 evidence diff --git a/bench/docs/PARITY-PROGRESS.md b/bench/docs/PARITY-PROGRESS.md deleted file mode 100644 index 55063a2..0000000 --- a/bench/docs/PARITY-PROGRESS.md +++ /dev/null @@ -1,116 +0,0 @@ -# FSR 3.1.5 Parity Progress - -## Program Purpose - -This ledger tracks controlled experiments that compare the local WebGPU upscaler with FidelityFX FSR 3.1.5. Source behavior is the design target; a local simplification is retained only when evidence shows that parity is unavailable, materially slower, or worse for this library on the web platform. - -- Pinned FidelityFX SDK source: `60f4ea81909200d8542eca14dccb2628b763a9a3` -- Initial local baseline: `5d6a65e5681e5e95590f3e9a11ce75e43354ca13` (`5d6a65e`) -- Baseline branch: `feat-match-fsr3` -- Baseline unit result: `npm test` passed `54/54` - -## Experiment State Machine - -Only the controller changes experiment state: - -`declared → implementing → static verification → GPU verification → task review → decision → documented` - -An experiment may move backward only to `implementing` for a consolidated fix round. A terminal blocker is recorded without pretending that later gates passed. Dependent experiments begin only from an adopted and documented integration state. - -## Strict Status Contract - -Every implementation or review handoff must use exactly one status: - -- `PASS`: the assigned scope is complete and every required gate available to that task passed. -- `FAIL`: the task ran, but an implementation, verification, or evidence gate failed. -- `BLOCKED`: the assigned scope cannot proceed because of an external, environment, capability, or dependency blocker. -- `SCOPE_BLOCKED`: completion requires a write, redesign, or investigation outside the immutable manifest. -- `USER_DECISION_REQUIRED`: evidence exposes a product, API, quality, performance, or scope trade-off that only the user may decide. - -A report must also include `changed_files`, `commands`, `artifacts`, `gates`, and `concerns`. `PASS` is invalid if required evidence or gate results are absent. Agents must not reinterpret a blocker as permission to broaden scope. - -## Blocker Taxonomy - -- `scope`: a required file or change is outside the exact allowlist; report `SCOPE_BLOCKED`. -- `dependency`: a prerequisite experiment is not adopted and documented; report `BLOCKED`. -- `tooling`: a command, browser, or local tool fails independently of the implementation; retry once, then report `BLOCKED`. -- `capability`: required WebGPU features, especially `timestamp-query`, are unavailable; report `BLOCKED` and preserve environment evidence. -- `validation`: WGSL compilation, WebGPU validation, uncaught runtime errors, device loss, or static verification fails; report `FAIL`. -- `evidence`: required captures, fresh timing samples, adapter metadata, or review records cannot be produced; report `BLOCKED`. -- `product-decision`: valid evidence leaves a public API, default, or quality/performance trade-off unresolved; report `USER_DECISION_REQUIRED`. - -The controller may assign one bounded read-only investigation for a blocker. That investigation cannot modify integration state, expand an allowlist, or redesign the experiment. - -## Fix-Round Limit - -Each experiment allows at most two consolidated fix rounds after its initial implementation. A round addresses one controller-approved batch of critical or important findings. After round two, the controller must redesign the manifest, retain the baseline, record a blocker, or escalate to the user. - -## Documentation Ownership - -- The controller owns this ledger, immutable manifests, state transitions, decisions, and cross-experiment dependencies. -- Concise measured results, recommendations, user votes, and resulting actions are tracked in `bench/docs/PARITY-DECISIONS.md`. -- Implementers own only files explicitly listed in their manifest and may not edit manifests or this ledger. -- Reviewers and research agents are read-only unless a separate exact allowlist says otherwise. -- Implementation agents whose exact allowlists share any path are serialized. The controller must finish or stop the active writer before starting another overlapping writer. -- Parallel work is limited to independent, read-only research, review, or artifact analysis with no shared mutable state. -- The controller updates `src/shaders/README.md` only after an experiment decision and updates public documentation only for shipped behavior. -- Raw captures and timing artifacts follow `bench/results/README.md` once the harness creates it; manifests remain versioned records. -- No agent may commit, branch, create a worktree, or add files to an allowlist without explicit controller authorization. - -## Experiment Index - -| ID | Experiment | State | Fix rounds | Manifest | Decision | -| --- | --- | ---: | ---: | --- | --- | -| E00 | Harness Foundation | `adopted` | 2 of 2 — maximum reached; controller redesign 4 | `bench/results/experiments/e00-harness.json` | Directional baseline established; publication-grade acceptance deferred | -| E01 | RCAS numeric parity | `documented` | 0 of 2 | Directional first pass | Source limiter and denoise math adopted; denoise default remains separate | -| E03 | Linear/HDR output domain | `documented` | 0 of 2 | Directional integration | Internal ACES/sRGB removed by user decision | -| E04–E07, E15 | Source reconstruction/filter bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +35–36% compute vs production; visually clean; not adopted (see PARITY-DECISIONS.md) | -| E08–E10 | Structural inputs/reactivity bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +6.2–6.9% over filter bundle; largely inert outputs; not adopted | -| E11–E14 | SPD temporal resolver/state bundle | `GPU verification + measured` | 1 of 2 | Cumulative authored candidate | +75–78% compute vs production; visually clean; not adopted; RCAS −47% anomaly worth study | - -E00 covers only the deterministic Phase 1 benchmark foundation and baseline-versus-baseline acceptance machinery. It does not authorize parity shader algorithm changes. Fix rounds 1 and 2 were controller-authorized pre-harness contract corrections; no harness implementation had begun when they were issued. Controller redesign 1 froze the readiness reset contract against installed three `0.185.1`. Controller redesign 2 closed the post-implementation acceptance boundary: immutable A/B roles, exact reviewer-record coverage, noise-first retry classification, fixed timing-pass identities, unique timing sequences, and explicit page teardown. Controller redesign 3 corrected two acceptance-analysis defects exposed by authoritative execution: E00 aggregate timing acceptance uses the compute-sum noise floor while noisy passes are ineligible for individual claims, and E00 visual acceptance uses a representative Q0/Q1/Q10 matrix instead of pre-running the full domain-experiment Cartesian product. Every selected tuple still receives all 45 numerical reload comparisons; human review samples 120 blinded pairs and records all declared ROI bounds. Controller redesign 4 followed the fresh task review: run and review evidence is now bound to one manifest plus complete working-tree digest, review persistence includes the evidence and ordered-record identity, review-only validates the authoritative numerical capture before accepting grades, and user-owned CDP targets are closed and awaited. - -The earlier timing and capture artifacts passed their numerical gates but were produced under superseded manifest digests and cannot close E00. They remain diagnostic evidence only. Revision 4 static verification passed with lint, typecheck, 83 tests, build, runner syntax, and diff whitespace. Q2-Q9 reduced WebGPU smokes passed, as did the user-owned CDP target-cleanup smoke. - -Revision 4 authoritative timing is blocked at ratio 1 after its one permitted cold-browser retry: a concentrated GPU slowdown affected roughly 100 frames in one 600-frame block, producing compute-sum p95 noise near 50 percent. The bound authoritative capture completed all 264 tuples and 11,880 pairs but failed 61 pairs across five localized Q0 tuples; differences cluster on the moving torus-knot highlight and one derived accumulation-age region. Two focused five-reload reruns of the failed tuples passed, confirming intermittence but not authorizing replacement of the complete failed set. - -A clean-boot revision 4 timing rerun reproduced monotonic drift across the long ABBA sequence: endpoint A runs rose from roughly 0.78 ms to 0.96–1.08 ms while adjacent B runs stayed closer. This prevents publication-grade claims near the strict 1.5–2.5 percent noise limits, but does not prevent the harness from identifying substantial directional shader changes. Per user direction, E00 is adopted as a first-pass engineering tool: repeatable changes of at least 5 percent are actionable, changes below 3 percent are treated as tied/noise, and 3–5 percent remains uncertain. Visual regressions still reject candidates regardless of speed. Formal cross-platform and fine-margin acceptance is deferred until a candidate survives the shader-parity program. - -E01's first pass compiled the FSR 3.1.5 lower limiter and source denoise luma/range as an isolated RCAS shader. The user adopted both math changes on 2026-07-17. They are now in the production RCAS pipeline; the prior implementation remains benchmark-only. A fresh linear/HDR rerun kept total compute within the practical noise band and retained only sparse lower-limiter differences. Enabling source denoise by default also remained within noise but changed high-contrast detail across the frame: Q0/Q1 RMSE was about 0.55–0.60/255 with maxima of 17–33/255. Denoise therefore stays opt-in until a targeted noisy-input fixture shows that the broad sharpening reduction is beneficial. - -`npm run bench:compare:rcas` reproduces the focused comparison end to end, generates short directional timing summaries plus blinded visual reviews, serves the report locally, and opens it in the default browser. The current reference is `bench/results/raw/E01/rcas-comparison-linear-hdr/index.html`; reopen it without GPU work using `npm run bench:compare:rcas -- --reuse bench/results/raw/E01/rcas-comparison-linear-hdr`. - -E03 removed the internal Narkowicz ACES approximation and sRGB encoding from EASU, RCAS, and blit. Final and debug output now use `rgba16float`, and the public texture remains in the caller's linear/HDR domain. The benchmark and examples choose three's ACES filmic tone mapping plus sRGB only when presenting to screen. Static verification passed lint, typecheck, 86 tests, build, and diff whitespace. Real WebGPU verification rendered temporal, spatial, bilinear, and depth-debug paths without validation failure. - -## Authored source-style candidates - -Three cumulative internal candidates are now authored for later A/B work. This is -implementation state only: no benchmark, browser GPU validation, visual review, timing -claim, or adoption decision has been made. - -See `bench/docs/PARITY-CANDIDATES.md` for the candidate hypotheses, cumulative dependencies, -fallbacks, and the performance-first test matrix required before any adoption decision. - -- `source-filter-bundle-v1` replaces current/history reconstruction, EASU implementation - math, and fused depth reconstruction with source-style radial approximate Lanczos2, - bicubic Lanczos history, approximate EASU helpers, atomic reconstructed-depth scatter, - and viewport/depth-scaled disocclusion. It also corrects history across changing - conditioning and host pre-exposure domains while retaining the local accumulation state. -- `source-structural-bundle-v1` cumulatively adds configurable source reactive generation, - farthest depth/current luma preparation, motion divergence, max-dilated application - reactivity with reset coupling, a distinct softer T&C channel, render-resolution - accumulation state, and atomic new-lock preparation. -- `source-spd-resolver-bundle-v1` cumulatively adds luma and signed-difference mip chains, - multi-mip shading change, persistent four-frame luma instability, and one coordinated - source-style accumulation/rectification/lock/state model. History alpha stores lock - lifetime in this candidate and is not compatible with the production sample-age alpha. - -The production graph remains the fallback and default. RCAS keeps the adopted source -limiter/math, linear/HDR output remains unchanged, and temporal RCAS denoise remains -opt-in. Candidate timing labels and shader/resource identities are registered, but should -not be interpreted as measured evidence until the later benchmark program runs. - -The bounded GPU-free verification pass completed with 156 tests, typecheck, lint, and the -library/declaration build passing. This advances the authored candidates only to static -verification; WebGPU compilation, validation, captures, timings, review, and adoption -remain pending. diff --git a/src/shaders/README.md b/src/shaders/README.md index fbdab20..899c577 100644 --- a/src/shaders/README.md +++ b/src/shaders/README.md @@ -494,8 +494,8 @@ and replace them only inside the coordinated parity resolver tested on #### Shading change -- **Current status:** Multi-scale detector adopted (2026-07-21, NEXT-STEPS item 4 / - the Phase-5 roadmap item), in a fused form. +- **Current status:** Multi-scale detector adopted (2026-07-21, NEXT-STEPS item 4), + in a fused form. - **Local implementation:** `shadingChange.ts` — one half-resolution dispatch whose 8×8 workgroup covers a 16×16 render tile, so 4×4 and 8×8 block reductions are workgroup-local. Current and jitter-aligned reprojected previous luma are averaged diff --git a/src/shaders/accumulate.ts b/src/shaders/accumulate.ts index 65133cd..e0121bd 100644 --- a/src/shaders/accumulate.ts +++ b/src/shaders/accumulate.ts @@ -3,8 +3,9 @@ import { assembleShader } from './wgsl'; /** * Reproject & accumulate — the core temporal upscaling pass (FSR2/3's - * "accumulate" stage, simplified: luminance-stability locks and the - * shading-change detector are Phase 3 work; see shaders README). + * "accumulate" stage, with luminance-stability locks, reactive-mask handling, + * and the shading-change response wired in; per-stage fidelity notes live in + * the shaders README). * * Per display pixel: * 1. Upsample the current jittered frame with a jitter-aware separable diff --git a/src/shaders/easu.ts b/src/shaders/easu.ts index a6a86b5..d6a92c7 100644 --- a/src/shaders/easu.ts +++ b/src/shaders/easu.ts @@ -17,7 +17,8 @@ import { assembleShader } from './wgsl'; * are replaced by exact `1/x` / `inverseSqrt` (negligible cost on modern * GPUs, and WGSL has no direct float bit-cast idiom for them), and taps are * `textureLoad`s instead of packed `textureGather`s — an acceptable trade - * for a test bench (noted in the package README as a Phase 5 optimization). + * (gather packing is a deliberately deferred perf-only optimization; see the + * package README's status section). * * Per the FSR1 spec, EASU expects perceptual input. The upscaler does not * choose or bake a presentation transform; callers that use the spatial path diff --git a/src/shaders/luminancePyramid.ts b/src/shaders/luminancePyramid.ts index 08457f6..922dfa2 100644 --- a/src/shaders/luminancePyramid.ts +++ b/src/shaders/luminancePyramid.ts @@ -6,9 +6,10 @@ import { assembleShader } from './wgsl'; * stage, in the pragmatic form the rest of this port needs). * * FSR2 downsamples the input into a full luminance mip chain with a single - * atomic SPD dispatch and reads the coarsest mip for exposure. We don't yet - * consume the intermediate mips (the shading-change detector — Phase 3 — will), - * so this pass computes only the value that is actually used today: a single + * atomic SPD dispatch and reads the coarsest mip for exposure. Nothing here + * consumes the intermediate mips (the shading-change detector does its own + * fused workgroup-local reduction in shadingChange.ts), so this pass computes + * only the value that is actually used: a single * scene-average luminance, reduced in one workgroup, mapped to an exposure and * eased over time for eye-adaptation. * diff --git a/src/shaders/reconstruct.ts b/src/shaders/reconstruct.ts index b722f45..04523a1 100644 --- a/src/shaders/reconstruct.ts +++ b/src/shaders/reconstruct.ts @@ -3,7 +3,7 @@ import { assembleShader } from './wgsl'; /** * Reconstruct pass — fuses FSR2/3's "reconstruct & dilate" and "depth clip" - * stages into one render-resolution dispatch (a Phase-5 merge: depth clip only + * stages into one render-resolution dispatch (fused deliberately: depth clip only * ever read the current pixel's own dilated depth and motion, both of which * this pass already has in-register, plus the previous frame's dilated depth). * diff --git a/src/shaders/shaders.test.ts b/src/shaders/shaders.test.ts index 04601ec..2c4cb3d 100644 --- a/src/shaders/shaders.test.ts +++ b/src/shaders/shaders.test.ts @@ -82,7 +82,7 @@ const BASELINE_FINGERPRINTS: Record = { rcas: '51bd6d54', // Updated 2026-07-21: AMD viewport/depth-scaled disocclusion (NEXT-STEPS item 3). reconstruct: '19104db7', - // Added 2026-07-21: Phase-5 signed-difference pyramid detector (NEXT-STEPS item 4). + // Added 2026-07-21: multi-scale shading-change detector (NEXT-STEPS item 4). shadingChange: '41ed97fa', // Updated 2026-07-21: DeltaPreExposure history correction (NEXT-STEPS item 2). accumulate: 'a2a4ec79', diff --git a/src/shaders/shadingChange.ts b/src/shaders/shadingChange.ts index 632c060..dc1df28 100644 --- a/src/shaders/shadingChange.ts +++ b/src/shaders/shadingChange.ts @@ -2,8 +2,8 @@ import { WGSL_COLOR, WGSL_CONSTANTS } from './common'; import { assembleShader } from './wgsl'; /** - * Shading-change detection — FSR3's signed luma-difference pyramid (the - * "coarse mip" detector from the source resolver, Phase 5), fused into one + * Shading-change detection — the concept of FSR3's signed luma-difference + * pyramid (the source resolver's "coarse mip" detector), fused into one * render-resolution dispatch. * * The pass averages current luma and last frame's reprojected luma over diff --git a/src/types.ts b/src/types.ts index 36e8c97..586cd73 100644 --- a/src/types.ts +++ b/src/types.ts @@ -30,9 +30,9 @@ export enum QualityMode { * baseline every other mode is compared against (and, at ratio 1, the * "native" passthrough mode). * - `spatial` — single-frame FSR1 (EASU + RCAS). No history, no motion - * vectors required. Phase 1 baseline. + * vectors required. * - `temporal` — FSR2/3-style jittered temporal accumulation. Requires depth - * and motion vectors. Phase 2. + * and motion vectors. */ export type UpscalePath = 'bilinear' | 'spatial' | 'temporal'; From 13ad4a16400801c4ff3542a6ff3ed51ec7f2338c Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 20:04:14 +0900 Subject: [PATCH 07/22] =?UTF-8?q?docs:=20temporal-guides=20spec=20?= =?UTF-8?q?=E2=80=94=20de-black-box=20plan=20for=20external=20consumers=20?= =?UTF-8?q?(SSGI/SVGF)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maps bench/docs/FSR3-BRIEF.md onto the actual pipeline: publish the existing internal textures as a contracted guides bundle, split dispatch into an early geometry stage and a late luma stage, add bidirectional reactive merge, and a standalone signal-agnostic MomentPyramid. Additive only; existing dispatch stays byte- and perf-identical. Co-Authored-By: Claude Fable 5 --- TEMPORAL-GUIDES-SPEC.md | 252 +++++++++++++++++++++++++++++++++++++++ bench/docs/FSR3-BRIEF.md | 176 +++++++++++++++++++++++++++ 2 files changed, 428 insertions(+) create mode 100644 TEMPORAL-GUIDES-SPEC.md create mode 100644 bench/docs/FSR3-BRIEF.md diff --git a/TEMPORAL-GUIDES-SPEC.md b/TEMPORAL-GUIDES-SPEC.md new file mode 100644 index 0000000..fa66e8a --- /dev/null +++ b/TEMPORAL-GUIDES-SPEC.md @@ -0,0 +1,252 @@ +# Temporal Guides — opening the upscaler's internals (spec) + +Status: **draft for review** (2026-07-21, branch `feat-temporal-guides`). +Request: [bench/docs/FSR3-BRIEF.md](bench/docs/FSR3-BRIEF.md) — the consuming +pipeline (SSGI temporal pass, an SVGF-style denoiser, any TAA-class effect) +wants the upscaler's early data products as first-class outputs instead of +re-deriving worse versions privately. This spec maps that request onto what +this codebase actually is, states where we deviate from the brief and why, and +lays out the delivery plan. + +Non-negotiable constraint: **nothing on `feat-match-fsr3` regresses.** Every +change here is additive; the existing `dispatch()` path must stay +byte-identical and perf-identical (bench A/B gated) when no guide is consumed. + +--- + +## 1. The good news: the products already exist + +The brief assumes the port is a monolith that must be split. It isn't — the +pipeline is already discrete compute passes over named internal textures, and +**every texture the brief's "temporal guides" bundle names is already +allocated and written every temporal frame** (`Upscaler._allocateTextures`): + +| Brief asks for | We already have | Where it's produced | +|---|---|---| +| `dilatedVelocity` | `_dilatedMotion` (rgba16float, **UV** delta in .xy) | `reconstruct.ts` (fused dilate + depth-clip) | +| `dilatedDepth` | `_dilatedDepth[cur]` (r32float, linear view depth) | `reconstruct.ts` | +| `prevDepth` | `_dilatedDepth[prev]` — the ping-pong's other half | free (kept for depth-clip) | +| `disocclusion` (graded 0..1) | `_masks.r` (rgba8unorm) — AMD's confidence-voted grade | `reconstruct.ts` | +| `reactive` | `_reactiveGenerated` / caller's mask | `generateReactive.ts` or app | +| `lockStatus` | `_locks[cur]` (rgba16float: r = lifetime, g = locked luma, b = shading) | `accumulate.ts` | +| `historyLength` | history `.a` (accumulation age, display res) | `accumulate.ts` | +| luminance statistics | *(does not exist — see §5)* | — | + +So the deliverable is **contracts and plumbing, not new algorithms**: make +these textures reachable from outside (raw-WebGPU and TSL), make the +geometry-only subset runnable without the upscale, and document each +product's space, resolution, and latency so a consumer can't mis-apply it. + +## 2. Design principle: publish products, not pass boundaries + +The brief prescribes a 4-pass decomposition ("pass boundaries, not a +monolith"). We decline the *dispatch* shape and keep the *data* shape: + +- Our fusions are measured wins from the parity program (see `PARITY.md`): + dilate + depth-clip fused is 0.035 ms where the source's split form costs + ~3×; the shading detector is one fused dispatch at 0.044 ms vs the + two-pass candidate's 0.231 ms. Splitting dispatches to mirror the brief's + diagram would regress performance for zero consumer benefit — a consumer + binds textures, not passes. +- What the brief *actually needs* from the split is (a) the products, and + (b) the ability to produce the early ones **without running the upscale**. + Both are satisfiable with the fusions intact. + +The contract is therefore: **named textures with documented format, space, +resolution, and production stage.** Internally we stay free to fuse, reorder, +or re-derive as long as the contracts hold. + +## 3. The two-stage frame contract + +The real seam in the pipeline is not the brief's pass 1/2/3/4 — it's the line +the brief's own addendum draws ("signal & space"): **geometry products are +signal-agnostic and need only depth + velocity; everything else needs the +final beauty color.** + +``` +frame start + │ (previousDepth, previous lockStatus, previous history already valid — + │ last frame's products are readable before anything runs this frame) + ├─ scene G-buffer/depth/velocity available + ├─ ► dispatchGuides() EARLY stage — geometry guides + │ reconstruct.ts only: dilatedMotion, dilatedDepth, disocclusion + ├─ effects run (SSGI march, SSGI temporal, denoiser…) consuming guides + ├─ final beauty color available + ├─ ► dispatchUpscale() LATE stage — luma-dependent products + upscale + │ exposure → shadingChange → generateReactive → accumulate → RCAS + └─ present +``` + +- **`dispatchGuides(inputs)`** takes `{ depth, velocity }` (+ optional + `reset`/`deltaTime`) and encodes *only* the reconstruct pass. Its three + outputs plus `previousDepth` are then valid for every downstream effect — + this is the brief's "pass 1 runnable without 3–4", at 0.035 ms. +- **`dispatchUpscale(inputs)`** encodes the rest. The existing monolithic + `dispatch()` becomes exactly `dispatchGuides(); dispatchUpscale();` on one + command encoder / one submit — the split must be observable only when the + caller opts into calling the halves. +- **`path: 'guides'`** in `UpscalerConfig`: allocates only the early working + set and never runs the late stage — the "app that never upscales" case. + `outputTexture` throws on this path (there is no output). +- Queue ordering on three's shared device gives correctness for free, same + as today: effects submitted after `dispatchGuides` see its writes. + +**Late products have one-frame latency for render-stage consumers.** Locks, +history age, and shading change are computed *after* the effects that would +consume them have already run. This is not a defect to engineer away — FSR3's +own locks derive from the final color's luminance, so no implementation can +provide same-frame locks to a pass that runs before final color exists. The +brief's SVGF use ("anti-ghost / history-rejection prior") is a prior, and +frame N−1's lock state is the correct prior for frame N. The contract +publishes them as **previous-frame products** with that label. + +## 4. The published bundle + +Access: `upscaler.guides` — a `TemporalGuides` object of three `Texture`s +(consumable as TSL `texture()` nodes and by raw bind groups alike). Getters +resolve ping-pongs to the correct half for "current" vs "previous". + +| product | resolution | format | space / convention | stage | notes | +|---|---|---|---|---|---| +| `dilatedMotion` | render | rgba16float (.xy) | **UV delta**, `prevUV = uv − motion`; y-flip (0.5, −0.5) already applied | early | see deviation D1 | +| `dilatedDepth` | render | r32float | linear view depth (eye-Z), reversed-depth already resolved | early | | +| `previousDepth` | render | r32float | linear view depth, frame N−1 | frame start | retires the consumer's ≥3 private depth copies | +| `disocclusion` | render | rgba8unorm (.r) | graded 0 (stable) → 1 (fresh), AMD confidence-voted | early | | +| `reactive` | render | rgba8unorm (.r) | 0..1 reactivity, post-merge (§6) | late | valid same frame *after* `dispatchUpscale` | +| `shadingChange` | ceil(render/2) | r32float | 0..1 response, block-mean metric | late | | +| `exposure` | 1×1 | rgba16float | r = conditioning pre-exposure, g = avg luma (**exposed beauty luma — not for GI**), b = host pre-exposure | late | space-labeled per the brief's one rule | +| `lockStatus` | **display** | rgba16float | r = lock lifetime, g = locked luma (conditioned tonemap space), b = shading age | late, frame N−1 | consumer downsamples or samples at display UV | +| `historyAge` | **display** | (history `.a`) | accumulated frame count, 0..maxAccumulation | late, frame N−1 | see deviation D3 | + +Consumers MUST treat every field not listed here (e.g. `dilatedMotion.zw`, +`masks.gba`) as reserved — we keep the right to pack new signals into spare +channels, as we already did with locks `.b`. + +### Implementation note: three-visible textures + +Guide textures are currently raw `GPUTexture`s. To publish them we flip the +allocation to the same mechanism as `outputTexture`: allocate as three +`StorageTexture`s (with `generateMipmaps = false`), `renderer.initTexture()`, +and keep using the raw handle internally via `getGPUTexture()`. Zero shader +changes; the pipeline can't tell the difference. Ping-ponged products +(`dilatedDepth`, `locks`, exposure, history) allocate both halves this way +and the `TemporalGuides` getters return the right half per frame; the TSL +guide nodes update their texture reference in `updateBefore` (the same +per-frame re-point pattern `UpscalerNode` already uses internally). If a +consumer needs *stable* texture identity across frames (some graph setups +cache hard), the fallback is an opt-in copy into a stable target — spec'd but +not built until the lab shows it's needed. + +## 5. The statistics primitive (SVGF addendum B) — new code, not a refactor + +The brief assumes FSR3-style per-pixel luminance moments exist to refactor. +**They don't, here.** Our exposure pass is a single 1×1 log-average (no +pyramid, no per-pixel moments — `luminancePyramid.ts` deliberately computes +only what's consumed), and our variance clip computes its 3×3 YCoCg moments +inline in `accumulate.ts` in conditioned tonemap space — exactly the space +the brief forbids for GI variance. + +So `MomentPyramid` is a **new, standalone, signal-agnostic pass**: + +- in: any texture + `{ space: 'linear' | 'ycocg', channels }` — hardcodes no + exposure/tonemap/albedo assumption, per the brief's contract. +- out: `moments` rg16float `(E[x], E[x²])` per pixel; optional mip chain. +- Lives in its own files (`shaders/moments.ts` + a small `MomentsPass` + driver / `moments()` node), exported `@experimental`. It touches nothing + in the core pipeline — zero regression surface. +- Our own pipeline does **not** adopt it initially (accumulate's inline 3×3 + is fused and cheap; swapping it for a consumed pyramid is a perf/quality + trade to measure separately, if ever). One primitive, external consumers + first. +- Verification split: the CPU-reference `Var = E[x²]−E[x]²` check the brief + asks for cannot run in our GPU-free CI; we ship the structural shader test + + a bench-side scripted GPU check, and the consumer's SVGF lab is the + acceptance test (their own stated criterion). + +## 6. Reactive becomes bidirectional (merge, not overwrite) + +Today: explicit `reactive` input wins, else auto-generate from +`reactiveOpaqueColor`, else zero dummy — mutually exclusive. New contract per +the brief: + +- `generateReactive.ts` gains an optional incoming-mask binding; response = + `max(generated, incoming)`. Supplying both inputs now composes instead of + the explicit mask silencing the generator. +- The published `guides.reactive` texture is storage-writable: an effect + (SVGF flagging high-variance GI) can write into it between `dispatchGuides` + and `dispatchUpscale`, and the late stage consumes the merged result. +- This is the only production-shader change in the whole program + (`generateReactive` hash pin updates; GPU re-verify on example 05, which is + the reactive acceptance demo). + +## 7. Deviations from the brief (flagged for the consumer's review) + +- **D1 — motion is a UV delta, not an NDC delta.** We publish the + already-converted form (`velocity.xy · (0.5, −0.5)`, `prevUV = uv − m`). + Rationale: the brief's own trap list says the y-mirror seam "cost us the + demo 03 speckle hunt" — publishing the convention-free, directly-usable + form deletes that seam for every consumer. If the raw NDC delta is truly + needed, the consumer already owns the velocity MRT it came from. +- **D2 — locks/reactivity are late-stage, not "early-mid".** See §3; locks + need final color by construction. Published as frame N−1 priors. +- **D3 — `historyLength` is display-res history `.a`, not a render-res + r16float.** The brief itself allows the consumer to keep producing its own + (demo 03 already does). We publish what exists; a dedicated render-res + count texture is added only if the guides lab shows sampling display-res + age is insufficient. +- **D4 — formats are ours, not the brief's suggestions** (rgba16float + motion vs rg16float, rgba8unorm disocclusion vs r16float). The suggested + formats save memory, not correctness; we won't fork allocations for it. + Revisit under real memory pressure. +- **D5 — no dispatch-level 4-pass split** (§2). Contract is textures. +- **D6 — environment notes in the brief that don't apply here:** we bridge + three internals via `internal/threeWebGPU.ts` (not the bench harness path), + RCAS is already tonemap-agnostic (the ACES-in-RCAS bug is fixed and E03 in + `bench/docs/PARITY-DECISIONS.md` is its record), and `RenderPipeline` + naming is already handled in the node docs. + +## 8. Delivery plan + +Branch `feat-temporal-guides` off `feat-match-fsr3`; each milestone lands +green (165 unit tests / typecheck / lint), GPU-verified per the CLAUDE.md +headless-CDP protocol, and bench-A/B'd where perf could move. Merge back only +when the consumer lab has accepted. + +- **M0 — contract review (this document).** Send to the SSGI/SVGF side; + resolve D1–D5 and the open questions (§9) before code. +- **M1 — internal seam.** Split `_encodeTemporal` into `_encodeGuides` / + `_encodeLate` private halves composed on one encoder; no public API change. + Gate: bench A/B production vs `005de6d` shows Δ within noise (<3%); all + debug views unchanged. +- **M2 — publish the bundle.** Three-visible allocation flip, the + `TemporalGuides` accessor, `dispatchGuides`/`dispatchUpscale` public split, + `path: 'guides'`. New example (`11-temporal-guides`) rendering each guide — + doubles as the GPU acceptance harness. Gate: monolithic `dispatch()` + byte-identical output (capture diff vs `005de6d`). +- **M3 — reactive merge** (§6). Gate: example 05 GPU re-verify; new + `generateReactive` fingerprint; merged-mask capture test in the bench. +- **M4 — TSL surface.** `temporalGuides(depth, velocity, camera)` node + producing guide texture nodes in-graph; `upscale(..., { guides })` to share + one computation between the effect graph and the upscale. Gate: examples + 07/09 unchanged; new node demo consumes `disocclusion` in a toy effect. +- **M5 — `MomentPyramid`** (§5), `@experimental`. Gate: structural test + + scripted GPU check vs CPU reference in the bench (not CI). +- **M6 — cross-repo acceptance.** The consumer's demo-10 guides lab and SVGF + lab run against a tarball/linked build; their A/B (guides-fed SSGI temporal + vs private logic) is the program's exit criterion, per the brief. + +Sequencing note: M1+M2 unblock the consumer's guides lab; M3–M5 can proceed +in parallel with their integration. The guides API ships marked +`@experimental` until M6 passes, so `main` never carries a frozen contract we +haven't seen consumed. + +## 9. Open questions for the consumer side + +1. Is the UV-delta motion convention (D1) acceptable, or is raw NDC needed? +2. Is frame N−1 lock/age latency (D2) sufficient for the SVGF prior? +3. Does the SSGI temporal pass need `historyLength` at render res on day + one, or does display-res age sampling suffice (D3)? +4. Moment pyramid mip chain: how many levels does the short-history spatial + fallback actually read? (We'd rather allocate 3 than "a full chain".) +5. Consumption mechanism: TSL `texture()` nodes, raw bind groups, or both? + (Both are spec'd; if only one is consumed we defer the other's example.) diff --git a/bench/docs/FSR3-BRIEF.md b/bench/docs/FSR3-BRIEF.md new file mode 100644 index 0000000..d66ebd5 --- /dev/null +++ b/bench/docs/FSR3-BRIEF.md @@ -0,0 +1,176 @@ +# FSR3-BRIEF.md — how the FSR3 port should decompose to flow with this pipeline + +Audience: the agents working on Dennis's separate FSR3-for-three.js port. That +repo stays self-contained; this brief defines the *decomposition and contracts* +so its internals can be produced early and consumed by effects (SSGI temporal, +denoisers) well before the upscale — instead of shipping as one monolith that +only runs last. + +## The architectural thesis (why decompose) + +Disocclusion, dilated motion vectors, reactivity, and history validity are +**frame properties, not upscaler properties**. FSR3 computes them internally +and hides them; meanwhile every temporal effect upstream (our SSRT3 temporal +pass, bilateral denoisers, any TAA) re-derives worse versions privately. The +port should expose FSR3's early data products as standalone passes with clean +texture contracts, so one computation feeds everything. See +`advantagous-concepts.md` ("temporal guides") for the consumer-side motivation. + +## Required decomposition (pass boundaries, not a monolith) + +1. **Reconstruct & dilate** (early — immediately after G-buffer): + - in: depth, motion vectors, prev depth + - out: `dilatedVelocity` (closest-depth 3×3 dilation), `dilatedDepth`, + `disocclusionMask` (graded 0..1, depth-reprojection based — NOT binary) +2. **Locks / reactivity** (early-mid): + - in: luma, history, app-provided reactive hints + - out: `reactiveMask`, `lockStatus` (or nearest FSR3 equivalents) +3. **Accumulate + upsample** (late): consumes 1+2 plus color; owns history. +4. **RCAS sharpen** (last): consumes 3 only. + +Each pass = explicit input/output textures with documented formats and a +uniform block; no pass reaches into another's internals. Passes 1–2 must be +runnable WITHOUT 3–4 (that is the whole point): an app that never upscales can +still run them as its temporal-guides provider. + +## Contracts this bench expects (the "temporal guides" bundle) + +Produced once per frame, post-G-buffer, pre-effects: + +| texture | format (suggested) | contents | +|---|---|---| +| `dilatedVelocity` | rg16float | closest-depth-dilated NDC delta | +| `disocclusion` | r16float | graded disocclusion 0 (stable) .. 1 (fresh) | +| `historyLength` | r16float | per-pixel valid-history frame count N | +| `reactive` | r8unorm | app/effect-flagged fast-changing pixels | + +Notes: +- `historyLength` may be produced by the consumer instead (demo 03 already + maintains one for frame-count accumulation); the contract just names it. +- `reactive` flows the other way too: GI/multibounce passes can WRITE into it + (fast-changing GI ⇒ don't lock/ghost) — design the pass to accept an + optional pre-populated mask and merge rather than overwrite. + +## three.js r185 / WebGPU environment facts (traps we already paid for) + +- `PostProcessing` is deprecated → `RenderPipeline`; after reassigning + `outputNode` you MUST set `needsUpdate = true` or the graph silently keeps + rendering the old output. +- Velocity convention (three MRT `velocity`): ndc delta; UV reprojection is + `prevUV = uv - velocity.xy * vec2(0.5, -0.5)` (TRAANode convention). +- Compute/raw passes: top-left UV origin, y-down. Anything derived from Unity + or Shadertoy (y-up) needs the y-mirror at the screen-step seam — this bug + cost us the demo 03 "box-top speckle" hunt; check every direction that is + used in BOTH view space and screen space. +- Raw WGSL passes never allocate GPU textures: allocate as three + RenderTarget/StorageTexture, bridge via `renderer.backend.get(...)` + (isolated in one file, see `bench/src/harness/three-internals.ts`), so + outputs stay consumable as TSL `texture()` nodes. +- Camera jitter: `setViewOffset` (TRAANode convention); harness helpers exist + (`bench/src/harness/temporal.ts`, Halton(2,3)). +- Known port bug from earlier work: **ACES tonemapping baked into RCAS** — + RCAS must be tonemap-agnostic (sharpen in the same space FSR3 specifies, + don't embed a display transform; three applies output encoding itself). + Shadertoy-derived passes similarly must not carry a trailing `pow(x, 0.45)`. + +## Verification expectations (same discipline as this bench) + +- Headed browser only for WebGPU verification (headless chromium = software + adapter = black canvas). `--enable-unsafe-webgpu`, no Vulkan flag on macOS. +- Each pass verified standalone with a scripted capture before integration; + numbers recorded with config + commit (see TESTING.md methodology). +- The bench will consume passes 1–2 in a lab (demo 10 extension) A/B-ing + guides-fed SSGI temporal vs its private logic — that lab is the acceptance + test for the decomposition being real and not cosmetic. + +## Addendum — data products SVGF wants (denoiser as first-class consumer) + +Motivation: the bench is adding an SVGF-style GI denoiser (variance-guided +à-trous). Its temporal front-end is *identical* to FSR3's, and its missing +half (per-pixel variance) reuses FSR3's luminance-statistics machinery. This +addendum expands the decomposition so the port serves the denoiser directly. +See `advantagous-concepts.md` (SVGF-style variance moments) for the +consumer-side motivation; the bench SVGF lab is the acceptance test for the +statistics primitive below being real and not cosmetic. + +**The one rule that governs this addendum — signal & space.** FSR3's +frame-property outputs (motion, disocclusion, depth, history length) are +*signal-agnostic*: correct for any temporal consumer. Its luminance-derived +products are NOT — they live on tonemapped display-luminance of the full +beauty, in FSR3's exposure/YCoCg conventions, sometimes at display res. SVGF +variance must live on **GI irradiance: pre-albedo, linear HDR, render res**. +So the denoiser consumes FSR3's *geometry/frame data* directly, but must NOT +consume its *luminance buffers* — it reuses the luminance *pass* on a +different input. Every luma-derived output MUST carry a space label so a +consumer cannot mis-apply display-luma to a linear-HDR denoiser. + +### A. Promote three internals into the published contract + +These already exist inside passes 1–2 but aren't in the consumed bundle. SVGF, +the SSGI march, and the temporal pass each re-derive them privately today; +publish once. + +| texture | format (suggested) | contents | space | +|---|---|---|---| +| `dilatedDepth` | r32float | closest-depth 3×3 dilated depth (from pass 1) | linear eye-Z | +| `prevDepth` | r32float | previous frame's depth (pass-1 owns the copy) | linear eye-Z | +| `lockStatus` | rg16float | FSR3 lock state / trust (or nearest equivalent) | unitless | + +- `prevDepth`: the bench maintains ≥3 private copies right now + (`ssrt-copydepth.wgsl`, `ssrt12-copydepth.wgsl`, guides-commit). One + published copy retires all of them. +- `lockStatus` feeds SVGF as an anti-ghost / history-rejection prior; if the + port has no clean lock equivalent, omit rather than approximate. + +### B. The reusable statistics primitive (SVGF's missing half) + +FSR3 already computes luminance moments/pyramids for its own stability and +neighborhood-clamp logic. SVGF's variance stages want the SAME computation on +a DIFFERENT signal. Factor it as a signal-agnostic pass, not a beauty-hardcoded +internal: + + Pass: MomentPyramid (or nearest FSR3 luma-pyramid refactor) + in: + source : any texture (FSR3 passes beauty-luma; SVGF passes GI) + space : enum { linear-irradiance, tonemapped-display, ycocg } + channel : which channel(s) form the scalar "luminance" + out: + moments : rg16float — (E[x], E[x^2]) per pixel → Var = E[x^2]-E[x]^2 + pyramid : optional mip chain of the above (spatial fallback, + short-history pixels borrow a coarser level) + + Contract: the pass reads `source`/`space` as parameters and hardcodes + NO exposure/tonemap/albedo assumption. FSR3 instantiates it on beauty-luma + for its stability term; the bench instantiates the same code on pre-albedo + GI irradiance for SVGF variance. One primitive, two consumers — the guides + thesis applied to statistics instead of geometry. + +If the accumulate stage already tracks a per-pixel accumulation weight / +confidence distinct from `historyLength`, expose it too (`r16float`) — SVGF +consumes it as a variance prior for freshly-accumulated pixels. + +### C. What SVGF does NOT need from the port (avoid over-serving) + +- **The à-trous / spatial wavelet filter** — SVGF builds its own; RCAS is a + display-res sharpen at the wrong stage and is not a substitute. +- **Reconstructed-from-depth normals** — the bench has real G-buffer view + normals (`normalView`); FSR3's depth-reconstructed normals are strictly + worse. Do not consume them. +- **Any display-res or post-tonemap luminance buffer** — see the space rule; + wrong signal, wrong space for GI denoising. + +### D. Reactive is bidirectional here too + +SVGF is a `reactive` PRODUCER as well as a consumer: high GI variance ⇒ flag +reactive (fast-changing GI shouldn't lock/ghost in the upscaler). Same +merge-not-overwrite contract as the base `reactive` note above. + +### E. Verification + +- The `MomentPyramid` primitive is verified standalone: instantiate on a known + synthetic input, assert `Var = E[x^2]-E[x]^2` matches a CPU reference within + tolerance, in BOTH a linear and a ycocg space, before any denoiser consumes + it. Numbers recorded with config + commit per TESTING.md. +- The SVGF lab (bench, demo-13 candidate) A/Bs variance-guided denoise vs the + current fixed-radius bilateral — that lab is the acceptance test for this + addendum's statistics primitive being real and not cosmetic. From 77ea660d92c62980de5e902e7eb4e12b80963fb3 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 21:47:35 +0900 Subject: [PATCH 08/22] =?UTF-8?q?docs:=20record=20M0=20resolution=20?= =?UTF-8?q?=E2=80=94=20contract=20frozen,=20all=20deviations=20accepted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- GUIDES-SPEC-RESPONSE.md | 113 ++++++++++++++++++++++++++++++++++++++++ TEMPORAL-GUIDES-SPEC.md | 36 ++++++++++++- 2 files changed, 147 insertions(+), 2 deletions(-) create mode 100644 GUIDES-SPEC-RESPONSE.md diff --git a/GUIDES-SPEC-RESPONSE.md b/GUIDES-SPEC-RESPONSE.md new file mode 100644 index 0000000..ceabdd4 --- /dev/null +++ b/GUIDES-SPEC-RESPONSE.md @@ -0,0 +1,113 @@ +# GUIDES-SPEC-RESPONSE.md — consumer-side M0 review (bench → FSR port) + +Response to `TEMPORAL-GUIDES-SPEC.md` §7 (deviations) and §9 (open +questions), plus the pre-build decisions `SVGF-SPEC.md` §8 asks for. +Written 2026-07-21 from the bench/consumer side. Nothing here is code; per +M0, contract agreement precedes build on both sides. + +## Verdict up front + +The spec's central move — **publish products, not pass boundaries (D5)** — +is accepted and is better than what FSR3-BRIEF asked for. The brief wanted +the split because it wanted (a) the products and (b) early-stage-without- +upscale; `dispatchGuides()` / `path: 'guides'` delivers both while keeping +the port's measured fusions. Our own harness prototype went through the same +reasoning at smaller scale (one class, three dispatches, texture getters — +consumers bind textures, never passes), so the contract shapes already match +in spirit. FSR3-BRIEF's "Consumption requirements v2" section (commit +c7ba382) should be read with its pass-boundary language superseded by D5; +its Req-1 acceptance (the live demo 10 rig) stands unchanged and maps to +your M6. + +## §9 answers + +**1. D1 — UV-delta motion: ACCEPTED, and preferred.** Publishing the +directly-usable form (`prevUV = uv − motion`, y-flip pre-applied) deletes +the convention seam that cost us the demo 03 speckle hunt. Raw NDC is not +needed — every consumer that wants it owns the velocity MRT it came from. +Consumer-side consequence we own: our harness `TemporalGuidesPass` currently +publishes the raw-velocity convention (consumers apply `· (0.5, −0.5)` +themselves, e.g. `ssrt3-guides-temporal.wgsl`). Before the swap we will +align the harness pass to publish YOUR convention, so the port's bundle is +drop-in against unmodified consumer shaders. That alignment is bench-side +work, scheduled with the swap, not before. + +**2. D2 — frame N−1 locks/age: SUFFICIENT.** A prior is previous-frame by +definition. Our only shipping consumer of history age (demo 13 `adaptive`) +already reads the *previous* frame's N — that is how it works today, at +3.17× — so N−1 latency is the semantics we validated, not a compromise. +Same for the SVGF history-rejection prior. + +**3. D3 — display-res age sampling: fine for day one.** Accepted with one +recorded risk: our adaptive march reads N per march-res pixel (currently +half render-res); sampling display-res age at march UV is a filtered +approximation and may dilate age across silhouettes → transient over-effort +at edges (safe direction: over-effort, never under-quality). The guides lab +measures exactly that. Fallback exists and costs you nothing: demo 03/12/13 +keep their own render-res counter (the brief always allowed the consumer to +produce `historyLength`); we only retire it if the lab shows display-res +sampling is clean. + +**4. Moment pyramid: 3 levels.** SVGF's short-history fallback is one +coarse-neighborhood read (7×7-equivalent ≈ mip 2); nothing in the SVGF-SPEC +chain reads deeper. Allocate mip 0 + 2. + +**5. Consumption mechanism: raw bind groups first, TSL second.** Every +hot-path consumer (march, temporal, à-trous) is raw WGSL; TSL consumption is +wanted only composite-side (display/debug and the bent-normal/spec-occlusion +env terms). Per-frame texture re-pointing is fine — our `RawComputePass` +re-binds per execute, and our TSL usage follows the same `updateBefore` +re-point pattern you cite — so the stable-identity copy can stay spec'd-only. +If M4 needs trimming, defer the TSL example, not the raw path. + +**D4 (formats), D6 (environment):** accepted; format is read from the +texture at bind time on our side, channel conventions (.r reads) noted. +Reserved-channel rule acknowledged — we will never read unlisted fields. + +**Producer-of-record confirmed:** the SVGF addendum (§A/§B/§E, +`MomentPyramid`, the signal-&-space rule) is present in this repo's +`FSR3-BRIEF.md` and matches what both specs cite. (An earlier draft of this +response flagged a sync gap — retracted; it was a stale read on the consumer +side.) One brief-side reconciliation was applied instead: the brief's +"Consumption requirements v2" and bundle table now note that the accepted +deviations D1 (UV-delta motion) and D4 (port-native formats) supersede the +brief's suggested convention/formats — see the brief's "Contract resolution" +note. + +## SVGF-SPEC §8 pre-build decisions (recommendations) + +1. **Moments: scalar luminance in YCoCg-Y** (spec default). Two channels; + revisit RGB variance only if chroma noise visibly survives — the + noiseinject arm will show it cheaply. +2. **History feedback: A/B it, default to the paper convention** + (first-iteration output feeds history). The over-stabilization risk is + real but it is exactly what the stability metric + reconvergence protocol + measure — let the numbers pick. +3. **À-trous full-res first.** Clean H1/H5 read; half-res à-trous is a + follow-up arm measured against the lab-09 chain it would ride. +4. **`svgf-noiseinject` first; the path tracer only if demo 14 is + ambiguous.** Agree with the spec's own recommendation and scope-honesty: + the PT is the largest build in the spec and is justified solely by + unbiased-GT + noise knob. One addition: if the PT is built, its + 1024-spp accumulation doubles as the long-deferred *absolute* ground + truth for cornell (the Cycles stand-in) — worth stating in 15-ptref's + goals so the build buys two things. + +Also endorsed explicitly: SVGF-lite as a first-class arm (H1 isolation at +bilateral cost is the likeliest shippable outcome if H0 holds on the SSGI +signal), and the honest H0 framing itself — a measured "no" on our +already-temporally-filtered signal would match our lab-10 experience and +would still leave demo 15 standing as an independent artifact. + +## Sequencing from the consumer side + +- Now: this M0 response goes to the port side; both sides resolve the + addendum sync flag. +- Your M1+M2 unblock our guides lab (demo 10 rig, already live) — that A/B + is the exit criterion and needs nothing new built here. +- Demo 14/15 build starts only after M2 (guides bundle) + M5 + (`MomentPyramid`) exist to consume — per SVGF-SPEC's own thesis, building + it standalone would duplicate the front-end and invalidate the "small + build" premise. +- Bench-side jitter/AA consumption (FSR3-BRIEF Req 2) remains post-pass-3; + unaffected by this review. diff --git a/TEMPORAL-GUIDES-SPEC.md b/TEMPORAL-GUIDES-SPEC.md index fa66e8a..05aed9d 100644 --- a/TEMPORAL-GUIDES-SPEC.md +++ b/TEMPORAL-GUIDES-SPEC.md @@ -1,6 +1,8 @@ # Temporal Guides — opening the upscaler's internals (spec) -Status: **draft for review** (2026-07-21, branch `feat-temporal-guides`). +Status: **M0 resolved — contract frozen for build** (2026-07-21, branch +`feat-temporal-guides`; consumer review in +[GUIDES-SPEC-RESPONSE.md](GUIDES-SPEC-RESPONSE.md), resolution in §10). Request: [bench/docs/FSR3-BRIEF.md](bench/docs/FSR3-BRIEF.md) — the consuming pipeline (SSGI temporal pass, an SVGF-style denoiser, any TAA-class effect) wants the upscaler's early data products as first-class outputs instead of @@ -150,7 +152,10 @@ So `MomentPyramid` is a **new, standalone, signal-agnostic pass**: - in: any texture + `{ space: 'linear' | 'ycocg', channels }` — hardcodes no exposure/tonemap/albedo assumption, per the brief's contract. -- out: `moments` rg16float `(E[x], E[x²])` per pixel; optional mip chain. +- out: `moments` rg16float `(E[x], E[x²])` per pixel, plus one coarse level + (mip-2-equivalent, 4× reduction) — the consumer's short-history fallback + reads exactly one coarse neighborhood and nothing deeper (§10, answer 4). + No full chain. - Lives in its own files (`shaders/moments.ts` + a small `MomentsPass` driver / `moments()` node), exported `@experimental`. It touches nothing in the core pipeline — zero regression surface. @@ -229,6 +234,8 @@ when the consumer lab has accepted. producing guide texture nodes in-graph; `upscale(..., { guides })` to share one computation between the effect graph and the upscale. Gate: examples 07/09 unchanged; new node demo consumes `disocclusion` in a toy effect. + Priority per §10 answer 5: every hot-path consumer binds raw — if M4 needs + trimming, defer the TSL example, never the raw path. - **M5 — `MomentPyramid`** (§5), `@experimental`. Gate: structural test + scripted GPU check vs CPU reference in the bench (not CI). - **M6 — cross-repo acceptance.** The consumer's demo-10 guides lab and SVGF @@ -250,3 +257,28 @@ haven't seen consumed. fallback actually read? (We'd rather allocate 3 than "a full chain".) 5. Consumption mechanism: TSL `texture()` nodes, raw bind groups, or both? (Both are spec'd; if only one is consumed we defer the other's example.) + +## 10. M0 resolution (2026-07-21) + +Consumer review: [GUIDES-SPEC-RESPONSE.md](GUIDES-SPEC-RESPONSE.md). Every +deviation D1–D6 accepted; the brief's pass-boundary language and suggested +formats are superseded by D5/D1/D4 (reconciled on the brief side). §9 +answers, now binding: + +1. **UV-delta motion accepted and preferred.** The consumer harness will + align its own guides pass to *our* convention before the swap (their + work, scheduled with the swap) so the bundle is drop-in. +2. **Frame N−1 locks/age sufficient** — it is the semantics their shipping + consumers already validated, not a compromise. +3. **Display-res age sampling accepted for day one**, with one recorded + risk their guides lab measures: filtered age may dilate across + silhouettes → transient over-effort at edges (fails safe). Their + render-res counter remains as fallback; we build nothing extra unless + the lab rejects display-res sampling. +4. **Moments: mip 0 + one coarse level (mip-2-equivalent) only.** Nothing + reads deeper. +5. **Raw bind groups first; TSL second.** Per-frame texture re-pointing is + fine on their side; the stable-identity copy (§4) stays spec'd-only. + +Standing acceptance criteria unchanged: their live demo-10 guides rig is +M6's exit A/B; their SVGF/moments lab builds only after our M2 + M5 exist. From 3603a146f9746bf44a90f0ad648a2a990647274d Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 21:50:46 +0900 Subject: [PATCH 09/22] refactor: split temporal encode into guides/late stages (M1, no API change) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit _encodeTemporal now composes _encodeGuides (reconstruct only — the signal-agnostic geometry stage) and _encodeLate (everything needing beauty color) on one encoder, so the monolithic dispatch keeps its single submit. Seam for the split-dispatch guides API (TEMPORAL-GUIDES-SPEC M2). Co-Authored-By: Claude Fable 5 --- src/Upscaler.ts | 72 +++++++++++++++++++++++++++++++------------------ 1 file changed, 46 insertions(+), 26 deletions(-) diff --git a/src/Upscaler.ts b/src/Upscaler.ts index e50af82..b773eaf 100644 --- a/src/Upscaler.ts +++ b/src/Upscaler.ts @@ -637,8 +637,19 @@ export class Upscaler { return; } - const depthGPU = getGPUTexture(this._renderer, inputs.depth); - const velocityGPU = getGPUTexture(this._renderer, inputs.velocity); + // The frame's two stages (TEMPORAL-GUIDES-SPEC §3): geometry guides + // need only depth + velocity; everything after needs the beauty color. + // Composed on one encoder here so the monolithic dispatch keeps its + // single submit — the seam exists for the split-dispatch guides API. + this._encodeGuides(encoder, inputs); + this._encodeLate(encoder, colorGPU, inputs); + } + + // Early stage — the reconstruct pass (fused dilate + depth clip). Produces + // the signal-agnostic geometry guides: dilated depth/motion, disocclusion. + private _encodeGuides(encoder: GPUCommandEncoder, inputs: DispatchInputs): void { + const depthGPU = getGPUTexture(this._renderer, inputs.depth!); + const velocityGPU = getGPUTexture(this._renderer, inputs.velocity!); this._checkMsaa(depthGPU, 'depth'); this._checkMsaa(velocityGPU, 'velocity'); // Stencil-less depth formats bind directly; combined formats need a @@ -646,9 +657,41 @@ export class Upscaler { const depthView = depthGPU.createView( depthGPU.format.includes('stencil') ? { aspect: 'depth-only' } : undefined, ); - const depthCur = this._dilatedDepth![this._depthIndex]; const depthPrev = this._dilatedDepth![1 - this._depthIndex]; + + //* Reconstruct — dilate (nearest-depth motion/depth over 3×3) + depth + //* clip (disocclusion vs last frame's dilated depth) fused into one pass. + const reconstructBindGroup = this._reconstructPass.createBindGroup([ + { buffer: this._constants.buffer }, + depthView, + velocityGPU.createView(), + depthPrev.createView(), + depthCur.createView(), + this._dilatedMotion!.createView(), + this._masks!.createView(), + ]); + const reconstructPass = encoder.beginComputePass({ + label: 'upscale-reconstruct', + timestampWrites: this._timer.passDescriptor('reconstruct'), + }); + this._reconstructPass.dispatch( + reconstructPass, + reconstructBindGroup, + this._renderWidth, + this._renderHeight, + ); + reconstructPass.end(); + } + + // Late stage — everything that needs the final beauty color: reactive, + // exposure, shading change, accumulate, and the output pass. + private _encodeLate( + encoder: GPUCommandEncoder, + colorGPU: GPUTexture, + inputs: DispatchInputs, + ): void { + const depthCur = this._dilatedDepth![this._depthIndex]; const historyIn = this._history![this._historyIndex]; const historyOut = this._history![1 - this._historyIndex]; const locksIn = this._locks![this._historyIndex]; @@ -713,29 +756,6 @@ export class Upscaler { this._exposurePass.dispatch(exposurePass, exposureBindGroup, 8, 8); exposurePass.end(); - //* Reconstruct — dilate (nearest-depth motion/depth over 3×3) + depth - //* clip (disocclusion vs last frame's dilated depth) fused into one pass. - const reconstructBindGroup = this._reconstructPass.createBindGroup([ - { buffer: this._constants.buffer }, - depthView, - velocityGPU.createView(), - depthPrev.createView(), - depthCur.createView(), - this._dilatedMotion!.createView(), - this._masks!.createView(), - ]); - const reconstructPass = encoder.beginComputePass({ - label: 'upscale-reconstruct', - timestampWrites: this._timer.passDescriptor('reconstruct'), - }); - this._reconstructPass.dispatch( - reconstructPass, - reconstructBindGroup, - this._renderWidth, - this._renderHeight, - ); - reconstructPass.end(); - //* Shading Change — signed luma-difference pyramid (skipped entirely //* when the detector is off; accumulate then reads a zero dummy). let shadingSignalView = this._reactiveDummy!.createView(); From 4f4a7f0f4b8b08d142de126a0f5b2261da823976 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Tue, 21 Jul 2026 21:58:00 +0900 Subject: [PATCH 10/22] =?UTF-8?q?docs:=20M1=20verified=20=E2=80=94=20perf?= =?UTF-8?q?=20within=20noise,=20captures=20byte-identical?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- TEMPORAL-GUIDES-SPEC.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/TEMPORAL-GUIDES-SPEC.md b/TEMPORAL-GUIDES-SPEC.md index 05aed9d..a04c94a 100644 --- a/TEMPORAL-GUIDES-SPEC.md +++ b/TEMPORAL-GUIDES-SPEC.md @@ -219,10 +219,19 @@ when the consumer lab has accepted. - **M0 — contract review (this document).** Send to the SSGI/SVGF side; resolve D1–D5 and the open questions (§9) before code. -- **M1 — internal seam.** Split `_encodeTemporal` into `_encodeGuides` / - `_encodeLate` private halves composed on one encoder; no public API change. - Gate: bench A/B production vs `005de6d` shows Δ within noise (<3%); all - debug views unchanged. +- **M1 — internal seam. DONE (`3603a14`, 2026-07-21).** Split + `_encodeTemporal` into `_encodeGuides` / `_encodeLate` private halves + composed on one encoder; no public API change. Gates met (evidence: + `bench/results/raw/GUIDES-M1/`): worktree-vs-worktree smoke A/B vs + `005de6d` — compute-sum −2.7%, within the <3% noise policy; Q0 captures + (3 frames × final + motion-vectors/disocclusion/accumulation-age) + byte-identical except frame-119 `final`, where the *pre-M1 run's own two + arms already disagree (harness nondeterminism at deep accumulation, + 0.098% px, max 23/255) while the M1 run's arms agree. Bench caveat + learned: runs launched from a scratchpad worktree read ~3× slower + absolute (GPU stays in a low power state; likely cold-vite frame + delivery) — uniform across passes, so A/B *within* that environment is + valid, but never compare worktree absolutes against repo-run records. - **M2 — publish the bundle.** Three-visible allocation flip, the `TemporalGuides` accessor, `dispatchGuides`/`dispatchUpscale` public split, `path: 'guides'`. New example (`11-temporal-guides`) rendering each guide — From 7f452f50ca501a0d84907e0bc2413c34b76fa4d3 Mon Sep 17 00:00:00 2001 From: Dennis Smolek Date: Wed, 22 Jul 2026 00:13:17 +0900 Subject: [PATCH 11/22] feat: publish the temporal-guides bundle + split dispatch (M2, experimental) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - upscaler.guides (TemporalGuides): the production working set as three StorageTextures — dilated motion/depth, disocclusion, reactive, shading change, exposure, locks, history — with ping-pongs resolved to the most recently written half. - dispatchGuides()/dispatchUpscale(): the split frame, so geometry guides exist post-G-buffer, pre-color; path 'guides' runs the early stage alone. - GpuTimer merges per-label results (a split frame is two submits). - examples/12-temporal-guides: split-dispatch demo + headless GPU harness. GPU gates: Q0 captures byte-identical vs pre-publish (24/24), example 12 and the guides-only path validation-clean headlessly. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 35 ++ README.md | 14 +- TEMPORAL-GUIDES-SPEC.md | 16 +- examples/12-temporal-guides/index.html | 53 +++ examples/12-temporal-guides/main.ts | 187 +++++++++++ examples/index.html | 11 + examples/vite.config.ts | 1 + src/UpscalePass.ts | 6 + src/Upscaler.ts | 431 ++++++++++++++++++++----- src/index.ts | 2 + src/internal/GpuTimer.ts | 8 +- src/types.ts | 87 ++++- 12 files changed, 770 insertions(+), 81 deletions(-) create mode 100644 examples/12-temporal-guides/index.html create mode 100644 examples/12-temporal-guides/main.ts diff --git a/CLAUDE.md b/CLAUDE.md index d00d3f9..0ed56ad 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -68,6 +68,15 @@ When something breaks after an edit, expect failures in this order of likelihood Don't trust "it builds" as "it works." Drive the real bench. +Bench caveat (measured 2026-07-21): benchmark runs launched from a scratchpad +**git worktree** read ~3× slower absolute GPU times, uniformly across all passes +(the GPU never leaves its low power state — likely cold-vite frame delivery). +A/B comparisons *within* that environment are valid; never compare worktree +absolutes against repo-run records. Also: a `node_modules` **symlink** in a +worktree isn't matched by the root `.gitignore`'s `node_modules/` pattern +(trailing slash ≠ symlink) and crashes the benchmark's working-tree digest — +`.git/info/exclude` carries a slash-less `node_modules` entry for this. + --- ## Commands @@ -205,6 +214,32 @@ explicit acceptance test), RCAS denoise on `06-screenspace-gi`. - **Jitter default = ON for temporal** (`UpscalerConfig.jitter`, `UpscalerNodeOptions.jitter`): jitter buys *reconstruction* (detail beyond render res) but only if the input is re-rendered under the jittered projection each frame. Because a composable node's inputs are graph dependencies three renders *in-pipeline* — after this node's `onBeforeRenderPipeline` jitter hook offsets the camera — the offset **does** land on them, so **both** `upscale()` and `upscaleScene()` default jitter **on** (this is how real FSR/DLSS run). Opt **out** (`{ jitter: false }`) only when the input is *not* re-rendered in-graph — an externally-filled `texture()`, or a noisy GI/RT buffer you want reprojected/denoised but not reconstructed (the raw `Upscaler` / example 06 is usually the better fit there). Jitter-off stays a full temporal upscale (reproject + accumulate + denoise), just no sub-pixel offset. When off, `beginFrame` no-ops `setViewOffset`, jitter constants stay zero, and the node skips the hook + velocity compensation. A temporal node that never receives depth+velocity `console.warn`s once. `09-kitchen-sink` toggles jitter on the same in-graph pipeline to A/B it. - Color path (both): the node emits linear/HDR color. When it is the final graph node, three's RenderPipeline applies the renderer's configured tone mapping and output color space; otherwise it can feed later linear post-processing. **Note:** three renamed `PostProcessing` → `RenderPipeline` (deprecation warning only). - The composable node *does* render its inputs in-graph, so an SSGI-in-a-graph pipeline jitters correctly and there's no owning-render/consuming-inputs split. An imperative pipeline that composites *outside* the post render (its own RT loop) still wants the raw `Upscaler` — `examples/06-screenspace-gi` stays on it as the imperative reference. +- **Temporal guides (experimental, branch `feat-temporal-guides`).** The production + working set is published as `upscaler.guides` (`TemporalGuides` — dilated + motion/depth, disocclusion, reactive, shading change, exposure, locks, history) + and the frame can be driven split: `dispatchGuides({depth, velocity})` right + after the G-buffer (geometry guides only — reconstruct is the whole early + stage), then `dispatchUpscale({color, …})`; `path: 'guides'` runs the early + stage alone with no output texture. Contracts + program plan: + `TEMPORAL-GUIDES-SPEC.md` (root); consumer M0 review: `GUIDES-SPEC-RESPONSE.md`. + **Mechanisms a change must not break:** (1) guide textures are allocated via + `_createSharedTexture` — a three `StorageTexture` + `initTexture()`, with the + raw handle fetched back through `getGPUTexture()`; passes bind the raw handle, + consumers sample the three texture, and the two must stay the same allocation. + r32float products are pinned `NearestFilter` (non-filterable format — a linear + sampler on them is a WebGPU validation error). (2) Ping-ponged products resolve + through `_latestDepthWrite`/`_latestHistoryWrite` (set at encode time), NOT the + frame-end-flipped `_depthIndex`/`_historyIndex` — the getters must be correct + both mid-frame (between the split dispatches) and after the frame. (3) The + monolithic `dispatch()` stays one submit (`_encodeGuides` + `_encodeLate` on one + encoder) — GPU-verified byte-identical (Q0 captures) and perf-neutral (−2.7%, + within noise) against the pre-split pipeline; keep it that way. (4) A split + frame is two submits, so `GpuTimer` merges per-label results instead of + replacing the map. (5) Frame-end bookkeeping (index flips, `_frameIndex`, + `_pendingReset`) happens exactly once per frame: in `dispatch()`, in + `dispatchUpscale()`, or — guides path only — in `dispatchGuides()`. + `examples/12-temporal-guides` is the live reference + headless-verification + target (it exposes `window.__guidesExample` for the CDP harness). - **MSAA input — rejected by design.** FSR's temporal path *is* the anti-aliaser (Native AA mode is exactly that), so the correct input is an aliased, single-sample, jittered render with MSAA **off** — MSAA is redundant with FSR's own AA, costs perf, and a multisampled texture can't even bind to the compute passes. (Stacking a *temporal* AA — TAA/`traa` — before FSR is worse still: double-jitter smear; example 06 already drops `traa` for this reason.) `Upscaler` warns once if handed a multisampled input (`_checkMsaa`). **Performance structure:** dilate + depth-clip are fused into the single `reconstruct.ts` dispatch (GPU-verified disocclusion unchanged); the shading detector is one fused workgroup-local reduction instead of the source's SPD mip chain + resolve pair. The measured story of these divergences from FSR 3.1.5 — and the four upstream behaviors adopted in re-derived form — is `PARITY.md` (root) with evidence in `bench/docs/NEXT-STEPS.md`. diff --git a/README.md b/README.md index bd6c740..5853cf6 100644 --- a/README.md +++ b/README.md @@ -141,9 +141,21 @@ Full per-pass details and deviations from the FidelityFX reference: [`src/shader Three doesn't expose its WebGPU internals publicly, so the upscaler grabs `renderer.backend.device` and the `GPUTexture` handles behind render-target attachments (`internal/threeWebGPU.ts` documents exactly which internals we touch and throws loudly if a three upgrade changes them). Compute passes are encoded on our own `GPUCommandEncoder` and submitted between three's scene render and the presentation draw — queue order guarantees correctness with zero synchronization code. The final image lands in a three `StorageTexture` so presenting it is ordinary three code. +### Temporal guides (experimental) + +Dilated motion, dilated depth, and disocclusion are **frame properties, not upscaler properties** — every temporal effect upstream (SSGI/SSR temporal reprojection, denoisers, any TAA-class pass) needs them and usually re-derives worse versions privately. The upscaler publishes its internal working set as the **temporal guides** bundle (`upscaler.guides`, ordinary three textures), and the frame can be driven split so the geometry guides exist *before* the final color does: + +```ts +upscaler.dispatchGuides({ depth, velocity }, camera); // right after the G-buffer +// … effects sample upscaler.guides.dilatedMotion / .disocclusion / .dilatedDepth … +upscaler.dispatchUpscale({ color, deltaTime }, camera); // finish the frame +``` + +An app that never upscales can run `path: 'guides'` for the geometry products alone. Per-product contracts (format, space, resolution, latency) are documented on the `TemporalGuides` type and in [`TEMPORAL-GUIDES-SPEC.md`](./TEMPORAL-GUIDES-SPEC.md); `examples/12-temporal-guides` is the live reference. Marked experimental until the first external consumer integration lands. + ## Status -The pipeline is **feature-complete and GPU-verified**: spatial (FSR1) and temporal paths, luminance-stability locks, auto-exposure (+ external and host pre-exposure inputs), multi-scale shading-change detection, reactive masks (explicit + auto-generated), RCAS with opt-in denoise, imperative `UpscalePass`, and the composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`). A benchmarking program A/B-compared this implementation against source-style FSR 3.1.5 pass graphs on-GPU; the adopted results and remaining divergences — with measurements — are written up in [`PARITY.md`](./PARITY.md). +The pipeline is **feature-complete and GPU-verified**: spatial (FSR1) and temporal paths, luminance-stability locks, auto-exposure (+ external and host pre-exposure inputs), multi-scale shading-change detection, reactive masks (explicit + auto-generated), RCAS with opt-in denoise, imperative `UpscalePass`, and the composable TSL nodes (`upscale` / `upscaleScene` / `upscaleSpatial`). The experimental **temporal guides** surface (above) is in active development on its own track. A benchmarking program A/B-compared this implementation against source-style FSR 3.1.5 pass graphs on-GPU; the adopted results and remaining divergences — with measurements — are written up in [`PARITY.md`](./PARITY.md). Deliberately **not** planned: diff --git a/TEMPORAL-GUIDES-SPEC.md b/TEMPORAL-GUIDES-SPEC.md index a04c94a..4be5a89 100644 --- a/TEMPORAL-GUIDES-SPEC.md +++ b/TEMPORAL-GUIDES-SPEC.md @@ -232,11 +232,17 @@ when the consumer lab has accepted. absolute (GPU stays in a low power state; likely cold-vite frame delivery) — uniform across passes, so A/B *within* that environment is valid, but never compare worktree absolutes against repo-run records. -- **M2 — publish the bundle.** Three-visible allocation flip, the - `TemporalGuides` accessor, `dispatchGuides`/`dispatchUpscale` public split, - `path: 'guides'`. New example (`11-temporal-guides`) rendering each guide — - doubles as the GPU acceptance harness. Gate: monolithic `dispatch()` - byte-identical output (capture diff vs `005de6d`). +- **M2 — publish the bundle. DONE (2026-07-22).** Three-visible allocation + flip (`_createSharedTexture`: three `StorageTexture` + raw handle, one + allocation), the `guides` accessor, `dispatchGuides`/`dispatchUpscale` + public split, `path: 'guides'`, `@experimental` tags. New example + `12-temporal-guides` (11 was taken) renders final + three geometry guides + from the split frame — the GPU acceptance harness. Gates met: Q0 captures + byte-identical vs M1 (24/24, incl. frame 119 and all debug views); example + 12 headless-verified (no validation errors, guides sampled as TSL nodes, + split-frame timings read out); guides-only path CDP-driven under a + validation error scope (clean across both ping-pong halves, API guards + throw, late products null). Evidence: `bench/results/raw/GUIDES-M1/capture-m2`. - **M3 — reactive merge** (§6). Gate: example 05 GPU re-verify; new `generateReactive` fingerprint; merged-mask capture test in the bench. - **M4 — TSL surface.** `temporalGuides(depth, velocity, camera)` node diff --git a/examples/12-temporal-guides/index.html b/examples/12-temporal-guides/index.html new file mode 100644 index 0000000..b641756 --- /dev/null +++ b/examples/12-temporal-guides/index.html @@ -0,0 +1,53 @@ + + + + + + 12 · Temporal Guides + + + +
+
disocclusion
+
dilated depth (eye-z)
+
final upscale
+
dilated motion (uv delta)
+ + + diff --git a/examples/12-temporal-guides/main.ts b/examples/12-temporal-guides/main.ts new file mode 100644 index 0000000..4cd9f92 --- /dev/null +++ b/examples/12-temporal-guides/main.ts @@ -0,0 +1,187 @@ +import * as THREE from 'three/webgpu'; +import { fract, mix, mrt, output, step, texture, uv, vec3, vec4, velocity } from 'three/tsl'; + +import { Upscaler } from '@pmndrs/upscaler'; + +import { bootRenderer, displaySize } from '../shared/boot'; +import { addStudioLighting, createGridFloor } from '../shared/props'; + +//* Temporal guides — the upscaler as a data-products provider. +// The frame is driven with the SPLIT dispatch: dispatchGuides() right after +// the G-buffer produces the geometry guides (dilated motion/depth, +// disocclusion) that other temporal effects (SSGI temporal passes, denoisers) +// can consume *before* the final color exists; dispatchUpscale() then +// finishes the frame. The 2×2 view samples the published guide textures +// straight from `upscaler.guides` as ordinary TSL texture() nodes — exactly +// the consumer contract (ping-ponged products are re-pointed per frame). + +const { renderer, dpr } = await bootRenderer(); + +//* Scene — moving knot + orbiting spheres over a grid floor, so motion, +//* disocclusion trails, and depth all have something to show. +const scene = new THREE.Scene(); +scene.background = new THREE.Color(0x10141a); +addStudioLighting(scene); +scene.add(createGridFloor()); + +const knot = new THREE.Mesh( + new THREE.TorusKnotGeometry(1.1, 0.34, 220, 28), + new THREE.MeshStandardMaterial({ color: 0xc0c8d8, metalness: 0.9, roughness: 0.22 }), +); +knot.position.y = 2; +scene.add(knot); + +const spheres: THREE.Mesh[] = []; +for (let i = 0; i < 3; i++) { + const sphere = new THREE.Mesh( + new THREE.SphereGeometry(0.5, 48, 24), + new THREE.MeshStandardMaterial({ color: 0xf59e0b, metalness: 0.2, roughness: 0.4 }), + ); + scene.add(sphere); + spheres.push(sphere); +} + +const camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 200); + +//* Upscaler — raw driver (no UpscalePass: we need the split dispatch). +const upscaler = new Upscaler({ renderer }); +upscaler.init(); +// Motion vectors must be jitter-free. +velocity.setProjectionMatrix(upscaler.unjitteredProjectionMatrix); + +const mrtFull = mrt({ output, velocity }); +let rt: THREE.RenderTarget | null = null; + +function configure(): void { + const { width, height } = displaySize(dpr); + upscaler.configure({ + displayWidth: width, + displayHeight: height, + customUpscaleRatio: 2, + path: 'temporal', + }); + + rt?.dispose(); + const depthTexture = new THREE.DepthTexture(upscaler.renderWidth, upscaler.renderHeight); + depthTexture.type = THREE.FloatType; + rt = new THREE.RenderTarget(upscaler.renderWidth, upscaler.renderHeight, { + count: 2, // MUST match the MRT output count + type: THREE.HalfFloatType, + depthTexture, + }); + // MRT routes node outputs to attachments BY TEXTURE NAME. + rt.textures[0].name = 'output'; + rt.textures[1].name = 'velocity'; +} +configure(); + +//* Present — one quad, 2×2 quadrants sampling the published guides. +// top-left: disocclusion top-right: dilated depth +// bottom-left: final bottom-right: dilated motion +const tile = uv().mul(2); +const local = fract(tile); +const right = step(1.0, tile.x); +const top = step(1.0, tile.y); + +// Texture nodes are created once; ping-ponged guides (dilated depth) get +// their `.value` re-pointed every frame — the documented consumer pattern. +const finalNode = texture(upscaler.outputTexture, local); +const motionNode = texture(upscaler.guides.dilatedMotion, local); +const disocclusionNode = texture(upscaler.guides.disocclusion, local); +const depthNode = texture(upscaler.guides.dilatedDepth, local); + +function refreshGuideNodes(): void { + const guides = upscaler.guides; + finalNode.value = upscaler.outputTexture; + motionNode.value = guides.dilatedMotion; + disocclusionNode.value = guides.disocclusion; + depthNode.value = guides.dilatedDepth; +} + +const motionVis = vec3(motionNode.xy.mul(40.0).add(0.5), 0.5); +const disocclusionVis = vec3(disocclusionNode.r); +// Linear eye-Z → a readable gradient (near bright, far dark). +const depthVis = vec3(depthNode.r.div(depthNode.r.add(8.0)).oneMinus()); + +// Note: the quad's uv v axis runs top-down on screen, so `top = 1` selects +// the LOWER half — order the rows accordingly. +const quadColor = mix( + mix(disocclusionVis, depthVis, right), // screen top row + mix(finalNode.rgb, motionVis, right), // screen bottom row + top, +); +const quadMaterial = new THREE.NodeMaterial(); +quadMaterial.colorNode = vec4(quadColor, 1.0); +quadMaterial.depthTest = false; +quadMaterial.depthWrite = false; +quadMaterial.fog = false; +const quad = new THREE.QuadMesh(quadMaterial); + +const badge = document.getElementById('badge')!; +function updateBadge(): void { + const reconstruct = upscaler.gpuTimings.get('reconstruct'); + badge.innerHTML = + `@pmndrs/upscaler temporal guides (split dispatch)\n` + + `render ${upscaler.renderWidth}×${upscaler.renderHeight}\n` + + `display ${upscaler.displayWidth}×${upscaler.displayHeight} (${upscaler.upscaleRatio.toFixed(1)}x)\n` + + `guides dispatched post-G-buffer, pre-color\n` + + (reconstruct !== undefined ? `reconstruct ${reconstruct.toFixed(3)} ms` : ''); +} + +window.addEventListener('resize', () => { + renderer.setSize(window.innerWidth, window.innerHeight); + camera.aspect = window.innerWidth / window.innerHeight; + camera.updateProjectionMatrix(); + configure(); + // configure() reallocates the working set — re-point every guide node. + refreshGuideNodes(); +}); + +// Exposed for the headless GPU-verification harness (drives the guides-only +// path against this page's live renderer + render target). +Object.assign(window as unknown as Record, { + __guidesExample: { upscaler, renderer, camera, Upscaler, getRenderTarget: () => rt }, +}); + +//* Loop — the split frame. +const timer = new THREE.Timer(); +renderer.setAnimationLoop(() => { + timer.update(); + const dt = Math.min(timer.getDelta(), 0.1); + const t = timer.getElapsed(); + + knot.rotation.y = t * 0.5; + knot.rotation.x = t * 0.35; + for (let i = 0; i < spheres.length; i++) { + const phase = t * 0.9 + (i * Math.PI * 2) / spheres.length; + spheres[i].position.set(Math.cos(phase) * 3.4, 1.1 + Math.sin(t * 1.3 + i) * 0.4, Math.sin(phase) * 3.4); + } + camera.position.set(Math.cos(t * 0.12) * 9, 4.5, Math.sin(t * 0.12) * 9); + camera.lookAt(0, 1.6, 0); + + //* 1. G-buffer: jittered scene render with color + velocity MRT. + upscaler.beginFrame(camera); + renderer.setMRT(mrtFull); + renderer.setRenderTarget(rt); + renderer.render(scene, camera); + renderer.setRenderTarget(null); + renderer.setMRT(null); + upscaler.endFrame(camera); + + //* 2. Early stage: geometry guides, valid from here on. + upscaler.dispatchGuides( + { depth: rt!.depthTexture!, velocity: rt!.textures[1], deltaTime: dt }, + camera, + ); + + // (A real pipeline runs its guide-consuming effects here — SSGI temporal + // reprojection, denoisers — then composites the final color.) + + //* 3. Late stage: accumulate + sharpen into outputTexture. + upscaler.dispatchUpscale({ color: rt!.textures[0], deltaTime: dt }, camera); + + //* 4. Present the quadrants (re-point ping-ponged guides first). + depthNode.value = upscaler.guides.dilatedDepth; + quad.render(renderer); + updateBadge(); +}); diff --git a/examples/index.html b/examples/index.html index 4789b3b..cd42e22 100644 --- a/examples/index.html +++ b/examples/index.html @@ -221,6 +221,17 @@

@pmndrs/upscaler — Examples

integration + +
12
+
Temporal guides
+
+ The upscaler as a data-products provider: the split + dispatchGuides() / dispatchUpscale() frame, with + the published dilated motion, disocclusion, and depth guides sampled live + as TSL texture nodes. +
+ integration +
+

1. Legacy RCAS vs lower limiter

+

Open blinded visual review

+ +${timingTable(limiterTimingRows)}
TimingThree A/B repetitionsMedianInterpretation
+

Pixel differences

+ +${captureTable(limiterCaptureRows)}
CaptureMaximumRMSE
+