feat: temporal guides + still-scene convergence fix - #1
Merged
Conversation
Fix five defects that prevented the authored source-parity bundles from ever compiling on a device (isInf, reserved keyword, r8unorm storage, unused sampler binding, /0.5 velocity falloff), fix the cross-variant analyzeAbba label crash so production-vs-candidate A/B runs work, and add --help to the benchmark runner. Measured (ratio 2, ABBA, noise floor <=2%): filter bundle +36%, structural +6.5% incremental, full SPD resolver +76% GPU compute vs production, with no visual regression or win on Q0/Q1/Q3 captures. No bundle adopted; evidence in bench/PARITY-DECISIONS.md and PARITY.md (new consumer-facing parity rationale). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PARITY.md stays at root as the consumer-facing rationale; the internal ledgers (PARITY-PROGRESS/CANDIDATES/DECISIONS, THREE-TEMPORAL-COMPARISON) move to bench/docs/. NEXT-STEPS.md captures the four surviving adoption items with per-item validation gates. CLAUDE.md points at the new layout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… DeltaPreExposure, AMD disocclusion Item 1 (RCAS cost investigation, resolved): the resolver candidate's "47% cheaper RCAS" was per-tap work, not value ranges — production paid a 1x1 exposure load + tonemapInvert division + exposure division per tap behind FLAG_INPUT_REINHARD. Production RCAS now sharpens the accumulate history's conditioned tonemap-space texels (the [0,1) range the limiter math assumes) and inverts the conditioning once on the result: RCAS 0.103 -> 0.068 ms (-34%), total pipeline -5.7% (warm ABBA blocks, ratio 2). Captures on Q0/Q1/Q3 + Q9 HDR stress: RMSE <= 1.8/255, HDR-bulb ROI <= 9/255 — visually identical, no overshoot. The per-tap form is frozen as RCAS_PER_TAP_SHADER under the rcas-fsr315-limiter / rcas-fsr315-numeric bench identities; both isolating timing variants (rcas-hoisted-exposure-v1, rcas-tonemap-space-v1) stay in the registry. Item 2 (host pre-exposure, DeltaPreExposure semantics): the pyramid publishes preExposureTexture's value in the exposure texel's .b and meters auto-exposure host-invariantly (GPU-found: without this the conditioning chases a host step for ~2s and the drift fires the shading-change detector full-screen on flat regions). Accumulate ratio-corrects reprojected history in linear space across a host change (new binding 11 = previous exposure texel). Validated on the new Q11 host-pre-exposure scenario (bench drives the scene MRT color and preExposureTexture together; E00 manifest extended): a 2.5x step + ramp leaves the detector at baseline, never resets accumulation age, and output brightness tracks the drive. Without the input the pass is byte-identical (PNG-compared pre/post). Item 3 (AMD disocclusion constant): the fused reconstruct pass replaces the DEPTH_SEPARATION_SCALE / DEPTH_SIMILARITY_FLOOR guesses with AMD's per-bilinear-tap confidence voting and viewport/depth-scaled tolerance (1.37e-5 * halfViewportWidth * maxDepth, ffx_fsr2_depth_clip.h via the GPU-verified candidate port). Q3: thin stable silhouette outlines, still scenes near-black, age resets confined to trails; finals RMSE <= 1.1/255; pass time unchanged. Fused single-pass structure kept (source's scatter + separate pass measured +30%/+22% for no visual win). GPU verification: all runs on Apple Metal via CDP; examples 01/05/07 smoke clean (no WGSL errors) on the default-constructed Upscaler. Docs updated (PARITY.md, shaders README fidelity table, NEXT-STEPS, CLAUDE.md). Only NEXT-STEPS item 4 (Phase-5 SPD shading-change detector) remains open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…PS 4) Replaces accumulate's inline 3x3-neighborhood shading heuristic with src/shaders/shadingChange.ts: one fused half-resolution dispatch whose 8x8 workgroup covers a 16x16 render tile, making the 4x4 and 8x8 block reductions workgroup-local — each luma comparison is evaluated once with hoisted frame-info loads, no mip textures, no separate resolve pass. The pass maintains a 1-frame luma history, reprojects it jitter-delta-aligned (bilinear), neutralizes disoccluded texels, and compares block-MEAN luma per scale gated by a base + contrast-scaled (coefficient of variation) noise floor. The strongest gated scale feeds accumulate's existing SHADING_AGE aging path (new binding 12; zero dummy when settings.detectShadingChanges is off — the pass simply isn't dispatched). Locks keep their self-referential break, untouched. Five GPU tuning iterations (evidence in bench/results/raw/E00/pre-spd-* and post-spd-v*): the source-style mean-of-per-texel-signed-ratios floors at ~0.10 still-scene response — the ratio metric weights the darker side of alias residue, a coherent bias signed averaging cannot cancel — and neither jitter alignment nor bilinear reprojection alone fixes it. Averaging luma BEFORE the ratio, adaptive floors, and dropping the flicker-dominated 2x2 scale hit the full acceptance matrix: - Q1 still scene: mean ~2/255 (old detector ~1.1) — near-black - Q4 camera orbit over high-frequency content: worst 3.6 vs old 4.9 — measurably FEWER false positives (the item's whole point) - Q9 light steps: clean single-frame spikes at 137/113 per 255 (old: 84 with a ~20-frame decay tail); slow ramps deliberately quiet (1-frame comparison; blend tracks ramps — Q9 ramp finals show no lag, RMSE 0.78) - Q11 host pre-exposure step: quiet (hostRatio correction inside the pass) - Finals within 1.6/255 RMSE of the old detector everywhere Cost: 0.044 ms at ratio 2 — 5x cheaper than the candidate's two-pass form (0.231 ms measured). Examples 01/07/09 GPU-smoke clean. Bench gains the 'shadingChange' timing label. SHADING_LO/HI are gone (the floors live in shadingChange.ts); SHADING_AGE stays in accumulate.ts. Docs updated: CLAUDE.md (roadmap item closed, landmine + debug-protocol entries rewritten), shaders README (adopted section with the rejected designs and why), NEXT-STEPS item 4, PARITY.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e status everywhere - PARITY.md: expanded into the shareable design-notes document with a new 'What we changed — enhancements beyond a port' section (fused reconstruction, conditioned-space RCAS, the multi-scale shading-change detector incl. the block-mean-ratio bias finding, host-invariant exposure), each with upstream behavior, our form, and measurements. Stale 'open items' section closed. - README: 'Phases' table replaced by a current-state Status section; PARITY.md now linked from How-it-works and Status. - bench/docs: PARITY-PROGRESS.md (concluded program's process scaffolding) deleted, its load-bearing facts folded into PARITY-DECISIONS.md; NEXT-STEPS.md retitled as a closed adoption record (header wrongly said item 4 still open); PARITY-CANDIDATES.md gained the program-conclusion note; DENOISING-DIRECTION.md moved into bench/docs/. - CLAUDE.md: phase-ledger roadmap restructured into 'Feature status' + 'Deferred / out of scope', keeping every mechanism note and trap. - Code comments: stale phase references removed; accumulate.ts and luminancePyramid.ts headers corrected (they described shipped features as future work). No WGSL strings touched — shader fingerprints unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… (SSGI/SVGF) Maps bench/docs/FSR3-BRIEF.md onto the actual pipeline: publish the existing internal textures as a contracted guides bundle, split dispatch into an early geometry stage and a late luma stage, add bidirectional reactive merge, and a standalone signal-agnostic MomentPyramid. Additive only; existing dispatch stays byte- and perf-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hange) _encodeTemporal now composes _encodeGuides (reconstruct only — the signal-agnostic geometry stage) and _encodeLate (everything needing beauty color) on one encoder, so the monolithic dispatch keeps its single submit. Seam for the split-dispatch guides API (TEMPORAL-GUIDES-SPEC M2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mental) - upscaler.guides (TemporalGuides): the production working set as three StorageTextures — dilated motion/depth, disocclusion, reactive, shading change, exposure, locks, history — with ping-pongs resolved to the most recently written half. - dispatchGuides()/dispatchUpscale(): the split frame, so geometry guides exist post-G-buffer, pre-color; path 'guides' runs the early stage alone. - GpuTimer merges per-label results (a split frame is two submits). - examples/12-temporal-guides: split-dispatch demo + headless GPU harness. GPU gates: Q0 captures byte-identical vs pre-publish (24/24), example 12 and the guides-only path validation-clean headlessly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cross-frame gather comparison (our deliberate cheaper form of upstream's same-frame scatter) shipped with three instabilities, visible as per-jitter- phase full disocclusion flicker on a distant ground plane (found via the temporal-guides example): a running-AND tap veto that upstream doesn't have (any tap at/behind the current surface zeroed the pixel, order-dependently), no jitter-delta compensation in the reprojection, and a fixed tolerance that reads a steep slope's per-texel depth change as surface separation. Fix: reference per-tap skip semantics, jitter-delta-compensated prevUV (same derivation as shadingChange), and the separation tolerance widened by the 3x3 dilation ring's own depth relief (free, geometry-derived, no tuned constants). Measured: example-12 disocclusion quadrant frame-to-frame flips drop from 12-14% of pixels to ~1.5% (moving-content baseline); Q3 fence trails and silhouette outlines unchanged; Q0 accumulation age rises slightly (history surviving where it was falsely reset); final-image delta RMSE ~1.2/255. Candidate bundle shaders keep the old form (frozen bench identities). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The generator gains an incoming-mask binding and max-merges it with the opaque-diff response instead of being suppressed by an explicit mask; the published guides.reactive target is the merged result, and effects may write reactivity into it between the split dispatches. Aliasing guard: passing guides.reactive back as `reactive` while reactiveOpaqueColor is set throws (the generator writes that texture). Filter-bundle candidate path binds the inert dummy; structural bundle's source-policy generator is untouched. GPU-verified on example 05 (manual / auto / off modes, no validation errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…des M5) Per-pixel (E[x], E[x²]) of a configurable scalar (Rec.709 linear luma or YCoCg Y, runtime flag in the pass's own constants buffer) over any float texture, plus one coarse level of 4x4 block means — SVGF's statistics half, decoupled from every beauty/exposure assumption. One fused 8x8 dispatch, workgroup-local reduction; rgba16float outputs (.rg used; rg16float is not a core WebGPU storage format). Not wired into the upscaling pipeline at all. GPU-verified against a CPU reference on a seeded 32x32 DataTexture in BOTH spaces: validation-clean, max relative error <0.1% (f16 tolerance 1%), coarse variance non-negative. Example 12 exposes MomentsPass + THREE for the headless harness. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…priority Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed-build config GUIDES-HANDOFF-RESPONSE.md (demo-16, first full-pipeline consumer) landed: linked build + M2 split-dispatch contract accepted and verified live on cornell/sponza, nothing blocked. Two friction items, both ours: - UpscalerNode._renderer was assigned in setup() and never read — deleted (it broke the consumer's noUnusedLocals when aliasing at src/). - The handoff's "no duplicate-three hazard" claim was wrong for out-of-root consumers: two node_modules → two three cores. Folded their verified vite block (three→three/webgpu collapse, dedupe, optimizeDeps exclude) into GUIDES-HANDOFF.md and repointed the tsconfig-paths guidance at dist/index.d.ts, with the keep-dist-current commitment. dist/ rebuilt so the consumer's declaration-based typecheck matches src. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
temporalGuides(depth, velocity, camera) publishes the temporal-guides
products into a THREE.PostProcessing graph as texture nodes
(getTextureNode(name): stable identity, ping-ponged products re-pointed
per frame, nearest 1x1 placeholder pre-configure so r32float format
inference never sees a filterable stand-in). Two modes, decided by wiring:
- standalone: the node owns a guides-only Upscaler sized to its depth
input; late products are null and consuming one warns once with the
linked-mode pointer.
- linked: upscale(..., { guides }) adopts the node's upscaler
(_acquireUpscaler) and the split frame runs in-graph — guides dispatch
right after the G-buffer, effects consume the products, the upscale
node finishes with dispatchUpscale, falling back to the monolithic
dispatch on frames where the early stage couldn't run (inputs not yet
GPU-backed, mid-frame reconfigure). New Upscaler.guidesPending getter
is the branch point.
Gates (spec M4): examples 07/09 GPU-verified unchanged; new
examples/13-guides-node consumes disocclusion in a toy effect (orange
trailing-silhouette tint, pre-upscale) — dispatch-spy probe shows the
pure split path steady-state (120 guides + 120 late, 0 monolithic over
1s) on one shared upscaler; standalone mode CDP-driven under a
validation error scope (clean, early products live, late null + warning).
172 tests, typecheck, lint, build green; examples/README.md table brought
current (07-13); dist rebuilt for the consumer's declaration typecheck.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reproduced the ssgiDev demo-16 non-convergence in our own bench (Q1 still camera: 0.211 consecutive-frame meanAbsDiff sustained, 0.182 at the SAME jitter phase one period apart — aperiodic history churn, not the benign per-phase pattern). Three stacked core defects: 1. reconstruct: the depth-clip vote only counted positive-separation taps, so one bilinear tap straddling the previous frame's texel-quantized dilation boundary re-disoccluded every still silhouette per phase. Now every valid tap votes (agreement = full confidence) and the best tap wins; genuine trails still read ~1. 2. accumulate: clip-magnitude aging of the sample count made convergence unreachable at any contrasty edge (rolling accumulation-age moiré at every ratio incl. NativeAA). Removed — FSR2 never ages on rectification strength; the clip + shading detector own stale shading. 3. accumulate: the blend stores the CLIPPED history, so each jitter phase's variance box re-snapped the buffer regardless of alpha (phase-locked 0.183 with 1+2 fixed; 0.005 with the clip off). New STILL_CLAMP_RELAX: still + converged + quiet pixels get a ×9-widened box; motion, disocclusion, shading change, or reactivity restores full rectification. Shipped ladder (Q1 2x, consecutive/phase-locked): 0.211/0.182 → 0.112/0.018. NativeAA 0.081/0.003. New Q12 cornell-still-convergence scenario (enclosed box + IGN-dithered Vogel point-light shadows, the consumer's aggravator + pose): 0.024/0.012, disocclusion view fully black, age saturated. Q3/Q4 motion scenarios verified regression-free on GPU. New headless churn meter: scripts/measure-convergence.mjs (consecutive + same-phase diffs, debug-view PNGs). Evidence: bench/docs/NEXT-STEPS.md §5; PAPER-NOTES §6; consumer reply appended to GUIDES-HANDOFF-RESPONSE.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e raw surface The consumer's cross-repo A/B (report 2) passed: their SSGI temporal stack fed by our bundle measured bit-identical still-camera stability against their private front-end, 1.275 s vs 1.288 s teleport reconvergence, and MomentsPass was separately field-verified in their SVGF demo. That was the spec's own exit criterion for the tag. Dropped from `Upscaler.guides` / `guidesPending` / `dispatchGuides` / `dispatchUpscale`, the `TemporalGuides` type, and `MomentsPass`. Kept on `TemporalGuidesNode` / `temporalGuides` / `UpscalerNode`'s `guides` option, with the reason narrowed in the doc: the contract those expose is the accepted one, but the node's graph plumbing (node identity, per-frame re-pointing, linked-vs-standalone wiring) has only our own GPU verification (examples/13-guides-node) and no external consumer, so its ergonomics may still shift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Jul 29, 2026
Member
Author
|
Follow-ups from this branch are now tracked as issues rather than left in the description:
None block merging. |
The shader tests import bench modules, which import the package by its own name. Node's self-reference resolves that through package.json `exports` → `dist/`, so the test run depended on build output that CI hasn't produced yet — `npm test` runs before `npm run build`. It passed locally only because a stale dist/ was lying around. Alias it to src/index.ts in the vitest config, matching what bench/vite.config.ts already does. Verified by deleting dist/ and running the CI sequence in order: lint, typecheck, test (172), build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Temporal guides program (M1–M6) + the post-parity core work, plus a still-scene
convergence fix found by the program's first full-pipeline consumer.
Merging this triggers
publish.ymlpath B — the branch carriesfeat:commitssince
v0.1.0, so it auto-bumps and publishes 0.2.0.Temporal guides — a new public surface
Dilated motion, dilated depth, and disocclusion are frame properties, not
upscaler properties: every temporal effect upstream (SSGI/SSR reprojection,
SVGF-class denoisers, TAA) re-derives worse versions privately. The upscaler now
publishes its internal working set as
upscaler.guides(ordinary threetextures), and the frame can be driven split so the geometry guides exist
before the final color does:
path: 'guides'runs the early stage alone, for apps that never upscale.max)with the auto-generated one; effects can write into
guides.reactivemid-frame.MomentsPass— standalone, signal-agnostic(E[x], E[x²])over any floattexture plus one coarse level, with zero coupling to the upscaling pipeline.
temporalGuides(depth, velocity, camera)publishes the same bundle as TSLtexture nodes;
upscale(..., { guides })shares one computation across the graph.TemporalGuidestype and inTEMPORAL-GUIDES-SPEC.md. Live references:examples/12-temporal-guides(raw),examples/13-guides-node(TSL).Cross-repo acceptance (M6): PASS. An external SSGI/SVGF consumer ran its
identical temporal stack against its private guides pass vs this bundle:
still-camera stability bit-identical, teleport reconvergence 1.275 s vs
1.288 s, across-arm diff below within-arm noise. Their verdict — drop-in
replacement.
@experimentalis therefore off the raw surface andMomentsPass;it stays on the TSL node only, whose contract is the accepted one but whose
graph plumbing has no external consumer yet.
The monolithic
dispatch()is unchanged in behavior: byte-identical capturesand perf-neutral (−2.7%, within noise) against the pre-split pipeline.
Core fixes
Still-scene convergence (consumer report 3, confirmed ours). On a still
camera the output never converged — flickering
Disocclusionsilhouettes and aforever-rolling
AccumulationAge, at every ratio including NativeAA. Reproducedin our own bench with no consumer code. The decisive metric was the frame diff
between two frames at the same jitter phase one period apart, where benign
per-phase shimmer nets ~0 and we measured 0.182. Three stacked defects:
reconstruct.ts) — only taps in front of thecurrent surface could vote or carry weight, so one bilinear tap straddling
last frame's texel-quantized dilated-depth boundary became the sole voter and
disoccluded still edges every phase. Now every valid tap votes and the best
tap wins; genuine trails still read ~1.
accumulate.ts, removed) — aging the samplecount by clip strength pinned equilibrium age low at any contrasty edge, so
alpha never shrank. FSR2 never does this.
accumulate.ts) — the blend stores the clippedhistory, so each phase's variance box re-snapped the buffer regardless of
alpha. Fixed with
STILL_CLAMP_RELAX, gated on stillness × convergence ×absence of disocclusion/shading-change/reactivity.
Q1 2×: 0.211 → 0.112 consecutive, 0.182 → 0.018 phase-locked. NativeAA:
0.081 / 0.003. New Q12 cornell scenario: 0.024, disocclusion view black, age
saturated. Motion scenarios (Q3/Q4) verified regression-free.
Grazing-angle disocclusion flicker — jitter-delta-compensated reprojection
plus a tolerance widened by the dilation ring's own depth relief.
Post-parity adoptions (all measured)
−5.7% total, capture-identical.
preExposureTexture) honored end-to-end: DeltaPreExposurehistory correction + host-invariant metering. Byte-identical when absent.
shadingChange.ts) replacing the 3×3heuristic — 0.044 ms, 5× cheaper than the source-style candidate, measurably
fewer false positives under motion.
Tooling
scripts/measure-convergence.mjs— headless CDP convergence meter(consecutive + phase-locked frame diffs, debug-view PNGs).
Verification
npm test(172 passed) ·npm run typecheck·npm run lint·npm run buildall clean. GPU verification is recorded per item in
bench/docs/NEXT-STEPS.mdand
TEMPORAL-GUIDES-SPEC.md; consumer-side evidence inGUIDES-HANDOFF-RESPONSE.md.Known follow-up (not blocking)
src/shaders/candidate*.ts— ~55 kB of parity-research shader source — isstatically imported by
Upscaler.tsand so ships indist. Bench-only, gatedbehind
_candidateBundle, and not tree-shakeable as wired. Worth moving behinda bench-only entry point in a later release; deliberately not touched here,
since it means editing a core path right before a publish.
🤖 Generated with Claude Code