Skip to content

Commit a96419f

Browse files
ukimsanovukimsanov
committed
feat(cli): add agent-first media treatment tools
1 parent f9bf4d0 commit a96419f

29 files changed

Lines changed: 3309 additions & 75 deletions

‎docs/concepts/variables.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,8 +87,8 @@ the shader grading, finishing details, blur/pixelate effects, and optional LUT:
8787

8888
```html compositions/hero.html
8989
<html data-composition-variables='[
90-
{"id":"gradingPreset","type":"enum","label":"Preset","default":"natural-lift",
91-
"options":[{"value":"natural-lift","label":"Natural Lift"},{"value":"warm-daylight","label":"Warm Daylight"}]},
90+
{"id":"gradingPreset","type":"enum","label":"Preset","default":"clean-studio",
91+
"options":[{"value":"clean-studio","label":"Clean Studio"},{"value":"warm-daylight","label":"Warm Daylight"}]},
9292
{"id":"gradingIntensity","type":"number","label":"Preset strength","default":0.75,"min":0,"max":1,"step":0.05},
9393
{"id":"gradingExposure","type":"number","label":"Exposure","default":0,"min":-2,"max":2,"step":0.05},
9494
{"id":"gradingVibrance","type":"number","label":"Vibrance","default":0.08,"min":-1,"max":1,"step":0.01}

‎docs/guides/color-grading.mdx‎

Lines changed: 161 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: Color Grading
3-
description: "Apply real-time color grading, presets, LUTs, vignette, grain, blur, and pixelate to video and image media in Studio and final renders."
3+
description: "Apply real-time color grading, presets, LUTs, finishing, and media effects to video and image media in Studio and final renders."
44
---
55

66
HyperFrames Studio can color grade project-local `<video>` and `<img>` media directly in the preview. The same `data-color-grading` settings are used by the render pipeline, so the exported video should match the look you preview.
@@ -15,8 +15,9 @@ This is a lightweight media color tool for generated videos, uploaded footage, s
1515
| Manual controls | Supported | Exposure, contrast, highlights, shadows, white point, black point, warmth, tint, vibrance, saturation. |
1616
| Presets | Supported | Named HyperFrames presets backed by shader settings, not bundled third-party LUT packs. |
1717
| Custom LUT upload | Supported | Project-local 3D `.cube` LUT files with strength control. |
18-
| Finishing | Supported | Vignette and grain, with advanced settings behind the settings icon. |
19-
| Effects | Supported | Blur and pixelate on the selected media surface. |
18+
| Finish | Supported | Vignette and grain, with advanced settings behind the settings icon. |
19+
| Studio Effects panel | Supported | Essentials, Retro & Glitch, Print, and Art effects share the same media shader and persisted `effects` object. |
20+
| Studio Overlays panel | Supported | Inserts selected Registry overlays at the media timing as ordinary timeline layers. |
2021
| Before preview | Supported | Hold the compare button to temporarily show the ungraded media. |
2122
| Render parity | Supported | The render pipeline redraws the color-grading shader after video-frame injection. |
2223

@@ -36,6 +37,7 @@ Studio labels `whites`, `blacks`, and `temperature` as White Point, Black Point,
3637
| Rec.709 creative LUTs | Yes | Best LUT path today. Use project-local 3D `.cube` files. |
3738
| Camera conversion LUTs | Partial | Technically accepted if they are supported 3D `.cube` files, but correctness depends on the source footage matching the LUT's expected input color space. |
3839
| Full-scene grading including text/DOM | Not yet | Color Grading is media-only. Captions, text, SVG, and regular DOM overlays stay unchanged. |
40+
| Face/region-only grading or privacy | Not yet | Realtime effects process the whole selected `<img>` or `<video>`. Isolate the region into a separate cropped/masked media layer or use an external segmentation/tracking tool first. |
3941
| Remote media URLs | Partial | WebGL pixel processing requires compatible CORS headers. Project-local assets are the reliable path. |
4042
| Professional ACES/OCIO/HDR finishing | Not yet | Future render/color-management work, not this Studio shader path. |
4143

@@ -70,15 +72,51 @@ Color grading is stored on media elements as `data-color-grading`:
7072
"grainRoughness":0.55
7173
},
7274
"effects":{
73-
"blur":0.08
75+
"blur":0.08,
76+
"chromaBleed":0,
77+
"tapeDamage":0,
78+
"tapeTracking":0,
79+
"tapeNoise":1,
80+
"tapeSpeed":0.5,
81+
"filmArtifacts":0,
82+
"halftone":0,
83+
"halftoneSize":0,
84+
"twoInkPrint":0,
85+
"twoInkPrintSize":0,
86+
"ascii":0,
87+
"asciiSize":0.066,
88+
"asciiInvert":0,
89+
"dither":0,
90+
"ditherSize":0.25
7491
},
92+
"palette":["#0b0d0d","#eee9db"],
7593
"colorSpace":"rec709"
7694
}'
7795
></video>
7896
```
7997

8098
The runtime creates a sibling WebGL canvas for the media element, samples the current video or image frame, applies shader uniforms, then hides the native media only after a shader frame is ready.
8199

100+
## Studio Workflow
101+
102+
Select a real `<img>` or `<video>` element in Studio to open the media tools:
103+
104+
1. **Presets** shows every built-in starting point in one two-column preview
105+
grid. Hover to preview it on the selected media, click to apply it, or choose
106+
**Original** to return to the neutral preset.
107+
2. **Adjust** provides tonal and color correction.
108+
3. **Effects** groups configurable shader controls under Essentials, Retro &
109+
Glitch, Print, and Art.
110+
4. **Finish** provides vignette and grain.
111+
5. **Custom LUT** accepts a project-local 3D `.cube` file.
112+
6. **Overlays** installs Camcorder HUD, Editorial Flash, Organic Light Leak,
113+
or Freeze-Frame Cutout from the existing Registry.
114+
115+
An overlay inserted from this panel starts with the selected media, uses its
116+
duration, and is placed on the next visual track when one is known. It then
117+
behaves like any other composition layer: edit or remove it through Layers and
118+
Timeline. The left Catalog remains the browse-all Registry surface.
119+
82120
<Note>
83121
Color Grading is intentionally **media-only**. It applies to `<video>` and `<img>` sources. Captions, text, divs, SVG, and UI graphics remain ungraded unless you render them into media first.
84122
</Note>
@@ -91,7 +129,7 @@ Project-local media is the safest path. Remote media must be served with compati
91129

92130
```json
93131
{
94-
"preset": "natural-lift",
132+
"preset": "clean-studio",
95133
"intensity": 1,
96134
"adjust": {
97135
"exposure": 0,
@@ -116,8 +154,24 @@ Project-local media is the safest path. Remote media must be served with compati
116154
},
117155
"effects": {
118156
"blur": 0,
119-
"pixelate": 0
157+
"pixelate": 0,
158+
"chromaBleed": 0,
159+
"tapeDamage": 0,
160+
"tapeTracking": 0,
161+
"tapeNoise": 1,
162+
"tapeSpeed": 0.5,
163+
"filmArtifacts": 0,
164+
"halftone": 0,
165+
"halftoneSize": 0,
166+
"twoInkPrint": 0,
167+
"twoInkPrintSize": 0,
168+
"ascii": 0,
169+
"asciiSize": 0.066,
170+
"asciiInvert": 0,
171+
"dither": 0,
172+
"ditherSize": 0.25
120173
},
174+
"palette": ["#0b0d0d", "#eee9db"],
121175
"lut": {
122176
"src": "assets/luts/look.cube",
123177
"intensity": 0.75
@@ -128,6 +182,88 @@ Project-local media is the safest path. Remote media must be served with compati
128182

129183
Omit `enabled`; the presence of `data-color-grading` implies that grading is active. All numeric controls are clamped by the runtime. The current color grading path is Rec.709/sRGB-oriented and assumes browser-decoded media frames.
130184

185+
`chromaBleed` is a bounded `0` to `1` treatment primitive that horizontally
186+
softens chroma detail while preserving the center sample's luma. It is useful
187+
inside restrained creator-camera/camcorder treatments; it is not VHS, CRT, RGB
188+
split, tracking noise, or a complete camera emulation.
189+
190+
Most effect amounts and settings are normalized from `0` to `1`. Bloom supports
191+
up to `3`, bloom radius uses pixels, and enum controls use the integer choices
192+
shown in Studio:
193+
194+
- `tapeDamage` adds deterministic horizontal time-base instability, lower luma
195+
bandwidth, restrained ghosting, noise, sparse dropouts, and bottom-edge head
196+
switching. `tapeTracking` adds bounded moving horizontal tracking tears,
197+
`tapeNoise` scales tape noise and row jitter, and `tapeSpeed` controls their
198+
deterministic motion (`0.5` is normal speed). These three subordinate
199+
controls do nothing without `tapeDamage`. A complete analog-tape treatment
200+
may also use restrained `chromaBleed`, scanlines, RGB separation, and rare
201+
row tears; none of these controls adds CRT curvature.
202+
- `filmArtifacts` adds deterministic sparse dust and short scratches. Combine
203+
it with existing grain, vignette, color, and seek-safe GSAP gate weave for an
204+
8mm treatment. It does not change the frame by itself when set to `0`.
205+
- `halftone` blends in a four-angle CMYK print screen. `halftoneSize` controls
206+
its resolution-aware dot-cell size from fine to coarse. The channel angles
207+
and edge response use fixed print-oriented defaults to keep authored output
208+
consistent across agents.
209+
- `twoInkPrint` maps warm midtones and deep/cool shadows to fixed original
210+
vermilion and teal spot screens with a dark overprint on warm paper.
211+
`twoInkPrintSize` controls its resolution-aware screen size. Do not combine
212+
it with `halftone` or describe it as a named commercial print process.
213+
- `ascii` converts the selected media to a procedural 5x7 glyph field.
214+
`asciiSize` controls cell size and `asciiInvert` switches the light/dark ink
215+
polarity. It uses the first and last colors from `palette`.
216+
- `dither` applies a temporally stable ordered 4x4 Bayer dither.
217+
`ditherSize` controls cell size and all `palette` colors participate in the
218+
result. This is ordered dithering, not Floyd-Steinberg or another sequential
219+
error-diffusion algorithm.
220+
- `bloom` extracts bright pixels and runs a bounded half-resolution separable
221+
blur. `bloomRadius` controls its radius.
222+
- `monoScreen` provides configurable mono print shapes, angle, spread, invert,
223+
and palette controls.
224+
- `scanlines`, `crtCurvature`, and `chromaticAberration` provide display
225+
geometry and channel treatments with their related settings.
226+
- `digitalGlitch` provides deterministic line tears, blocks, displacement,
227+
selective pixelation, channel split, opacity, and speed controls.
228+
- `engraving`, `crosshatch`, and `kuwahara` provide stylized art treatments with
229+
their calibrated settings. Kuwahara uses bounded half-float intermediate
230+
targets when the browser supports them and otherwise reports unavailable.
231+
232+
`palette` accepts two to six exact `#RRGGBB` colors in authored order. Use
233+
dark-to-light order for normal luminance mapping; intentionally reverse the
234+
array for an inverted result. The runtime validates and lowercases colors but
235+
does not reorder them. ASCII, dither, mono screen, engraving, and crosshatch use
236+
it; a palette or subordinate setting alone does not allocate an effect. Omit it
237+
for the family default.
238+
239+
When effects are combined, HyperFrames evaluates them in one fixed,
240+
deterministic order: source framing and multipass blur/Kuwahara preparation;
241+
chromatic and digital glitch; primary color grading and LUT blended by global
242+
intensity; grain and film
243+
artifacts; mono/engraving/crosshatch/halftone/two-ink/dither/ASCII; bloom,
244+
scanlines, vignette, and CRT display masking; then before/after comparison.
245+
Effects cannot currently be reordered. The fixed
246+
pipeline keeps Studio, playback, seeking, and final render behavior
247+
predictable.
248+
249+
Studio exposes these controls in the selected media element's **Effects**
250+
accordion. Agents should choose one primary intent through the `media-use`
251+
skill, then either seed from a source-aware treatment recipe or inspect the
252+
canonical toolbox and assemble a bespoke combination:
253+
254+
```bash
255+
hyperframes media-treatment --capabilities --json
256+
hyperframes media-treatment --capabilities --capability kuwahara --json
257+
```
258+
259+
The first command reports a concise overview of every capability family. The
260+
focused query reports one family's or effect's controls, calibrated apply
261+
payload, render lane, palette support, and seek-safe animation paths. Use
262+
`--all` only for exhaustive tooling. Recipes are tested shortcuts, not the
263+
complete allowed surface. Persist the final combined payload with the same
264+
`hyperframes media-treatment` command; unknown keys are rejected before the
265+
composition is changed.
266+
131267
## Custom LUTs
132268

133269
HyperFrames supports project-local 3D `.cube` LUT files:
@@ -157,6 +293,15 @@ Color Grading is part of the media runtime, so render uses the same settings as
157293

158294
For 4K output, use the existing [4K Rendering](/guides/4k-rendering) workflow. Color Grading can run at 4K when the composition/render surface is 4K, but a 1080p source video does not become sharper just because the final render is 4K.
159295

296+
Performance follows the total number of treated pixels and the selected render
297+
lane, not only the number of media elements. Several tiled videos can be
298+
cheaper than several overlapping full-frame videos. Blur, Bloom, and Kuwahara
299+
use multipass rendering. When more than two full-frame multipass-treated media
300+
elements are visible together, verify continuous playback on the target
301+
machine and simplify or pre-render the stack if frames drop. HyperFrames does
302+
not impose a universal hard cap because GPU and decoder capacity varies by
303+
device.
304+
160305
For HDR output, use the existing [HDR Rendering](/guides/hdr) workflow. Color Grading currently warns on detected HDR media, but the grading controls themselves are not HDR-aware.
161306

162307
When grading a video, animate opacity on a wrapper element instead of directly on the `<video>` element. The runtime hides the native media and draws the graded result through a sibling canvas, so wrapper opacity preserves preview/render parity.
@@ -168,7 +313,16 @@ When grading a video, animate opacity on a wrapper element instead of directly o
168313
| Make uploaded footage look cleaner | Color Grading preset + adjust controls |
169314
| Use a look from another editor | Custom 3D `.cube` LUT |
170315
| Add polish to a product shot | Vignette, subtle grain, contrast, vibrance |
171-
| Blur or pixelate selected media | Effects inside Color Grading |
316+
| Blur or pixelate selected media | Effects panel |
317+
| Add restrained camcorder chroma softness | Effects panel or `effects.chromaBleed` through an agent |
318+
| Build an analog VHS treatment | `effects.tapeDamage` + bounded tracking/noise/speed + restrained chroma, scanline, row-tear, grain, and color settings |
319+
| Build an 8mm home-movie treatment | `effects.filmArtifacts` + grain/vignette/color + seek-safe GSAP weave |
320+
| Build a print/editorial treatment | `effects.halftone` + `effects.halftoneSize` |
321+
| Build a two-spot editorial print | `effects.twoInkPrint` + `effects.twoInkPrintSize` |
322+
| Build a terminal/editorial character treatment | `effects.ascii` + `effects.asciiSize` + an optional two-color `palette` |
323+
| Build a restrained multi-color pixel treatment | `effects.dither` + `effects.ditherSize` + a two-to-six-color `palette` |
324+
| Add an organic light leak | Install the finite `organic-light-leak-overlay` Registry block |
325+
| Build a freeze-frame cutout | Existing background removal + the `freeze-frame-cutout` Registry overlay block + host GSAP |
172326
| Make a presenter float over graphics | Existing [Remove Background](/guides/remove-background) workflow |
173327
| Put text behind a presenter | Existing `remove-background --background-output` workflow |
174328
| Render HDR delivery files | Existing [HDR Rendering](/guides/hdr) workflow |

‎packages/cli/src/capture/agentPromptGenerator.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,10 @@ Source: ${url}
200200
201201
To create a video from this capture, use the \`product-launch-video\` skill.
202202
203+
For later changes to how real footage or images look or reveal, load \`media-use\`
204+
and its media-treatment policy before editing; do not recreate canonical
205+
treatments with CSS/SVG filters or overlays.
206+
203207
## What's in This Capture
204208
205209
| File | Contents |

‎packages/cli/src/cli.commands.test.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,12 @@ describe("CLI command registration", () => {
3535
);
3636
});
3737

38+
it("registers media-treatment as the only treatment authoring command", () => {
39+
const loaders = commandLoaderBlock();
40+
expect(loaders).toContain('"media-treatment"');
41+
expect(loaders).not.toContain('"color-grading"');
42+
});
43+
3844
// A command actively reconciling skills (`skills check`/`skills update`)
3945
// must not also nudge the user to go reconcile skills — that nudge is
4046
// either redundant (it just ran) or misleading (a stale cached count from

‎packages/cli/src/cli.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,8 @@ const commandLoaders = {
152152
events: () => import("./commands/events.js").then((m) => m.default),
153153
validate: () => import("./commands/validate.js").then((m) => m.default),
154154
snapshot: () => import("./commands/snapshot.js").then((m) => m.default),
155+
"media-treatment": () =>
156+
import("./commands/color-grading.js").then((m) => m.mediaTreatmentCommand),
155157
"grade-compare": () => import("./commands/grade-compare.js").then((m) => m.default),
156158
compare: () => import("./commands/compare.js").then((m) => m.default),
157159
capture: () => import("./commands/capture.js").then((m) => m.default),

0 commit comments

Comments
 (0)