Layout is measured on both layers. L1 covers it through DOM.getBoxModel and Page.getLayoutMetrics across the driver stacks, and L2 covers it through the layout capabilities in #19. Raster is measured on neither. The screenshot and PDF commands are held out of scope, and the validator rejects their feature tags, so no task can cover them.
Raster is the stage after layout. Layout gives every box a geometry, and raster turns those boxes into pixels. An engine can have the first without the second, which makes them separate targets, and only the first one is a target here.
Why it belongs in the benchmark
A rollout agent that observes through screenshots reads pixels, not the DOM. For a vision-based agent the screenshot is the observation channel itself, so an engine that cannot rasterise is unusable no matter how complete its protocol surface is. Nothing in the current task set would reveal that.
Why this is not just a matter of writing tasks
Pixel equality against Chrome is not available. Font hinting, anti-aliasing, subpixel positioning and colour management all differ between engines, and they differ between Chrome versions too. So the first question is what counts as the correct answer for an image. Two families of answer are on the table, and they cost different things.
The first is a coarse, quantised fact about the image: the colour at a named coordinate, the bounding box of a solid-colour region, the output dimensions, the device pixel ratio, whether a clip region was honoured. This fits the existing expected-answer model, where the fixture server holds the answer and the probe sends back a short string, so it needs no new grader.
The second is a perceptual diff against a reference image with a threshold. This introduces a tolerance value that somebody has to defend, in a repository where every other result is exact and hashed.
Layout is measured on both layers. L1 covers it through
DOM.getBoxModelandPage.getLayoutMetricsacross the driver stacks, and L2 covers it through the layout capabilities in #19. Raster is measured on neither. The screenshot and PDF commands are held out of scope, and the validator rejects their feature tags, so no task can cover them.Raster is the stage after layout. Layout gives every box a geometry, and raster turns those boxes into pixels. An engine can have the first without the second, which makes them separate targets, and only the first one is a target here.
Why it belongs in the benchmark
A rollout agent that observes through screenshots reads pixels, not the DOM. For a vision-based agent the screenshot is the observation channel itself, so an engine that cannot rasterise is unusable no matter how complete its protocol surface is. Nothing in the current task set would reveal that.
Why this is not just a matter of writing tasks
Pixel equality against Chrome is not available. Font hinting, anti-aliasing, subpixel positioning and colour management all differ between engines, and they differ between Chrome versions too. So the first question is what counts as the correct answer for an image. Two families of answer are on the table, and they cost different things.
The first is a coarse, quantised fact about the image: the colour at a named coordinate, the bounding box of a solid-colour region, the output dimensions, the device pixel ratio, whether a clip region was honoured. This fits the existing expected-answer model, where the fixture server holds the answer and the probe sends back a short string, so it needs no new grader.
The second is a perceptual diff against a reference image with a threshold. This introduces a tolerance value that somebody has to defend, in a repository where every other result is exact and hashed.