You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
skips the cascade when the interface is unchanged. This is default behaviour
335
335
and requires no configuration. The `mcpp (old)` column measures the previous
336
336
release at 81.72s, level with cmake, so the effect is new in this revision.
337
-
***`edit-body` is the control case**, and it is the row where the cascade is
338
-
genuinely owed: mcpp is 1.1x rather than 200x, and an engine faster here would
339
-
be omitting work. `+opt` does not omit it either — it performs the same work
340
-
2.9x faster. Worth stating precisely: the perturbation **inserts a line**, and
341
-
under GCC that shifts the recorded source location of every declaration after
342
-
it, which changes the BMI. The cascade follows from the changed BMI, not from
343
-
the edited body — measured in
337
+
***`edit-body` measures the case where the cascade is genuinely owed** — and
338
+
whether an edit owes one depends on where the body lives:
339
+
340
+
| the function body is in… | editing it | this row |
341
+
|---|---|---|
342
+
| a `.cppm`, and the edit **moves lines**| GCC records declaration positions, so the BMI changes → cascade owed |**what is measured: 1.1x, and 2.9x with `+opt`**|
343
+
| a `.cppm`, edited **in place** (same line count) | GCC does not serialise non-template bodies → BMI unchanged → no cascade |~200x, like `touch-hub`|
344
+
| a separate `.cpp` implementation unit | that file has no BMI at all → no cascade, on every compiler |~200x |
345
+
346
+
The perturbation here inserts a statement, so it takes the first row: every
347
+
engine has to rebuild the importers, and one that did not would be skipping
348
+
work. `+opt` does not skip it either — it does the same work 2.9x faster.
349
+
Splitting interface from implementation is the sturdiest of the three, because
350
+
it does not depend on GCC's body handling or on avoiding line shifts.
| reference mcpp |**2026.8.11.3**|`matrix.json` → `reference_mcpp`; the run records which release it actually resolved in `meta.json`, and the report names it in the column header|
217
217
| mcpp (the workload) |**2026.8.11.3** — `a749e9f`| submodule `projects/mcpp/mcpp-2026.8.11.3`|
Copy file name to clipboardExpand all lines: bench/SPEC.md
+50-14Lines changed: 50 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -62,6 +62,15 @@ table that was measuring something other than what it said:
62
62
| the workloads | git submodules under `bench/projects/`| xlings was cloned from its default branch at run time (`--hub src/xlings.cppm` named a file that had stopped existing); **mcpp's own sources were the checkout**, so every commit on a branch changed the thing being measured |
63
63
| the reference mcpp |`matrix.json.reference_mcpp`| a report said how fast this branch is, never whether it got faster |
64
64
65
+
> The reference pin is **not** required to equal the `.xlings.json` workspace
66
+
> pin. A guard once required that, on the theory that the reference arm is
67
+
> whatever CI bootstraps; neither half holds (the standard set runs on a
68
+
> developer box, and the bootstrap pin is a self-hosting floor that may lag a
69
+
> release), and bumping the pin after a release turned every e2e shard red on
70
+
> `main`. `run-standard.sh` resolves the arm by exact version and requires the
71
+
> binary to report that version itself, so a mismatch drops the column with a
72
+
> note instead of measuring the wrong release.
73
+
65
74
`--compiler payload:gcc` / `payload:clang` is the spelling that delivers the
66
75
second row: it resolves to the driver **inside mcpp's own registry**, so every
67
76
engine is handed the same binary. That is the suite's fairness rule
@@ -233,7 +242,7 @@ job: the cell still runs, and its note says what to distrust.
233
242
|`noop`| nothing | how cheap is "already up to date" |
234
243
|`touch-hub`| mtime bump on a widely-imported unit, **content unchanged**| can the engine prove the interface did not change? |
235
244
|`edit-comment`| a comment inserted into that same unit | the bytes *did* change but the interface did not — only an engine that compares the produced BMI avoids the cascade |
236
-
|`edit-body`| a real semantic edit inside a function body | the everyday loop — and whether a cascade is owed depends on **where the body lives**, not on the edit. See below. |
245
+
|`edit-body`| a real semantic edit inside a function body | the everyday loop — and whether a cascade is owed depends on **where the body lives and whether the edit moves lines**, not on what the body now does. See below. |
237
246
|`touch-leaf`| mtime bump on a unit nobody imports | recompile 1 + link |
238
247
239
248
#### ⚠️ `edit-body` perturbs a DIFFERENT FILE in each variant, and the two ask
@@ -301,25 +310,52 @@ Measured directly, GCC 16.1, comparing the BMI before and after:
301
310
302
311
| what is edited | BMI | cascade |
303
312
|---|---|---|
304
-
| a free exported function's body, in the `.cppm`|**byte-identical**| not owed |
305
-
| a **member function of an exported class**, inline in the `.cppm`|**differs**|**owed**|
306
-
| a body in a separate `.cpp` implementation unit |**byte-identical**|**not owed**|
307
-
308
-
A class's member function bodies are part of the class definition, which every
309
-
importer has to see, so they are serialised into the BMI. A free function's body
310
-
is not, and nothing in an implementation unit is.
311
-
312
-
So "editing one function rebuilt forty modules" is not inherent to named modules
313
-
— it is a consequence of where the body was written. `mcpp`'s own
314
-
`src/version_req.cppm` is the first case (the perturbation lands in
315
-
`Version::str()`, a member of an exported class), which is why its `edit-body`
316
-
row is a near-full rebuild and why that is correct.
313
+
| a body in a `.cppm`, edit **moves lines** (inserts or deletes one) |**differs**|**owed**|
314
+
| a body in a `.cppm`, edited **in place** (same line count) |**byte-identical**| not owed |
315
+
| a body in a separate `.cpp` implementation unit |**no BMI exists**| not owed |
316
+
317
+
GCC 16.1 does not serialise non-template function bodies, so changing what a
318
+
body *does* is invisible to importers. What it does serialise is the source
319
+
position of each declaration — so inserting a line moves every declaration
320
+
after it and the BMI changes for that reason alone.
321
+
322
+
⚠️ **An earlier version of this section said the deciding factor was whether the
323
+
body belonged to an exported class.** That was reasoning, and the measurement
324
+
refuted it: editing `Version::str()` — a member of an exported class — in place
325
+
rebuilt its object and left the BMI byte-identical, so no importer was touched.
326
+
The deciding factor is line movement, not class membership.
327
+
328
+
Two consequences:
329
+
330
+
* "editing one function rebuilt forty modules" is not inherent to named modules.
331
+
It follows from the edit moving lines in an interface unit.
332
+
* the third row is the sturdiest, because it holds for **every** compiler and
333
+
for every edit: a `.cpp` implementation unit produces no BMI, so nothing
334
+
downstream can depend on its contents. Clang, whose BMI carries more than
335
+
GCC's, cascades on an in-place body edit in a `.cppm` but not on a `.cpp`.
317
336
318
337
**This is what the two xlings pins measure.** Moving the implementations out of
319
338
the interface units takes `edit-body` from 88.33s to **1.77s** on the same
320
339
project — ~50x, the largest single effect anywhere in this suite, and a code
321
340
style rather than an engine feature.
322
341
342
+
#### KNOWN GAP: there is no scenario for an in-place body edit
343
+
344
+
The three rows in the table above are not equally covered. `edit-body` inserts a
345
+
statement, so only the **first** row is ever measured; the second — a semantic
346
+
edit that keeps the line count — has no scenario at all.
347
+
348
+
That is the everyday case, and it is the only one that would show cascade
349
+
suppression on a *real code change* rather than on a timestamp (`touch-hub`) or
350
+
a comment (`edit-comment`). Its absence makes the published tables read as
351
+
though the effect applies only when the code does not change, which understates
352
+
it.
353
+
354
+
Closing it is a `replace_in_first_body` beside `insert_into_first_body` (an
355
+
equal-length substitution, e.g. one integer literal for another of the same
356
+
width) plus a scenario token — and a re-run of the standard set, which is why it
357
+
is recorded here rather than half-added with no data behind it.
358
+
323
359
Real projects run five of the six: `touch-leaf` needs a unit nobody imports
324
360
*and* a stable name for it, which a generated fixture has by construction and a
0 commit comments