Skip to content

Commit 4c5ae96

Browse files
speak-agentspeak-agent
committed
docs(bench): add the bmi_schedule column for xlings — the two levers overlap
用户指出 xlings 表里的 `new` 列是默认构建(schedule 关着),所以对 cold 而言那 不是该看的对比。补上同一个二进制开了 `bmi_schedule` 的臂: | 树 | 场景 | 默认 | +bmi_schedule=on | | |---|---|---|---|---| | 合并式 | cold | 92.95s | **43.26s** | **2.15x** | | 合并式 | edit-body | 91.66s | **30.19s** | **3.04x** | | 分离式 | cold | 27.62s | 29.72s | 0.93x | | 分离式 | edit-body | 1.79s | 1.79s | 1.00x | **两个杠杆是重叠的,而且代码风格那个更大。** 调度靠「BMI 一就绪下游就开工」买 时间,所以只有在**存在级联可供重叠**时才有用。把实现拆出接口单元是直接**消除** 级联:cold 92.95→27.62,edit-body 91.66→1.79 —— 之后调度已经没有东西可赢了 (在 cold 上还小亏一点)。 要选一个的话,选代码风格;调度是给还没做这个改动的代码库用的。 顺带把 harness.sh 的清理 trap 改成 best-effort:Windows 上被 --timeout 杀掉的 构建会留下还持着日志句柄的编译器进程,`rm -rf` 报 "Device or resource busy", 而 trap 在 EXIT 上跑、它的状态就成了脚本的状态 —— 套件明明打印了 "bench harness OK",job 还是红的。删不掉一个临时目录不是测试结果。
1 parent 3a6fe5a commit 4c5ae96

6 files changed

Lines changed: 201 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -361,10 +361,16 @@ separate `.cpp`:
361361
| `touch-hub` | 89.39s → **1.76s** | 24.87s → **1.30s** | 1.35x |
362362
| `edit-body` | 89.46s → 88.33s | 2.73s → **1.77s** | **49.96x** |
363363

364-
Splitting implementations out of the interface units is worth **2.6x on a cold
364+
Splitting implementations out of the interface units is worth **3.1x on a cold
365365
build and ~50x on an edit** — a code style, not an engine feature, and the
366366
largest single effect in the suite.
367367

368+
It also overlaps with `bmi_schedule`. On the combined tree that setting takes
369+
`cold` from 92.95s to 43.26s (2.15x); on the split tree it changes nothing
370+
(27.62s → 29.72s), because there is no longer a cascade to overlap. **If you are
371+
choosing one, choose the code style** — the schedule is what helps a codebase
372+
that has not made that change.
373+
368374
Numbers are **n=1** except the split-tree `cold` row (n=3); read the ratios, not
369375
the digits. That row is why: at n=1 it read as a 23% regression, and at n=3 it is
370376
a marginal improvement — the single pair had caught one arm near the other's

‎bench/README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,25 @@ mcpp, because the cmake and xmake arms stop at the link here (SPEC.md §2).
193193
| `edit-body` | 89.46s → 88.33s | 2.73s → **1.77s** | **49.96x** |
194194
| `edit-comment` | 95.40s → 95.02s | 25.09s → 25.29s | 3.76x |
195195

196+
Those `new` columns are the DEFAULT build. With the opt-in BMI schedule on the
197+
same binary:
198+
199+
| tree | scenario | default | `+bmi_schedule=on` | |
200+
|---|---|---|---|---|
201+
| combined | `cold` | 92.95s | **43.26s** | **2.15x** |
202+
| combined | `edit-body` | 91.66s | **30.19s** | **3.04x** |
203+
| split | `cold` | 27.62s | 29.72s | 0.93x |
204+
| split | `edit-body` | 1.79s | 1.79s | 1.00x |
205+
206+
**The two levers overlap, and the code style is the bigger one.** The schedule
207+
buys time by letting importers start as soon as a BMI exists — so it only helps
208+
when there is a cascade to overlap. Splitting the implementations out removes
209+
the cascade instead: 92.95s → 27.62s cold and 91.66s → 1.79s on an edit, after
210+
which the schedule has nothing left to win (and costs a little on `cold`).
211+
212+
If you are choosing one, choose the code style. The schedule is what helps a
213+
codebase that has not made that change.
214+
196215
* **Splitting implementations out of the interface units is worth 2.6x on a cold
197216
build and ~50x on `edit-body`.** That is the largest single effect in this
198217
whole suite, and it is a *code style*, not an engine feature.

‎bench/results/pinned-workloads-20260813/report.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,22 @@ mcpp.
6868
* Splitting implementations out of the interface units is worth **3.1x cold** and
6969
**~50x on an edit**. A code style, not an engine feature — and the largest
7070
single effect anywhere in this suite.
71+
72+
### 2b. …and the opt-in schedule on top of it
73+
74+
| tree | scenario | default | `+bmi_schedule=on` | |
75+
|---|---|---|---|---|
76+
| combined | `cold` | 92.95s | **43.26s** | **2.15x** |
77+
| combined | `edit-body` | 91.66s | **30.19s** | **3.04x** |
78+
| split | `cold` | 27.62s | 29.72s | 0.93x |
79+
| split | `edit-body` | 1.79s | 1.79s | 1.00x |
80+
81+
**The two levers overlap, and the code style is the bigger one.** The schedule
82+
lets importers start as soon as a BMI exists, so it only helps where there is a
83+
cascade to overlap. Splitting the implementations removes the cascade instead,
84+
after which the schedule has nothing left to win — and costs a little on `cold`.
85+
86+
Raw: `xlings-combined-schedule-linux-gcc.json`, `xlings-split-schedule-linux-gcc.json`.
7187
* `touch-hub` reproduces the engine result on a codebase nobody tuned for it.
7288

7389
⚠️ **The `cold` row was nearly published as a 23% regression.** At n=1 it read

‎bench/results/pinned-workloads-20260813/xlings-combined-schedule-linux-gcc.json‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
{
2+
"protocol_version": 1,
3+
"started_at": "2026-08-13T07:05:05Z",
4+
"host": {
5+
"os": "linux",
6+
"arch": "x86_64",
7+
"cpu_model": "13th Gen Intel(R) Core(TM) i9-13900K",
8+
"logical_cores": 32,
9+
"physical_cores": 24,
10+
"heterogeneous": true,
11+
"ram_bytes": 67147722752,
12+
"toolchain": "/home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++"
13+
},
14+
"cells": [
15+
{
16+
"engine": "mcpp@2026.8.13.1",
17+
"compiler": "gcc",
18+
"profile": "release",
19+
"scenario": "cold",
20+
"fixture": "xlings-2026.8.11.2",
21+
"variant": "modules",
22+
"status": "ok",
23+
"note": "mcpp 2026.8.13.1",
24+
"runs": 1,
25+
"median_s": 92.951,
26+
"min_s": 92.951,
27+
"max_s": 92.951,
28+
"samples": [92.951]
29+
},
30+
{
31+
"engine": "mcpp@2026.8.13.1",
32+
"compiler": "gcc",
33+
"profile": "release",
34+
"scenario": "edit-body",
35+
"fixture": "xlings-2026.8.11.2",
36+
"variant": "modules",
37+
"status": "ok",
38+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
39+
"runs": 1,
40+
"median_s": 91.661,
41+
"min_s": 91.661,
42+
"max_s": 91.661,
43+
"samples": [91.661]
44+
},
45+
{
46+
"engine": "mcpp@2026.8.13.1+schedule=on",
47+
"compiler": "gcc",
48+
"profile": "release",
49+
"scenario": "cold",
50+
"fixture": "xlings-2026.8.11.2",
51+
"variant": "modules",
52+
"status": "ok",
53+
"note": "mcpp 2026.8.13.1",
54+
"runs": 1,
55+
"median_s": 43.263,
56+
"min_s": 43.263,
57+
"max_s": 43.263,
58+
"samples": [43.263]
59+
},
60+
{
61+
"engine": "mcpp@2026.8.13.1+schedule=on",
62+
"compiler": "gcc",
63+
"profile": "release",
64+
"scenario": "edit-body",
65+
"fixture": "xlings-2026.8.11.2",
66+
"variant": "modules",
67+
"status": "ok",
68+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
69+
"runs": 1,
70+
"median_s": 30.191,
71+
"min_s": 30.191,
72+
"max_s": 30.191,
73+
"samples": [30.191]
74+
}
75+
]
76+
}

‎bench/results/pinned-workloads-20260813/xlings-split-schedule-linux-gcc.json‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
{
2+
"protocol_version": 1,
3+
"started_at": "2026-08-13T07:11:42Z",
4+
"host": {
5+
"os": "linux",
6+
"arch": "x86_64",
7+
"cpu_model": "13th Gen Intel(R) Core(TM) i9-13900K",
8+
"logical_cores": 32,
9+
"physical_cores": 24,
10+
"heterogeneous": true,
11+
"ram_bytes": 67147722752,
12+
"toolchain": "/home/speak/.mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++"
13+
},
14+
"cells": [
15+
{
16+
"engine": "mcpp@2026.8.13.1",
17+
"compiler": "gcc",
18+
"profile": "release",
19+
"scenario": "cold",
20+
"fixture": "xlings-2026.8.13.1",
21+
"variant": "modules-impl",
22+
"status": "ok",
23+
"note": "mcpp 2026.8.13.1",
24+
"runs": 1,
25+
"median_s": 27.619,
26+
"min_s": 27.619,
27+
"max_s": 27.619,
28+
"samples": [27.619]
29+
},
30+
{
31+
"engine": "mcpp@2026.8.13.1",
32+
"compiler": "gcc",
33+
"profile": "release",
34+
"scenario": "edit-body",
35+
"fixture": "xlings-2026.8.13.1",
36+
"variant": "modules-impl",
37+
"status": "ok",
38+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
39+
"runs": 1,
40+
"median_s": 1.786,
41+
"min_s": 1.786,
42+
"max_s": 1.786,
43+
"samples": [1.786]
44+
},
45+
{
46+
"engine": "mcpp@2026.8.13.1+schedule=on",
47+
"compiler": "gcc",
48+
"profile": "release",
49+
"scenario": "cold",
50+
"fixture": "xlings-2026.8.13.1",
51+
"variant": "modules-impl",
52+
"status": "ok",
53+
"note": "mcpp 2026.8.13.1",
54+
"runs": 1,
55+
"median_s": 29.717,
56+
"min_s": 29.717,
57+
"max_s": 29.717,
58+
"samples": [29.717]
59+
},
60+
{
61+
"engine": "mcpp@2026.8.13.1+schedule=on",
62+
"compiler": "gcc",
63+
"profile": "release",
64+
"scenario": "edit-body",
65+
"fixture": "xlings-2026.8.13.1",
66+
"variant": "modules-impl",
67+
"status": "ok",
68+
"note": "mcpp 2026.8.13.1 · perturbation: in-body",
69+
"runs": 1,
70+
"median_s": 1.787,
71+
"min_s": 1.787,
72+
"max_s": 1.787,
73+
"samples": [1.787]
74+
}
75+
]
76+
}

‎bench/tests/harness.sh‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,13 @@ set -e
1010
# bench/tests -> two levels up is the repository root.
1111
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
1212
TMP=$(mktemp -d)
13-
trap "rm -rf $TMP" EXIT
13+
# `|| true`: on Windows a build killed by the --timeout test (§12) leaves the
14+
# compilers ninja spawned still running for a moment, and they hold the child
15+
# log open — `rm -rf` then fails with "Device or resource busy" and, because the
16+
# trap runs on EXIT, ITS status becomes the script's. The suite printed
17+
# "bench harness OK" and the job went red anyway. Failing to delete a temp
18+
# directory is not a test result.
19+
trap "rm -rf $TMP || true" EXIT
1420

1521
cd "$REPO/bench"
1622
"$MCPP" build > /dev/null

0 commit comments

Comments
 (0)