feat: [U4-B] benchmark kernels + lowering/correctness/limitations wri… - #34
Merged
Conversation
…te-ups Adds the two things the project needed once every per-operation lowering was verified: something that measures the payoff, and something that explains it. bench/ -- native two-build comparison. Each kernel is compiled twice from the same .ll: baseline goes straight to llc (the default legalizer scalarizes the narrow op), lowered goes through opt -passes=nybbler and then the *identical* llc line, so the only difference between the two binaries is the pass. A shared driver.c times only the kernel loop and FNV-1a checksums the output, which both defeats dead-code elimination and gates the comparison: run.sh refuses to report timings if the two builds disagree. Kernels: arith_i4 (<32 x i4> add/shl/sub), field_i2 (<64 x i2> add/lshr/sub), mask_i1 (<128 x i1> bitwise), cmp_i4 (icmp ult to a packed bitmask). Measured on Ryzen 9 5900HX / AVX2 / LLVM 22.1.8: arith_i4 67.25 -> 5.02 ns/vec 13.40x field_i2 144.85 -> 2.49 ns/vec 58.27x cmp_i4 48.50 -> 35.90 ns/vec 1.35x (partial, see below) mask_i1 1.04 -> 1.04 ns/vec 1.00x Three findings shaped the design and are documented rather than smoothed over: * Kernels must keep loads/stores at byte type and bitcast only around the op. Written the obvious way (load <32 x i4>), the lowered build is *worse* -- the narrow load scalarizes in both builds and nybbler must then repack. * mask_i1 shows no speedup because LLVM already reinterprets <K x i1> bitwise as bytes. Reported at face value: the speedup claim rests on the masked paths, not on the correct-by-construction bitwise case. * cmp_i4 is a lower bound. The pass lowers the icmp but not the bitcast that materializes its <K x i1> result, and that dominates both builds. test/bench_checksum.test wires the checksum gate into lit (correctness only -- CI runners are too noisy for a speedup threshold). docs/ -- lowering.md derives each handler's mask formula and why it is that formula; correctness.md gives the containment argument (bitwise correct by construction, then per-op: why no carry escapes add, no borrow escapes sub, and no shifted-in bit survives outside its field) and points at what verifies it; benchmarks.md holds methodology, results, and limitations. README refreshed -- it still described Slice 1 (bitwise only, non-byte-multiple vectors skipped), both superseded three commits ago.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
U4-B — Benchmark kernels + write-ups
Every per-operation lowering is derived and verified, so this ticket produces
what we benchmark and what we present.
Benchmark kernels
bench/compiles each kernel twice from the same.ll: baseline straight tollc -O3 -mcpu=native(the default legalizer scalarizes the narrow op), andlowered through
opt -passes=nybblerfollowed by the identicalllcline.The only difference between the two binaries is the pass.
A shared
driver.ctimes only the kernel loop — fixed-seed inputs, no I/O inthe timed region, 20 warmup + 50 timed reps reporting the minimum — and FNV-1a
checksums the output. The checksum both defeats dead-code elimination and gates
the comparison:
run.shrefuses to report timings if the two builds disagree.Kernels:
arith_i4(<32 x i4>add/shl/sub),field_i2(<64 x i2>add/lshr/sub),
mask_i1(<128 x i1>bitwise),cmp_i4(icmp ult→ bitmask).Measured on Ryzen 9 5900HX / AVX2 / LLVM 22.1.8:
arith_i4field_i2cmp_i4mask_i1Scalarization markers in the inner loop go 114→0 (
arith_i4) and236→0 (
field_i2).Three findings are documented rather than smoothed over:
(
load <32 x i4>), the lowered build is worse — the narrow load scalarizesin both builds and the pass must then repack. Recorded as a caller constraint.
mask_i1shows no speedup because LLVM already reinterprets<K x i1>bitwise as bytes. Reported at face value: the speedup claim rests on the
masked paths, not the correct-by-construction bitwise case.
cmp_i4is a lower bound — the pass lowers theicmpbut not the bitcastmaterializing its
<K x i1>result, which dominates both builds.Write-ups
docs/lowering.md— carrier dispatch, SWAR add/sub carry and borrowcontainment, per-field shift masking,
ashrsign handling, compare lowerings.docs/correctness.md— why bitwise is correct by construction, how the maskedpaths keep carries and shifted-in bits inside their field, and what verifies it.
docs/benchmarks.md— methodology, results, and limitations (no mul/div,vertical ops only,
icmplimited to eq/ne/ult/slt, untouched narrowloads/stores, the padding path's cost on non-byte-multiple widths).
README.mdrefreshed — it still described Slice 1 (bitwise only,non-byte-multiple vectors skipped), superseded three commits ago.
Testing
test/bench_checksum.testwires the checksum gate into lit — correctness only,since CI runners are too noisy for a speedup threshold. Full suite 58/58 on the
default seed and on
NYBBLER_DIFF_SEED=1234. No changes tolib/Nybbler.cpp.closes #10