Skip to content

Commit b5400a2

Browse files
claudeclaude
committed
docs(lab): fingering tie-break audit and decision record
Archives the optimum-set and learned tie-break results (aggregates only): exact DPs verified against CP-SAT optima and ceilings on 1,954 / 1,954 holdout lines; the tie-break ladder under v1-fit — floor 32.4%, uniform 42.9%, production 44.1%, learned local 44.0%, learned local + hand anchor 47.3%, ceiling 55.5%; tie structure and the v1 control; the diagnosis that tab authors' deviations are one-directional (lower string, higher fret) but context-dependent, with the preceding fretted event closer to the author's choice in 63% of differing lines; limitations (human-sourced anchor, linear non-convergent learner) and follow-ups. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 3adf76a commit b5400a2

3 files changed

Lines changed: 190 additions & 0 deletions

File tree

‎docs/audit/2026-09-fingering-tie-break.md‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# 2026-09 — Fingering tie-break: optimum sets and a learned secondary objective
2+
3+
Follow-up to the optimality-gap audit
4+
([`2026-09-fingering-optimality-gap.md`](2026-09-fingering-optimality-gap.md),
5+
PhysShell/griff#197), whose corrected numbers left one sharp question. Under
6+
the fitted `v1` weights (fret 0, open-string penalty 3, position shift 1,
7+
string change 0) the production DP keeps **44.1%** agreement with tab authors
8+
on holdout songs, while the set of cost-optimal fingerings contains **55.5%**.
9+
That 11.4-point gap needs no change to the primary objective, only a better
10+
choice among its ties. This audit measures the tie sets exactly, asks how much
11+
of the gap a **human-blind learned tie-break** recovers, and locates the
12+
information the tab authors used.
13+
14+
Research tooling only (`lab/`, excluded from the workspace); no production
15+
code changes. Corpus, protocol and holdout are those of the corrected
16+
optimality-gap audit: 410 Guitar Pro files, 1,149 guitar tracks, 9,045
17+
monophonic lines, song-level holdout (1,954 test lines, 70,933 notes). Only
18+
aggregates are recorded; tab content stays out of git (ADR-0005).
19+
20+
## What was built
21+
22+
`lab/src/ties.rs`, red → green per commit:
23+
24+
- **`Chain`** — the `v1` objective per line: candidates in production order,
25+
unary and pairwise costs.
26+
- **`optimum_set`** — exact chain DPs, no search. Forward and backward cost
27+
tables carry path counts (saturating `u64` plus an exact natural log). A node
28+
or edge lies on an optimal path iff `f + g = optimum`. From these come the
29+
number of optimal paths; the **least and most agreement** with a reference
30+
among them (min/max DPs over optimal edges); the **expected agreement under a
31+
uniform draw** from the optimum set (paths through a candidate / total, in
32+
log space); and a most-agreeing optimal path.
33+
- **`lexicographic_path`** — minimizes `(primary, secondary)`; with zero
34+
secondary weights it reproduces the production DP path exactly. It supports
35+
loss augmentation (a margin per reference-matching note).
36+
- **`latent_target`** — among the most-agreeing optimal paths, the cheapest
37+
under the current secondary weights: the achievable learning target. The
38+
human path itself is primary-optimal in only 31% of holdout lines.
39+
- **`train_secondary`** — averaged, loss-augmented, latent-target structured
40+
perceptron over integer weights, inside the primary optimum set.
41+
- **Secondary features** — 20 local fingering-geometry features (fret, open
42+
string, string one-hot; fret and string distance, string change, same fret,
43+
spans over 3/5 frets, open transitions, diagonal and box moves, direction),
44+
plus **`anchor_distance`** (|fret − anchor| per fretted note).
45+
`TabLine::anchor_fret` gives the anchor: the fret of the latest fretted note
46+
of the voice before the line, the lowest fret when that onset is a chord.
47+
- **Runner** (`fingering_gap`) — `ties-check` compares the DPs with verified
48+
CP-SAT records; `tiebreak` trains on train songs, picks the margin on a
49+
song-level validation bucket (5,111 fit / 1,980 validation lines), and reports
50+
on holdout songs.
51+
52+
## Verification
53+
54+
- Contract suite against brute force on exhaustive small families:
55+
- optimum, exact count, min/max/expected agreement;
56+
- secondary optimality, including anchored chains;
57+
- the loss-augmented least-agreeing path;
58+
- the latent target;
59+
- zero-secondary equality with `infer_positions`;
60+
- count saturation with an exact log;
61+
- perceptron convergence on separable synthetic tie-breaks, local and anchored.
62+
- **Exact DPs versus CP-SAT** (`ties-check`), on the verified agreement-pass
63+
records from the corrected optimality-gap run: for **1,954 / 1,954** holdout
64+
lines under both `v1` and `v1-fit`, the DP optimum equals the CP-SAT optimum
65+
and the DP ceiling equals CP-SAT's recounted ceiling. The ceiling now costs
66+
milliseconds instead of a solver pass. CP-SAT remains the spot-check oracle
67+
for these DPs.
68+
69+
## Results — the tie-break ladder (holdout songs)
70+
71+
Primary `v1-fit` (the under-discriminative objective):
72+
73+
| rung | agreement | lines changed vs production |
74+
|---|---|---|
75+
| floor — least-agreeing optimal path | 32.4% | |
76+
| uniform draw from the optimum set | 42.9% | |
77+
| production tie-break (lowest candidate index) | 44.1% | — |
78+
| learned tie-break, local features | 44.0% | 89 |
79+
| **learned tie-break, local features + hand anchor** | **47.3%** | 411 |
80+
| ceiling — most-agreeing optimal path | 55.5% | |
81+
82+
The anchored tie-break recovers **3.2 points, 28% of the gap** between the
83+
production tie-break and the ceiling, without touching the primary objective.
84+
The margin (10⁴) was chosen on validation. The validation curve is interior
85+
(47.0 / 46.9 / 47.7 / **48.8** / 48.6 / 48.6 / 48.5% for margins 0 … 10⁷),
86+
over 20 epochs of a non-separable problem (≈57k updates).
87+
88+
Tie structure under `v1-fit` (holdout):
89+
90+
- the optimum is unique in 45.3% of lines;
91+
- ln(#optimal paths): median 0.69 (2 paths), p75 1.79 (6), p90 5.55 (~250),
92+
p99 28.3;
93+
- the human fingering is itself optimal in 30.9% of lines.
94+
95+
**Control — production `v1` weights.** The optimum is unique in 87.1% of
96+
lines, and the whole ladder spans floor 35.7% → ceiling 36.2%. The learned
97+
tie-break reaches 36.1%, inside that band: the learner finds what little there
98+
is and cannot exceed the ceiling by construction.
99+
100+
## Results — where the tab authors' information lives
101+
102+
**The local features carry none of it.** With local features only, the
103+
learned tie-break converges on production's own choices: the latent target,
104+
margins up to 10⁷ and 20 epochs changed no conclusion (44.0% vs 44.1%). A
105+
diagnostic probe over train songs shows why:
106+
107+
- where a most-agreeing optimal path differs from production, the author plays
108+
the same pitch **one or two strings lower and higher up the neck** (+1 string
109+
/ +5 frets 50%, +1 / +4 40%, +2 / +9 7% of differing notes);
110+
- yet in about three quarters of lines the author agrees with production's
111+
"highest string, lowest fret" choice. No global weight on local geometry
112+
separates the two;
113+
- 42.5% of the differing notes sit in repeated-pitch runs, where hand travel
114+
inside the line is free.
115+
116+
**Part of it is just outside the line.** Lines are cut at chords and rests,
117+
and the author's choice tracks where the hand was before the cut. On
118+
differing notes the author's fret is closer to the anchor than production's
119+
in 56.7% of notes against 37.8% (5.5% equal); per line, 63.4% against 32.8%.
120+
Adding that one feature produced the only real gain.
121+
122+
## Reading
123+
124+
- **The v1-fit ceiling is not a local-geometry ceiling.** The optimum set does
125+
contain better fingerings, but a tie-break that sees only the line cannot
126+
tell them apart. The discriminating information is contextual.
127+
- **Context works as expected.** One boundary feature recovers about a
128+
quarter of the gap. The rest likely needs richer context than a single fret:
129+
the anchor string, time since the anchor, the shape of the chord before and
130+
after, and what follows the line.
131+
- **Lexicographic primary + secondary is the right production shape** for this
132+
kind of gain. It is exact, cheap, and leaves the primary objective's
133+
guarantees intact.
134+
135+
## Limitations (recorded, not hidden)
136+
137+
- **The anchor is taken from the human tab.** That is the tab-completion
138+
scenario. For MIDI-sourced material the preceding event is usually a chord,
139+
which production leaves unpositioned today (ADR-0019 §7): an anchor there
140+
needs chord voicing first. This ties the monophonic quality gain to the
141+
next Lab subject.
142+
- The secondary is linear and hand-featured, and the perceptron is not
143+
convergent on real data (averaged weights, 20 epochs). A max-margin or
144+
probabilistic learner may extract more from the same features.
145+
- One primary (`v1-fit`) and its control; the hand-position model is parked
146+
per the optimality-gap audit.
147+
- Agreement treats one tab as ground truth.
148+
149+
## Follow-ups proposed
150+
151+
1. Richer boundary context for the secondary: anchor string, anchor-to-line
152+
time, the following event's position, chord span. Measure each as an
153+
ablation on this ladder.
154+
2. Chord voicing (the next Constraint Lab subject), which also produces
155+
anchors for MIDI-sourced lines.
156+
3. Once the anchored tie-break is stable, an ADR for lexicographic
157+
primary + secondary fingering in `core`, with the secondary weights as
158+
versioned data (ADR-0017 §3).

‎docs/decisions.log.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2858,3 +2858,17 @@ Architectural decisions go to [`adr/`](adr/) instead.
28582858
venv adapter (`lab/cpsat/`), never a dependency; idea-level prior art
28592859
only (TablaZinc is MPL-2.0, `guitar-tab-generator` GPL-3.0 — no code
28602860
copied).
2861+
2862+
- 2026-09-17 — In the context of the fitted fingering objective being
2863+
under-discriminative (holdout: production tie-break 44.1% agreement, best
2864+
cost-optimal fingering 55.5%), we decided to **measure optimum sets with
2865+
exact chain DPs and to learn a lexicographic secondary tie-break instead of
2866+
refitting the primary objective**, to achieve an attribution of the gap
2867+
between search, local fingering geometry and context, accepting that the
2868+
first useful feature comes from the human tab (the hand anchor before a
2869+
line). Result (`docs/audit/2026-09-fingering-tie-break.md`): the DPs agree
2870+
with verified CP-SAT optima and ceilings on 1,954 / 1,954 holdout lines; a
2871+
tie-break over local geometry recovers nothing (44.0%); adding the anchor
2872+
recovers 3.2 points (47.3%, 28% of the gap). The discriminating
2873+
information is contextual, so the next Lab subject (chord voicing) is also
2874+
what makes such anchors available for MIDI-sourced material.

‎lab/README.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,24 @@ Everything written to `--out` is corpus-derived and stays local; `report`
8989
archives aggregates only (`report.json`). Results:
9090
[`../docs/audit/2026-09-fingering-optimality-gap.md`](../docs/audit/2026-09-fingering-optimality-gap.md).
9191

92+
## Optimum sets and the learned tie-break
93+
94+
`src/ties.rs` measures a fingering objective's optimum set exactly with chain
95+
DPs — how many optimal paths, and the least, most and expected (uniform draw)
96+
agreement with the tab author among them — and learns a human-blind secondary
97+
objective that breaks ties lexicographically after the primary cost
98+
(averaged, loss-augmented, latent-target perceptron). `ties-check` compares
99+
the DPs with verified CP-SAT records; `tiebreak` reports the ladder
100+
floor → uniform → production tie-break → learned → ceiling on holdout songs,
101+
with and without the line's hand anchor.
102+
103+
```sh
104+
./target/release/fingering_gap ties-check --tabs $T --out $O --v1 v1-fit=0,-3,1,0
105+
./target/release/fingering_gap tiebreak --tabs $T --out $O --v1 v1-fit=0,-3,1,0
106+
```
107+
108+
Results: [`../docs/audit/2026-09-fingering-tie-break.md`](../docs/audit/2026-09-fingering-tie-break.md).
109+
92110
## Known spike limits (deliberate)
93111

94112
- The reference solver is leaf-checked backtracking with two sound band

0 commit comments

Comments
 (0)