fix(learnings-search): rank by whole-word relevance so an exact match cannot be truncated away (#2762)

[szsunyuan]

Why (in your own words)

If you search your learnings store for something specific, gstack can tell you the thing isn't there when it is. gstack-learnings-search --query filters with token-OR over substrings, so every word admits its own set of entries, and the union is then ranked by confidence alone and cut to --limit (10 by default). Nothing errors, nothing comes back empty — you get ten confident, well-formed entries that simply aren't the one you asked for, and you read that as a complete answer. The search gets worse the more precisely you describe what you want: adding a discriminating word widens the candidate set, and the entry matching every word you typed can be pushed off the end by entries matching just one.

That's a false absence, which is the bad direction to fail in. A caller who already has reason to believe a learning exists gets a plausible list back and concludes it doesn't.

What this changes

Recall and relevance were the same computation. They are now two questions answered separately.

Recall is unchanged. The filter predicate is still substring containment across key, insight and files. Every entry that used to come back still comes back — verified set-identical to the base binary across nine query shapes on a real 95-entry store, and byte-identical on the no-query and --type paths.

Relevance is new, and deliberately stricter: the number of distinct query tokens present as whole words, counted in two tiers. What an entry says (its insight) decides first; what it is about (its key and file paths) only orders entries that already tie on substance. Confidence breaks a remaining tie, then recency.

Four properties of that definition carry the weight, and each exists because the looser version ranks badly on gstack's own shipped queries:

• Whole words, not substrings. A net that accepts bug inside debug, cause inside because, and fix inside fixture is right for finding candidates and useless for ordering them. /investigate ships the query debug investigation root cause hypothesis bug fix; under substring scoring, prose that merely says "debug output ran because the fixture was parallel" collects three free hits and outranks a real insight about root causes.
• Two tiers, not one. Keys here are long and descriptive by convention — this PR's own fixture key is six tokens. Scoring key, insight and files together lets verbosity beat quality: a thin entry whose name or path happens to carry the query words outranks an insight that answers it. But scoring the key at zero overcorrects, because an entry named exactly after the query then loses to incidental prose — the same truncation this fix exists to prevent. So naming is a tie-breaker, never a substitute for substance.
• Regular inflections count. A query is written in the base form; prose is written in whatever form the sentence needs. Matching only the exact form makes a real answer invisible to the query it answers — test could not see tests, merge could not see merged. A token now also scores against -s -es -d -ed -ing. The list holds only suffixes that are true suffixes of the base form (flaky is not flake+y), so reaching further would take stem rewriting rather than a longer list, and over-reach is bounded by the same word-boundary check that already applied: orbital still scores nothing for orbit.
• Distinct tokens. Repeating a word must not promote an entry that matches fewer concepts.

Word boundaries are found by case-folding rather than an ASCII range, so accented letters do not fake a boundary while scripts without case keep an embedded Latin term findable. Hyphens, dots, slashes and underscores all separate, so a snake_case key yields its individual words exactly as a kebab-case one does — gstack-learnings-log's key regex admits _, and file paths carry it constantly.

The score is initialized in the parse loop beside _effectiveConfidence and _crossProject, never inside the query filter. The filter runs only when a query was given, so a score assigned there would leave the no-query path reading whatever JSON.parse produced — and bin/gstack-learnings-log re-serializes unknown keys, so a stored value would reach the sort on the gstack-skill-start --limit 3 call that runs in every session. Every underscore-prefixed key is stripped at parse time, which immunizes fields added later rather than relying on each new one to remember to initialize itself.

When a query matched more than the limit shows, the summary line now states the part and the whole, following the phrasing already used in bin/gstack-retro-metrics (showing 300 of %d). It is gated on a non-empty query because that preamble call runs every session and must not grow a line, and written to stdout because the block redirects its own stderr to /dev/null.

One pre-existing defect is fixed here because this change is what exposes it. The formatter groups rows into a plain {} keyed by each row's own type. A row whose type names an inherited Object property — constructor, toString, valueOf, hasOwnProperty, __proto__ — finds that key already truthy, never builds the array, and throws on .push. The block ends 2>/dev/null || exit 0, so the throw becomes empty stdout at exit 0: one malformed row silently blanking every other row in the store, on every query, indistinguishable from "this project has no learnings". Relevance ranking is what makes it reachable, floating that row past the --limit cut that used to hide it. The map is now prototype-free.

Live evidence

Self-contained reproduction — one target entry whose key and insight contain all three query tokens at confidence 8, plus twelve confidence-10 decoys that contain line only as a substring (guideline, pipeline, deadline, …):

H=$(mktemp -d); C=$(mktemp -d); S=$(basename "$C"); mkdir -p "$H/projects/$S"
{
  echo '{"ts":"2026-05-01T00:00:00Z","skill":"t","type":"pitfall","key":"verify-preflight-project-line-before-trusting-report","insight":"Check the project line in the preflight report before trusting it","confidence":8,"source":"user-stated","files":[]}'
  for w in guideline pipeline deadline headline baseline timeline outline airline lifeline sideline streamline underline; do
    echo '{"ts":"2026-05-04T00:00:00Z","skill":"t","type":"pattern","key":"decoy-'\"$w\"'-rule","insight":"A '\"$w\"' related insight","confidence":10,"source":"user-stated","files":[]}'
  done
} > "$H/projects/$S/learnings.jsonl"
cd "$C"
for q in "preflight" "preflight project" "preflight project line"; do
  echo "$ gstack-learnings-search --query \"$q\""
  GSTACK_HOME="$H" bin/gstack-learnings-search --query "$q" | head -1
  GSTACK_HOME="$H" bin/gstack-learnings-search --query "$q" | grep -q 'verify-preflight-project-line' \
    && echo '  target: FOUND' || echo '  target: NOT IN LIST'
done

Before — upstream/main at 0d1bd561. The key contains all three tokens, in that order, and the third query is the one that loses it:

$ gstack-learnings-search --query "preflight"
LEARNINGS: 1 loaded (1 pitfall)
  target: FOUND
$ gstack-learnings-search --query "preflight project"
LEARNINGS: 1 loaded (1 pitfall)
  target: FOUND
$ gstack-learnings-search --query "preflight project line"
LEARNINGS: 10 loaded (10 patterns)
  target: NOT IN LIST

After — this branch:

$ gstack-learnings-search --query "preflight project line"
LEARNINGS: 10 of 13 matched (10 patterns); raise --limit for the rest

## Patterns
- [verify-preflight-project-line-before-trusting-report] (confidence: 8/10, user-stated, 2026-05-01)
  target: FOUND

The shipped single-keyword shape. --query "<keyword>" --limit 5 is what investigate/SKILL.md, qa/SKILL.md and ship/sections/adversarial.md actually run. Store: one answer at confidence 6 whose insight opens "Tests interleave when the runner is parallel", plus six confidence-10 rows whose only claim on the word is a ui/panelN.test.ts path:

$ gstack-learnings-search --query "test" --limit 5

main                   top hit = unrelated-5                  answer on page? 0
exact-form scoring     top hit = unrelated-5                  answer on page? 0
this branch            top hit = parallel-runner-interleave   answer on page? 1

Confidence-only ranking loses it, and so does scoring the exact form only — the insight says tests and the query says test. Matching a word together with its regular inflections is what finds it.

Same entry, different separator. Thirteen-row store, --query "preflight project line":

key = preflight-project-line   -> rank 1
key = preflight_project_line   -> rank 1   (was rank 13 of 13, off the page at --limit 10)

A malformed row can no longer blank the store. One row typed constructor among ten healthy rows:

$ gstack-learnings-search --query "alpha beta"

main                   rows printed = 10  (exit 0)
without the fix        rows printed = 0   (exit 0)   <-- total silence, caught pre-merge
this branch            rows printed = 10  (exit 0)

Verified across all five reachable poison types.

Recall is untouched. Per-token match counts on a real 95-entry store, base vs branch:

  token       base  branch
  pr           46     46
  merge        12     12
  ship          5      5
  version       1      1
  release       0      0
  changelog     0      0

Set-identity holds for whole queries too — the matched key sets are identical across nine shapes including all three shipped multi-token queries. The always-on preamble path is byte-identical:

$ diff <(tip --limit 3) <(branch --limit 3)                 # byte-identical
$ diff <(tip --limit 100) <(branch --limit 100)             # byte-identical
$ diff <(tip --type pitfall) <(branch --type pitfall)       # byte-identical

Thirty-two tests, up from six on main. Every behavioural claim is pinned by a test that dies when the corresponding line is reverted — each mutant applied in place and the suite re-run:

  reverted                                    tests that fail
  inflection list -> exact-form only                2
  underscore restored as a word character           1
  prototype-free type map -> {}                     5
  confidence decay disabled                         1
  confidence sorted ahead of relevance             11

Every commit in the series passes on its own, so the history is genuinely bisectable:

1c29eb73  test: run the binary through bash      22 pass 0 fail
7dea1d8e  fix: underscore as a word separator    23 pass 0 fail
942f04ca  fix: score regular inflections         25 pass 0 fail
f838af4f  fix: malformed row containment         30 pass 0 fail
e634744d  test: decay x relevance                32 pass 0 fail
f0abb322  refactor: drop dead _tokenHits         32 pass 0 fail

Full free suite:

$ bun run test
[test:free] PASS — 6 of 6 shards, 8,685 tests
exit 0

What this deliberately does not do

It does not narrow recall. Substring matching still admits pr inside provider, reports, and approach, which is why one broad token can pull in half a store. Narrowing the filter would change which entries are findable at all, for every query in every skill; that is a semantics decision deserving its own issue, and this change is careful to leave it alone. What changes is only that such an entry no longer outranks a real match.

The truncation notice is gated on a query, so a --type-only truncation stays silent. --type pitfall --limit 5 with no --query can still truncate without saying so. No shipped call site does that today, and widening the gate would risk the no-query preamble line, so I left it.

The formatter still groups by type, so the global rank order is visible within a type group rather than across the whole list. The limit is applied to the globally sorted list, so the exact match is no longer truncated away; but with mixed types it may not print first. Changing the output shape is a bigger call than this fix.

Relevance sorts ahead of confidence, and decay only feeds confidence. A row that has decayed to zero therefore still outranks a current row when it matches more of the query. That follows directly from the ordering #2762 asks for, so it is pinned as intended behaviour rather than asserted against, with a companion test showing decay still demotes the stale row as soon as the hit counts tie. Flagging it because it is a real consequence worth agreeing on, not a side effect to discover later.

Two pre-existing defects were found and left alone, both verified identical on main and neither caused by this change:

1. On Windows, Bun.stdin.text() throws above exactly 65536 bytes and the same || exit 0 turns it into the same false-absence signature — so a store large enough to need the new truncation notice returns nothing at all there, and --cross-project crosses that threshold much sooner. The fix is different plumbing (spool and redirect rather than pipe), not ranking, so it wants its own issue.
2. Ranking has no current-project preference, so under --cross-project a foreign store whose rows restate the query can outrank local ones. main has no locality key either — this adds a second lever rather than the first — but whether locality should be structural is a product call, not a quiet one-liner here.

Diagnosis, mechanism breakdown, and both implementation traps — gate the notice on a non-empty query, and write it to stdout — are the issue author's from #2762.

Scope

• Changed: bin/gstack-learnings-search (+109/-5), test/gstack-learnings-search.test.ts (+401/-2). No VERSION, no CHANGELOG, no generated files, no templates.

• Verified live by: the temp-fixture reproduction above against both the base binary and this branch; matched-key set identity across nine query shapes on a real 95-entry store; byte-identical output on the no-query and --type paths; thirty-two tests, with every behavioural claim mutation-verified by reverting the line it pins and confirming the test dies; each commit in the series run independently green; the full free suite green on all six shards (8,685 tests, exit 0); bin/gstack-redact clean on both files; bun run slop:diff upstream/main reporting no new findings. bash -n clean, and no $, backtick, backslash or double-quote character was added anywhere inside the double-quoted bun -e block.

• Did NOT test: CI itself — fork PRs don't receive eval secrets, and this is a deterministic bash/JS change with no model in the path, so no paid evals were run. The 95-entry recall measurement came from one real store on one machine, so treat the per-token counts as one data point rather than a distribution. scripts/free-test-durations.json still records the pre-existing 282 ms for this file; the added cases make it slower, and I left the shared durations seed alone rather than fold an unrelated regeneration into this diff.

  On Windows: the suite in this file previously could not execute at all — gstack-learnings-search is a shebang script and Windows has no shebang handling, so all 22 tests died at execFileSync with "Executable not found in $PATH" before reaching an assertion. They now run (0 pass / 22 fail → 32 pass / 0 fail, verified locally on Windows 10) via the explicit bash invocation already used by browse/test/learnings-injection.test.ts, which is a no-op where the shebang would have been honoured anyway. This does not change CI coverage: scripts/test-free-shards.ts excludes the file from the windows lane by scanning its content for path.join(ROOT, 'bin', …), and that pattern still matches regardless of whether the invocation is now correct. Making the heuristic reflect actual runnability is a change to the shared runner and belongs in its own PR.

Liveness proof (required)

• Liveness screenshot attached: GSTACK PR typed live into a real surface (not edited onto the image)

Checklist

• Liveness screenshot attached: GSTACK PR typed live into a real surface (not edited onto the image)
• This is not a generated-file-only diff (I edited the source/template and regenerated)
• No ETHOS.md edits, and no changes to voice / founder perspective / YC references
• New public command / external service / host adapter has an accepted issue linked (or N/A) — N/A, no new surface
• Linked issue or reproduction: gstack-learnings-search: a more specific query pushes the exact match off the list (token-OR + confidence-only ranking + silent truncation) #2762

Fixes #2762

[trunk-io]

Merging to main in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here