feat(fsst): branch-free matching (ShortCodeTable + LossyPerfectHashTable + Matcher)

Implements the FSST paper's Algorithm 4 ("lossy perfect hashing", §5.1) as the
matching core for the extracted fsst module (issue #287, PR 2 of the plan in
goofy-roaming-moler.md).

WHY: the current writer's FsstEncodingEncoder.SymbolTable.longestMatch probes
symbol lengths 8 down to 1 in a per-position loop — up to eight sequential
open-addressing hash lookups per input byte. That variable-length loop is
exactly the non-uniform hot-loop body CLAUDE.md forbids (it blocks C2
auto-vectorization). This replaces it with an O(1) branch-free structure:

- ShortCodeTable: a 65536-entry array direct-indexed on the first two input
  bytes, resolving 0/1/2-byte matches in one array read.
- LossyPerfectHashTable: 2048 slots (matching the spiraldb/fsst Rust reference's
  L1D-cache-line-split reasoning and the vortex-jni benchmark target, not the
  paper's literal 4096), keyed on the first three bytes, resolving 3-8 byte
  matches with one hash lookup plus one masked 8-byte compare — no probing.
- Matcher: composes the two branch-free, no loop over candidate lengths.

Collision-ordering invariant (load-bearing): the hash table inserts in a single
forward pass with first-writer-wins. The caller must pass symbols in
descending-gain order, so on a 3-byte-prefix hash collision the higher-gain
symbol (seen first) keeps the slot and the lower-gain one is rejected, never
overwritten — the paper's rule that the more valuable symbol wins a lossy
collision. These classes do not re-sort their input. Tests cover a real
collision (higher gain wins, lower gain correctly misses) and an adversarial
hash-collision-but-byte-mismatch case proving the masked compare, not the hash
alone, gates a hit.

Purely additive: writer/reader untouched; these are wired into training in PR 3.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: dfa1

fsst/src/main/java/io/github/dfa1/vortex/fsst/LossyPerfectHashTable.java

@@ -0,0 +1,143 @@
+package io.github.dfa1.vortex.fsst;
+
+import java.util.List;
+
+/// Lossy perfect hash table resolving 3-8 byte FSST matches from the first three bytes of an input
+/// word (the FSST paper's Algorithm 4, §5.1 "Predicated Scalar Compression").
+///
+/// Each slot holds at most one candidate symbol. A lookup hashes the input word's first three bytes
+/// to exactly one slot, reads it (no probing, no chaining), and confirms the match with a single
+/// masked 8-byte compare: the input word's high bits beyond the candidate's length are masked off,
+/// then compared against the candidate's packed bytes. A slot miss or a failed compare is a "no
+/// match" — this is what "lossy" means: a genuine 3-8 byte match can occasionally miss because a
+/// higher-gain symbol won its slot on a hash collision. That is harmless, because greedy parsing
+/// then falls back to a shorter match ([ShortCodeTable]) or an escape — never to wrong output.
+final class LossyPerfectHashTable {
+
+    /// Number of slots. The paper specifies 4096; the well-regarded Rust reference
+    /// `spiraldb/fsst` uses 2048, citing avoidance of L1D cache-line splits (each slot is small, so
+    /// 2048 slots keep the whole table within a handful of cache lines that a scan touches
+    /// repeatedly), and 2048 is this project's benchmark target (`vortex-jni`) — so we deliberately
+    /// match the reference here rather than the paper's literal 4096. A power of two lets the hash
+    /// mask instead of taking a modulo, which the hot-loop rule (no per-element modulo/division)
+    /// requires.
+    private static final int SLOTS = 2048;
+
+    /// Mask selecting a slot index from a hash; valid because [#SLOTS] is a power of two.
+    private static final int SLOT_MASK = SLOTS - 1;
+
+    /// Multiply-xor mixing constant, the same shape as the old encoder's `SymbolTable.indexHash`
+    /// (the golden-ratio 64-bit odd constant `2^64 / phi`). Multiplying by a large odd constant and
+    /// folding the high bits down spreads the low three input bytes across the whole word so the
+    /// slot mask sees well-mixed bits, avoiding clustering when many symbols share a low byte.
+    private static final long HASH_MULTIPLIER = 0x9E3779B97F4A7C15L;
+
+    /// Low three bytes of the input word — the only bytes the hash keys on.
+    private static final long PREFIX_MASK = 0x00FF_FFFFL;
+
+    /// Per-slot packed symbol bytes, LSB-first ([Symbol] convention). Meaningful only where the
+    /// corresponding [#occupied] entry is set.
+    private final long[] packedBytes;
+
+    /// Per-slot mask `~0L >>> ignoredBits` (`ignoredBits = 64 - 8 * length`) applied to an input
+    /// word before comparing, clearing the high bytes past the candidate's length.
+    private final long[] keepMask;
+
+    /// Per-slot symbol length in bytes, 3-8.
+    private final int[] lengths;
+
+    /// Per-slot code (the symbol's list index), meaningful only where [#occupied] is set.
+    private final int[] codes;
+
+    /// Whether each slot holds a candidate.
+    private final boolean[] occupied;
+
+    private LossyPerfectHashTable(long[] packedBytes, long[] keepMask, int[] lengths, int[] codes,
+                                  boolean[] occupied) {
+        this.packedBytes = packedBytes;
+        this.keepMask = keepMask;
+        this.lengths = lengths;
+        this.codes = codes;
+        this.occupied = occupied;
+    }
+
+    /// Builds the table from the trained symbols in descending-gain order, keeping only those of
+    /// length 3-8 (shorter symbols are the [ShortCodeTable]'s job and are skipped here). The list
+    /// index is each symbol's code, matching the parallel-array convention
+    /// [Decompressor#of(long[], int[])] uses.
+    ///
+    /// Insertion is a single forward pass with first-writer-wins on collision. WHY this is
+    /// load-bearing: the caller passes symbols in descending gain order, so when two symbols' first
+    /// three bytes hash to the same slot, the one seen first (the higher-gain one) keeps the slot
+    /// and the later (lower-gain) one is skipped, never overwritten. This is exactly the paper's
+    /// rule that the more valuable symbol wins a lossy collision. This class must NOT re-sort its
+    /// input — it consumes whatever order the caller gives and inserts once.
+    ///
+    /// @param symbolsByGainDescending the trained symbols, code = list index, gain-descending
+    /// @return a hash table resolving 3-8 byte matches with first-writer-wins on collision
+    static LossyPerfectHashTable of(List<Symbol> symbolsByGainDescending) {
+        long[] packedBytes = new long[SLOTS];
+        long[] keepMask = new long[SLOTS];
+        int[] lengths = new int[SLOTS];
+        int[] codes = new int[SLOTS];
+        boolean[] occupied = new boolean[SLOTS];
+        for (int code = 0; code < symbolsByGainDescending.size(); code++) {
+            Symbol symbol = symbolsByGainDescending.get(code);
+            if (symbol.length() < 3) {
+                continue; // Length 1-2 belongs to ShortCodeTable.
+            }
+            int slot = slotFor(symbol.packedBytes());
+            if (occupied[slot]) {
+                continue; // First writer (higher gain) wins; skip the collision.
+            }
+            occupied[slot] = true;
+            packedBytes[slot] = symbol.packedBytes();
+            keepMask[slot] = keepMaskFor(symbol.length());
+            lengths[slot] = symbol.length();
+            codes[slot] = code;
+        }
+        return new LossyPerfectHashTable(packedBytes, keepMask, lengths, codes, occupied);
+    }
+
+    /// Looks up `word` and reports whether a stored 3-8 byte symbol really matches it.
+    ///
+    /// The input word's first three bytes select one slot; the candidate there matches only if the
+    /// masked compare passes (`(word & keepMask) == candidate.packedBytes`), which rules out both
+    /// hash collisions with unrelated bytes and empty slots. On a miss the returned [HashMatch] has
+    /// `hit == false` and the caller falls back to [ShortCodeTable].
+    ///
+    /// @param word an 8-byte little-endian input word starting at the current match position, with
+    ///             any bytes past the remaining input already zero-padded by the caller
+    /// @return the match result: `hit`, and when hit the matched `code` and `length`
+    HashMatch lookup(long word) {
+        int slot = slotFor(word);
+        boolean hit = occupied[slot] && (word & keepMask[slot]) == packedBytes[slot];
+        return new HashMatch(hit, codes[slot], lengths[slot]);
+    }
+
+    /// Computes the slot index a word hashes to, keyed on its first three bytes. Package-visible so
+    /// tests can construct deliberate hash collisions against a stable, inspectable hash rather than
+    /// blindly brute-forcing one — a stable hash is easier to reason about and to test.
+    ///
+    /// @param word an input word; only its low three bytes are hashed
+    /// @return the slot index in `0 .. SLOTS - 1`
+    static int slotFor(long word) {
+        long mixed = (word & PREFIX_MASK) * HASH_MULTIPLIER;
+        return (int) (mixed >>> 32) & SLOT_MASK;
+    }
+
+    private static long keepMaskFor(int length) {
+        int ignoredBits = 64 - 8 * length;
+        return ~0L >>> ignoredBits;
+    }
+
+    /// Result of a [LossyPerfectHashTable#lookup(long)]: whether a 3-8 byte symbol matched and, if
+    /// so, its code and length. When `hit` is false, `code` and `length` are unspecified and the
+    /// caller must ignore them.
+    ///
+    /// @param hit whether a stored symbol really matched the input word
+    /// @param code the matched symbol code, valid only when `hit`
+    /// @param length the matched symbol length in bytes (3-8), valid only when `hit`
+    record HashMatch(boolean hit, int code, int length) {
+    }
+}

fsst/src/main/java/io/github/dfa1/vortex/fsst/Matcher.java

@@ -0,0 +1,87 @@
+package io.github.dfa1.vortex.fsst;
+
+import java.util.List;
+
+/// Branch-free longest-match matcher composing a [LossyPerfectHashTable] (3-8 byte candidates) with
+/// a [ShortCodeTable] (0/1/2 byte candidates), the FSST paper's Algorithm 4.
+///
+/// This replaces the old encoder's `longestMatch`, which probed symbol lengths 8 down to 1 in a
+/// per-position loop (up to eight sequential hash lookups per input byte). Here a single hash-table
+/// lookup plus one masked compare yields the 3-8 byte candidate, a single array read yields the
+/// 0/1/2 byte candidate, and one conditional select picks the longer — no loop over candidate
+/// lengths, no per-length branch. That uniform body is exactly what the hot-loop rule (no
+/// modulo/division/variable-target branch per element) needs to stay JIT-vectorizable, which the
+/// old eight-iteration loop could not deliver.
+public final class Matcher {
+
+    /// Longest symbol length that can be resolved, in bytes.
+    private static final int MAX_SYMBOL_LENGTH = 8;
+
+    private final LossyPerfectHashTable hashTable;
+    private final ShortCodeTable shortCodes;
+
+    private Matcher(LossyPerfectHashTable hashTable, ShortCodeTable shortCodes) {
+        this.hashTable = hashTable;
+        this.shortCodes = shortCodes;
+    }
+
+    /// Builds a matcher from the trained symbols in descending-gain order.
+    ///
+    /// The list index is each symbol's code, matching the parallel-array convention
+    /// [Decompressor#of(long[], int[])] uses. The whole list is handed to both tables: each keeps
+    /// only the entries of its own length class ([ShortCodeTable] takes lengths 1-2,
+    /// [LossyPerfectHashTable] takes lengths 3-8), so a symbol's code stays equal to its index in
+    /// the original list without any placeholder padding. The caller's gain-descending order is
+    /// preserved, since it is load-bearing for the hash table's first-writer-wins collision
+    /// handling. The input is not re-sorted.
+    ///
+    /// @param symbolsByGainDescending the trained symbols, code = list index, gain-descending
+    /// @return a matcher resolving the longest 0-8 byte match at any input position
+    public static Matcher of(List<Symbol> symbolsByGainDescending) {
+        return new Matcher(
+                LossyPerfectHashTable.of(symbolsByGainDescending),
+                ShortCodeTable.of(symbolsByGainDescending));
+    }
+
+    /// Returns the longest match at the current input position packed as `code << 8 | length`.
+    ///
+    /// The hash table is consulted first for a 3-8 byte candidate; on a real hit that candidate is
+    /// returned, otherwise the result falls back to the short-code table's 0/1/2 byte answer. A
+    /// length of 0 (and code [ShortCodeTable#NO_CODE]) means no symbol matched and the caller must
+    /// escape the current byte. Callers that want the parts separately can use
+    /// [#codeOf(int)] and [#lengthOf(int)].
+    ///
+    /// @param word an 8-byte little-endian input word starting at the current match position, with
+    ///             any bytes past the remaining input already zero-padded by the caller
+    /// @return the longest match as `code << 8 | length`; length 0 signals "no match, escape"
+    public int longestMatch(long word) {
+        LossyPerfectHashTable.HashMatch hashMatch = hashTable.lookup(word);
+        if (hashMatch.hit()) {
+            return hashMatch.code() << 8 | hashMatch.length();
+        }
+        return shortCodes.codeFor(word) << 8 | shortCodes.lengthFor(word);
+    }
+
+    /// Extracts the code from a packed [#longestMatch(long)] result.
+    ///
+    /// @param packedMatch a `code << 8 | length` value from [#longestMatch(long)]
+    /// @return the matched symbol code, or [ShortCodeTable#NO_CODE] when the length is 0
+    public static int codeOf(int packedMatch) {
+        return packedMatch >> 8;
+    }
+
+    /// Extracts the length from a packed [#longestMatch(long)] result.
+    ///
+    /// @param packedMatch a `code << 8 | length` value from [#longestMatch(long)]
+    /// @return the matched symbol length in bytes, 1-8, or 0 when there is no match
+    public static int lengthOf(int packedMatch) {
+        return packedMatch & 0xFF;
+    }
+
+    /// Longest symbol length resolvable by this matcher, in bytes.
+    ///
+    /// @return the maximum symbol length, 8
+    public static int maxSymbolLength() {
+        return MAX_SYMBOL_LENGTH;
+    }
+}

fsst/src/main/java/io/github/dfa1/vortex/fsst/ShortCodeTable.java

@@ -0,0 +1,94 @@
+package io.github.dfa1.vortex.fsst;
+
+import java.util.List;
+
+/// Direct-indexed table resolving the shortest FSST matches — length 0 (no match), 1, or 2 —
+/// from the first two bytes of an input word (the FSST paper's Algorithm 4 `shortCodes`).
+///
+/// The key is the input word's low 16 bits read as an unsigned little-endian value, i.e. the first
+/// two bytes of the input at the current position (byte 0 in the low 8 bits, byte 1 next). That key
+/// indexes one of 65536 slots, each holding a packed `code << 8 | length` giving the matched symbol
+/// code and its length in one array read — no hashing, no probing.
+///
+/// Only length-1 and length-2 symbols populate the table; longer symbols are the
+/// [LossyPerfectHashTable]'s job. Each 16-bit key `hi:lo` is seeded so that, absent a longer match,
+/// it resolves to the length-1 symbol for its low byte `lo` (if any). A length-2 symbol then
+/// overwrites the specific `hi:lo` slot for its exact two bytes, taking precedence over the
+/// length-1 fallback. A key whose low byte has no length-1 symbol and no length-2 symbol resolves
+/// to [#NO_CODE] with length 0, telling the caller to escape that single byte.
+final class ShortCodeTable {
+
+    /// Number of slots — one per possible 16-bit (two-byte) key.
+    private static final int SLOTS = 1 << 16;
+
+    /// Sentinel code meaning "no symbol matched"; the caller must escape the current byte. Real
+    /// codes are `0..254` (`0xFF` is the escape), so `-1` can never collide with a real code.
+    static final int NO_CODE = -1;
+
+    /// Packed `code << 8 | length` per 16-bit key. A zero length marks "no match".
+    private final int[] slots;
+
+    private ShortCodeTable(int[] slots) {
+        this.slots = slots;
+    }
+
+    /// Builds the table from symbols in descending-gain order, keeping only the length-1 and
+    /// length-2 entries. The list index is each symbol's code, matching the parallel-array
+    /// convention [Decompressor#of(long[], int[])] uses (array index is the code).
+    ///
+    /// Length-1 symbols are seeded first across all 256 high-byte keys that share their low byte,
+    /// so any two-byte prefix falls back to its low byte's single-byte code; length-2 symbols then
+    /// overwrite their exact key, taking precedence. Input order beyond that does not matter here:
+    /// a length-2 symbol owns a unique key, so there is no gain-order contention within this table
+    /// (unlike [LossyPerfectHashTable], where collisions make insertion order load-bearing).
+    ///
+    /// @param symbolsByGainDescending the trained symbols, code = list index, gain-descending
+    /// @return a table resolving 0/1/2-byte matches for any two-byte input prefix
+    static ShortCodeTable of(List<Symbol> symbolsByGainDescending) {
+        int[] slots = new int[SLOTS];
+        for (int code = 0; code < symbolsByGainDescending.size(); code++) {
+            Symbol symbol = symbolsByGainDescending.get(code);
+            if (symbol.length() == 1) {
+                int low = symbol.byteAt(0) & 0xFF;
+                int packed = code << 8 | 1;
+                for (int high = 0; high < 256; high++) {
+                    int key = high << 8 | low;
+                    if (length(slots[key]) == 0) {
+                        slots[key] = packed;
+                    }
+                }
+            }
+        }
+        for (int code = 0; code < symbolsByGainDescending.size(); code++) {
+            Symbol symbol = symbolsByGainDescending.get(code);
+            if (symbol.length() == 2) {
+                int key = (int) (symbol.packedBytes() & 0xFFFF);
+                slots[key] = code << 8 | 2;
+            }
+        }
+        return new ShortCodeTable(slots);
+    }
+
+    /// Returns the code matched by the low two bytes of `word`, or [#NO_CODE] if none. One array
+    /// read, no branching.
+    ///
+    /// @param word an input word; only its low 16 bits (first two input bytes) are consulted
+    /// @return the matched symbol code, or [#NO_CODE] when there is no length-1 or length-2 match
+    int codeFor(long word) {
+        int packed = slots[(int) (word & 0xFFFF)];
+        return packed == 0 ? NO_CODE : packed >>> 8;
+    }
+
+    /// Returns the length of the symbol matched by the low two bytes of `word`: 2, 1, or 0 for no
+    /// match. One array read, no branching.
+    ///
+    /// @param word an input word; only its low 16 bits (first two input bytes) are consulted
+    /// @return the matched symbol length in bytes, or 0 when there is no match
+    int lengthFor(long word) {
+        return length(slots[(int) (word & 0xFFFF)]);
+    }
+
+    private static int length(int packed) {
+        return packed & 0xFF;
+    }
+}