Release: turbovec 0.9.0 (Rust crate) + 0.8.0 (Python package)#109
Merged
Conversation
Security-audit release (#108): untrusted-load hardening, binding panic fixes, integration data-integrity fixes, and the x86 scalar-fallback correctness fix. Resolves #104, #105, #106. Minor bump on both surfaces because a few inputs that previously panicked or were silently accepted now return typed errors. See CHANGELOG.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fact-checked the README prose against benchmarks/results/. Several figures had drifted (mostly from before TQ+ landed): - ARM speed 12-20% -> 10-19% (actual range 10.3-19.4%) - OpenAI R@1 +0.4-3.4 pts -> +0.2-1.9 pts (no config reaches 3.4) - GloVe R@1 +0.3/-1.2 -> +0.9/tied (TQ+ closed the 2-bit gap) - x86 2-bit 'within ~1% ST, 2-4% MT' -> trails 3-8% on both ST and MT - softened 'matches the Shannon lower bound' -> 'near-optimal' (the how-it-works section already states within 2.7x of the bound) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- intro: 'matches the Shannon lower bound' -> 'near-optimal distortion'; trim to 'no separate training phase' - speed bullet/x86 prose: characterize x86 2-bit honestly (behind, most visibly d=1536 ST ~8%) instead of 'match-or-beat'/'3-8%' - recall: OpenAI converge to 1.0 by k=8 (>=0.997 at k=4) - baselines note: drop stale 'visible gap on GloVe' (TQ+ closed it; GloVe is now level at 2-bit, ahead at 4-bit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audited docs/ against the merged behavior changes: - api.md: document that search() raises ValueError on non-finite/oversized query coords; dim must be a positive multiple of 8 and <= 65536 (MAX_DIM); zero-width add raises; load now validates the header before allocating. - agno.md: duplicate derived doc_id is now keep-all (both kept and deletable, matching LanceDb) — previous text implied last-write-wins; clarify delete_by_name/_content_id/_metadata target only matching docs. - agno/langchain/haystack/llama_index: loading a side-car out of sync with its .tvim now raises ValueError at load instead of a later KeyError. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adding the DimTooLarge variant this release is already breaking for downstream exhaustive matches. Mark both public error enums #[non_exhaustive] now so future variant additions stop being breaking changes — this release is the one-time free moment to do it. The Python binding only uses Display (e.to_string()), so it is unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
turbovec's usize size/offset arithmetic in encode/pack/search assumes a 64-bit pointer width; on 32-bit/wasm those products can overflow and index out of bounds. The untrusted-load path is already gated by checked_mul in io.rs, so this is not a vulnerability via file input — but a developer adding very large data on a 32-bit target could still overflow. Refuse to compile on non-64-bit targets rather than ship a silently-unsafe build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Route security reports through GitHub private vulnerability reporting / Security Advisories instead of public issues, with reporting steps, what to include, expectations, supported-version policy, and scope. #105 came in as a public issue; this gives finders a private channel. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Private vulnerability reporting is the single channel; no alternate contact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
a-tokyo
added a commit
to a-tokyo/turbovec
that referenced
this pull request
Jun 13, 2026
…ure/ts Sync the Node-bindings branch onto the latest upstream before proposing it upstream. Upstream's RyanCodrai#108 security audit independently hardened the same core files this branch had touched, and RyanCodrai#109 cut crate 0.9.0 / Python 0.8.0. Reconciliation: - turbovec/src/io.rs, lib.rs: take upstream's hardening wholesale; drop our now -redundant copies. Removed our duplicate `pub use io::MAX_DIM` (upstream defines `pub const MAX_DIM` at the crate root); upstream already exposes `MAX_INPUT_MAGNITUDE` as `pub` for the bindings. Net core delta from this branch is now ~zero — the package is essentially additive. - turbovec-node/src/error.rs + ts/errors.ts + docs/api.md: handle upstream's new `DimTooLarge` variants (both enums are now `#[non_exhaustive]`) via a new `DIM_TOO_LARGE` JS error code, plus a wildcard arm so future core variants don't break the binding build. - turbovec/tests/io_versioning.rs: the lazy-header regression test now anchors on upstream's wording (clean InvalidData on a dim=0/n>0 header is unchanged). - CHANGELOG/docs/api.md: keep our Node section under Unreleased alongside the 0.9.0/0.8.0 release entry and upstream's richer Python tables. Verified against the synced core: turbovec-node clippy clean; 193 vitest pass; cargo test -p turbovec green (incl. io_versioning).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release turbovec 0.9.0 (Rust crate) + 0.8.0 (Python package) — the security-audit release (#108 fixes).
Version bumps
turbovec/Cargo.toml: 0.8.1 → 0.9.0turbovec-python/Cargo.toml+pyproject.toml: 0.7.1 → 0.8.0Minor bump on both because a few inputs that previously panicked or were silently accepted now return typed errors (NaN/Inf search →
ValueError,dimcapped at 65536, zero-dim lazy add rejected, side-car/index mismatch raises at load). Full details inCHANGELOG.md.Also in this PR
benchmarks/results/— ARM speed (12–20% → 10–19%), OpenAI R@1 (+0.4–3.4 → +0.2–1.9 pts), GloVe R@1 (TQ+ closed the 2-bit gap), x86 2-bit speed (trails 3–8%, not "within 1%"), and softened "matches the Shannon lower bound" to "near-optimal". The charts are data-generated and were already correct; only the prose had drifted.Publishing
The release workflows trigger on tags after this merges:
v0.9.0→release-crates.yml(crates.io)py-v0.8.0→release-pypi.yml(PyPI)I have not pushed those tags — say the word once this is merged and I'll push them (or you can). No on-disk format change (still
.tv/.tvimv3).