docs: update CLAUDE/SECURITY/CHANGELOG for proto-rewrite

CLAUDE.md:
- regenerate-sources flow: drop `brew install protobuf`, add
  `./mvnw compile -pl proto-gen` pre-step
- metadata-only encoding example uses `.encode()` not `.toByteArray()`
  and includes the decode-side snippet
- note `ProtoReader`/`ProtoWriter` are package-private + the
  `ofXxxValue` factory pattern for oneof messages

SECURITY.md:
- update threat-model intro to reference the in-tree `ProtoReader`
  instead of `protobuf-java`
- supported versions bumped: 0.6.x supported, 0.5.x critical-only,
  < 0.5 EOL
- drop `protobuf-java` from out-of-scope deps; add ProtoReader hardening
  to defensive guarantees (varint cap, truncated-len-delim, bounds)
- exception list: `IOException` from proto reader replaces
  "raw Protobuf parser exceptions"

CHANGELOG.md:
- new [0.6.0] section: proto-rewrite summary, ProtoReader/Writer,
  proto-gen module, oneof factories, dependency removal
  (`protobuf-java`), wire-format compatibility evidence (Rust
  integration tests), perf note (within noise on bulk reads)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: dfa1

CHANGELOG.md

@@ -5,6 +5,70 @@ All notable changes to **vortex-java** are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.0] — Unreleased
+
+The headline theme is the **proto-rewrite**: `protobuf-java` is dropped in favour of an
+in-tree MemorySegment-native proto3 codec, generated from `.proto` schemas by a new
+`proto-gen` module. CLI uber-jar shrinks ~14% and the JDK 25 `sun.misc.Unsafe` stderr
+warning (emitted by `protobuf-java`'s `UnsafeUtil`) is gone.
+
+### Added
+
+- **`proto-gen` module** — build-time `.proto` to Java code generator. Lexer + parser +
+  type registry + emitter. Outputs one immutable Java `record` per message and one Java
+  `enum` per proto enum, each carrying a `@Generated("io.github.dfa1.vortex.protogen.CodeGen")`
+  annotation. Records expose `decode(MemorySegment, long, long)` static factories and
+  `encode()` instance methods that operate directly on a memory segment — zero `byte[]`
+  copy, no `protobuf-java` runtime.
+- **`ProtoReader` / `ProtoWriter`** — package-private proto3 wire-format primitives
+  under `io.github.dfa1.vortex.proto`. Reads varint / sint64 / fixed32 / fixed64 /
+  length-delimited / packed-repeated payloads, with bounds checks and a 10-byte cap on
+  varint length. 42 unit tests cover happy path + truncation + bounds.
+- **Oneof factories** on generated records (e.g. `ScalarValue.ofInt64Value(123L)`) —
+  avoids the 11-arg constructor for `ScalarValue`'s oneof.
+- **`PatchedMetadata` / `VariantMetadata`** — added to `encodings.proto`. Previously
+  hand-parsed with `CodedInputStream`; now go through the generated record path.
+
+### Changed
+
+- **Build-time tooling**: `regenerate-sources` profile no longer shells out to `protoc`.
+  Run `./mvnw compile -pl proto-gen` once, then
+  `./mvnw generate-sources -pl core -P regenerate-sources`. `brew install protobuf` is
+  no longer needed for normal development.
+- **Encoding consumers**: 25 encoding classes (`ALP`, `Bitpacked`, `Dict`, `Rle`,
+  `Sparse`, `Sequence`, etc.) and 23 test files rewritten to use the new record API.
+  Constructor calls are positional; field accessors follow proto3 snake_case
+  (`meta.bit_width()`, not `meta.getBitWidth()`).
+
+### Removed
+
+- **`com.google.protobuf:protobuf-java`** dependency dropped from `core`, `reader`,
+  `writer`, and root `dependencyManagement`. The `protobuf.version` property is gone.
+  CLI uber-jar: **14 MB → 12 MB**. JDK 25 `sun.misc.Unsafe::arrayBaseOffset` stderr
+  warning emitted by `UnsafeUtil` on every cold start: **gone**.
+- `protoc` no longer required by the build. `brew install flatbuffers` covers `.fbs`
+  edits; `.proto` edits use the in-process generator.
+
+### Compatibility
+
+Wire-format compatibility with the Rust reference implementation is unchanged and is
+verified by the full integration suite:
+
+- `RustWritesJavaReadsIntegrationTest` (10 tests) — Rust writes, Java reads
+- `JavaWritesRustReadsIntegrationTest` (194 tests) — Java writes, JNI reads
+- `RustJavaReaderComparisonIntegrationTest` (25 tests) — both readers, same file
+- `ParquetImportIntegrationTest` (5 tests) — round-trip through ParquetImporter
+
+All 872 unit + 243 integration tests pass on JDK 25.
+
+### Performance
+
+No measurable change on bulk-read benchmarks (`RustVsJavaReadBenchmark.javaReadCascading`
+within 1% of main, stdev ±2 ops/s). Proto metadata parse is < 1% of work on multi-million-row
+scans; the win is architectural, not throughput.
+
+[0.6.0]: https://github.com/dfa1/vortex-java/compare/v0.5.0...main
+
 ## [0.5.0] — 2026-06-09
 
 The headline themes are an **interactive inspector TUI** for navigating Vortex files

CLAUDE.md

@@ -18,15 +18,24 @@ Never use `mvn install` or `./mvwn install`.
 
 Generated sources (`fbs`/`proto` → Java) are committed under `core/src/main/java`.
 Normal builds need no external tools.
+
+Proto-to-Java generation is in-process via the `proto-gen` module (no `protoc` needed).
+The generator emits one record per message with a {@code decode(MemorySegment, long, long)} static
+factory and an {@code encode()} method that operate directly on a memory segment — no `byte[]`
+copy, no `protobuf-java` runtime, no `sun.misc.Unsafe`.
+
 To regenerate after editing `.fbs` or `.proto` schemas:
 
 ```bash
-brew install flatbuffers protobuf
+brew install flatbuffers              # only needed for .fbs edits
+./mvnw compile -pl proto-gen          # build the proto generator (only on .proto edits)
 ./mvnw generate-sources -pl core -P regenerate-sources
 # then commit the updated files
 ```
 
 Any `flatc` version works — the profile strips the version guard automatically.
+`flatc` runs every time the profile is active; if you only changed `.proto` files, revert any
+spurious `fbs/` diffs with `git checkout -- core/src/main/java/io/github/dfa1/vortex/fbs/`.
 
 ```bash
 # Build all modules
@@ -276,18 +285,26 @@ Simple encodings (≤ ~80 lines total, e.g. `NullEncoding`, `BoolEncoding`) are
 
 ### Metadata-only encodings
 
-Some encodings store all data in protobuf metadata — no buffers, no children (e.g. `SequenceEncoding`).
+Some encodings store all data in proto3 metadata — no buffers, no children (e.g. `SequenceEncoding`).
 Their `EncodeResult` uses an `EncodeNode` with `metadata` set and an empty `bufferIndices` array:
 
 ```java
-ByteBuffer metaBuf = ByteBuffer.wrap(meta.toByteArray());
+ByteBuffer metaBuf = ByteBuffer.wrap(meta.encode());
 EncodeNode node = new EncodeNode(encodingId, metaBuf, new EncodeNode[0], new int[]{});
-return new
+return new EncodeResult(node, List.of(), null, null);
+```
 
-EncodeResult(node, List.of(), null,null);
+The decoder reads back via `ctx.metadata()`, not `ctx.buffer(n)`:
+
+```java
+MemorySegment metaSeg = MemorySegment.ofBuffer(ctx.metadata().duplicate());
+FooMetadata meta = FooMetadata.decode(metaSeg, 0, metaSeg.byteSize());
 ```
 
-The decoder reads back via `ctx.metadata()`, not `ctx.buffer(n)`.
+Generated proto records live in `io.github.dfa1.vortex.proto`. The runtime decoder
+(`ProtoReader`, `ProtoWriter`) is package-private — generated code calls it directly.
+For oneof messages (e.g. `ScalarValue`), prefer the static `ofXxxValue(v)` factory over
+the 11-arg constructor.
 
 ## Testing
 

SECURITY.md

@@ -1,9 +1,10 @@
 # Security Policy
 
 `vortex-java` reads and writes the [Vortex columnar file format](https://github.com/vortex-data/vortex).
-The reader memory-maps and parses untrusted binary input — trailers, FlatBuffers, Protobuf
-metadata, and per-segment encoded data. Robustness against malformed input is treated as a
-correctness contract, not a best-effort feature.
+The reader memory-maps and parses untrusted binary input — trailers, FlatBuffers, proto3
+metadata (via the in-tree MemorySegment-native `ProtoReader` — no `protobuf-java` runtime),
+and per-segment encoded data. Robustness against malformed input is treated as a correctness
+contract, not a best-effort feature.
 
 ## Supported versions
 
@@ -12,9 +13,9 @@ only if the vulnerability is critical and the fix is mechanical.
 
 | Version | Status                  |
 | ------- | ----------------------- |
-| 0.4.x   | Supported               |
-| 0.3.x   | Critical fixes only     |
-| < 0.3   | End of life             |
+| 0.6.x   | Supported               |
+| 0.5.x   | Critical fixes only     |
+| < 0.5   | End of life             |
 
 ## Reporting a vulnerability
 
@@ -46,7 +47,7 @@ In scope:
 - Any malformed `.vortex` input that causes the reader to throw an exception other than
   `io.github.dfa1.vortex.core.VortexException` (e.g. `IndexOutOfBoundsException`,
   `NegativeArraySizeException`, `OutOfMemoryError`, `StackOverflowError`, raw FlatBuffer
-  runtime exceptions, raw Protobuf parser exceptions, or a JVM crash via the FFM layer).
+  runtime exceptions, raw `IOException` from the proto3 reader, or a JVM crash via the FFM layer).
 - Any malformed `.vortex` input that causes the reader to allocate memory disproportionate
   to its on-disk size (zip-bomb-style amplification).
 - Any malformed `.vortex` input that causes silent data corruption — wrong row count,
@@ -58,9 +59,10 @@ Out of scope:
 
 - Denial of service from legitimately large inputs (multi-gigabyte files). Use the
   resource caps in `ReadOptions` (planned) to bound them.
-- Vulnerabilities in third-party dependencies (`vortex-jni`, `zstd-jni`, FlatBuffers runtime,
-  Protobuf runtime). Report those upstream; we'll bump the dependency once a fixed version
-  is available.
+- Vulnerabilities in third-party dependencies (`vortex-jni`, `zstd-jni`, FlatBuffers runtime).
+  Report those upstream; we'll bump the dependency once a fixed version is available.
+  Vortex no longer depends on `protobuf-java` — proto3 parsing is handled by the in-tree
+  `ProtoReader` (issues there are in scope).
 - Performance regressions or correctness bugs unrelated to malformed input — please open
   a regular issue.
 
@@ -76,9 +78,12 @@ exception**. Concretely:
   self-referential FlatBuffer cycles).
 - Layout metadata is capped at 4 MiB.
 - `Decimal` precision is restricted to `[1, 38]`; `scale` to `[0, precision]`.
-- `PType` ordinals from Protobuf are bounds-checked.
+- `PType` ordinals from proto3 are bounds-checked.
 - `ConstantEncoding` and dict-layout decode allocate `O(1)` memory regardless of the
   declared row count (zip-bomb mitigation).
+- `ProtoReader` enforces varint length ≤ 10 bytes, rejects truncated len-delim regions,
+  and validates segment bounds on every read. (0.6.0+ — replaces the `protobuf-java`
+  parser path; same exception contract.)
 
 The regression suite lives under `reader/src/test/java/.../*SecurityTest`. Run with
 `./mvnw test -Dtest='*SecurityTest'`.