Skip to content

Add opt-in snapshot policies#14

Merged
roodboi merged 5 commits into
codex/extract-schema-stream-v4from
codex/snapshot-policies
Jul 11, 2026
Merged

Add opt-in snapshot policies#14
roodboi merged 5 commits into
codex/extract-schema-stream-v4from
codex/snapshot-policies

Conversation

@roodboi

@roodboi roodboi commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Add opt-in snapshot cadence controls to parse() and iterate() while keeping the v4 per-input-chunk behavior as the default.

  • add chunk, completed-value, byte-threshold, and final snapshot policies
  • preserve trailing-content rejection, parser errors, cancellation, backpressure at emission boundaries, and immutable iterator snapshots
  • cover policy behavior across Bun, packed Node ESM/CommonJS consumers, Zod 3, Zod 4, and Zod Mini
  • add a deterministic large-payload benchmark and focused policy documentation
  • harden small-buffer UTF-8 and escaped UTF-16 surrogate handling
  • replace linear completion-path deduplication with an order-preserving Set
  • finish standalone-repository contributor guidance and the TypeScript 7 declaration build, while verifying published declarations with TypeScript 5.9
  • harden GitHub Actions with immutable action SHAs, read-only publish repository access, non-persisted checkout credentials where safe, and workflow security regression tests
  • migrate formatting and linting from Prettier to Ultracite/Biome, apply the full safe/unsafe fix pass, and document narrow parser/test invariants where generic rules do not apply
  • prevent empty sources from emitting a successful-looking schema stub while preserving valid empty-container snapshots across every policy

Why

Serializing cumulative JSON after every network chunk can dominate allocation and emitted-byte volume for large structured responses. Consumers that do not need chunk-level updates can now select a lower-frequency policy without changing the existing default or final result.

In the 10.25 MB benchmark, default chunk mode emitted 165 snapshots / 855.88 MB, while final mode emitted one 10.25 MB snapshot. The benchmark reports counts and throughput without fragile timing assertions.

Security review

A focused review of all three workflows found no path that exposes NPM_TOKEN to fork or unmerged pull-request code and no step that prints secret values. It did identify mutable action tags in privileged release jobs; all action references are now pinned to reviewed full commit SHAs. The publish job now uses contents: read, does not persist checkout credentials, and retains its exact merged/canonical-repository/release-branch gates.

The remaining external hardening options are to configure npm trusted publishing before replacing NPM_TOKEN with OIDC, add protection rules to the PUBLISH environment, and enable repository-level SHA-pin enforcement after these pinned workflows reach the base branch.

Validation

  • bun install --frozen-lockfile
  • bun x ultracite doctor — 4 passed, 0 warnings, 0 failed
  • bun run lint — 28 files checked, zero diagnostics
  • bun run format followed by bun run lint — idempotent, no remaining fixes
  • bun run test:coverage — 42 tests, 382 assertions; all source 90.48% line coverage and streaming parser 95.86%
  • bun run check — Ultracite formatting/linting, TypeScript 7 type-check, tests, build, and packed consumer matrix
  • packed TypeScript 5.9 declarations with Node ESM/CommonJS, Zod 3, Zod 4/Mini, OpenAI Agents SDK, and Vercel AI SDK
  • bun audit --production --audit-level=high — no vulnerabilities
  • workflow policy tests cover immutable action pins, pull-request token/secret boundaries, exact publish authorization, least privilege, and canonical release automation
  • bun run changeset status — minor release recognized
  • bun run benchmark:snapshots 10 — 10.25 MB, 165/5/42/11/1 snapshots across the policy matrix

Stack

This PR is based on and should merge after #13, which establishes the published v4 code as the standalone repository source of truth.

@roodboi
roodboi marked this pull request as ready for review July 10, 2026 23:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82b9edcfd5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/utils/streaming-json-parser.ts Outdated
@roodboi
roodboi merged commit 7107d62 into codex/extract-schema-stream-v4 Jul 11, 2026
4 checks passed
@roodboi
roodboi deleted the codex/snapshot-policies branch July 11, 2026 00:09
@github-actions github-actions Bot mentioned this pull request Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant