Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/calm-snapshots-flow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"schema-stream": minor
---

Add opt-in completed-value, byte-threshold, and final-only snapshot policies to `parse()` and `iterate()` while preserving per-input-chunk emission as the default.
20 changes: 12 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,25 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false

- name: Setup Bun
uses: oven-sh/setup-bun@v2
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Check formatting
run: bun run format:check
- name: Check formatting and linting
run: bun run lint

- name: Type-check
run: bun run type-check
Expand All @@ -59,15 +61,17 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
persist-credentials: false

- name: Setup Bun
uses: oven-sh/setup-bun@v2
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: ${{ matrix.node-version }}

Expand Down
10 changes: 6 additions & 4 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,24 +20,26 @@ jobs:
runs-on: ubuntu-latest
environment: PUBLISH
permissions:
contents: write
contents: read

steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0
persist-credentials: false

- name: Setup Bun
uses: oven-sh/setup-bun@v2
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24
registry-url: https://registry.npmjs.org
package-manager-cache: false

- name: Install dependencies
run: bun install --frozen-lockfile
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,25 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v7
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
fetch-depth: 0

- name: Setup Bun
uses: oven-sh/setup-bun@v2
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version: 1.3.14

- name: Setup Node.js
uses: actions/setup-node@v6
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 24

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Open or update the release PR
uses: changesets/action@v1
uses: changesets/action@a45c4d594aa4e2c509dc14a9f2b3b67ba3780d0d # v1.9.0
with:
version: bun run version
env:
Expand Down
19 changes: 19 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,16 @@ dist
.env
.env.*
!.env.example
.mise.local.toml

# Local credentials and signing material
*.key
*.p12
*.pfx
*.crt
*.cer
*.mobileprovision
*.secrets.*

# Package-manager and debug output
npm-debug.log*
Expand All @@ -20,3 +30,12 @@ yarn-error.log*
.idea
.vscode
*.pem

# Local agent and profiling state
.agents/
.claude/
.codex/
*.cpuprofile
*.heapprofile
*.heapsnapshot
*.trace
4 changes: 0 additions & 4 deletions .prettierignore

This file was deleted.

7 changes: 0 additions & 7 deletions .prettierrc.json

This file was deleted.

2 changes: 0 additions & 2 deletions .tool-versions

This file was deleted.

87 changes: 87 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Schema Stream Agent Guide

## Project contract

Schema Stream is a small, public, standalone TypeScript library for progressively parsing streamed JSON into typed, schema-shaped snapshots. Keep changes focused on the library, its tests, its documentation, and its release tooling.

- Treat this as a public OSS repository. Never add private URLs, credentials, customer data, internal project names, local absolute paths, transcripts, or personal information beyond the public package metadata already committed.
- Preserve consumer compatibility across Node.js and Bun, ESM and CommonJS, Zod 3, Zod 4, and Zod Mini.
- Do not edit generated output in `dist/`; regenerate it with `bun run build`.
- Prefer small, reviewable changes. Preserve unrelated user changes in a dirty worktree.

## Toolchain

Use the versions declared by the repository:

- `mise install` provisions Bun 1.3.14 and Node.js 24.
- `bun install` installs dependencies from `bun.lock`.
- TypeScript 7 is the authoritative development compiler.
- Published declarations must remain consumable by supported TypeScript 5.x projects; `bun run test:packed` verifies this with TypeScript 5.9.
- Ultracite with Biome is the repository's formatter and linter. Keep `biome.jsonc`, package scripts, CI, and existing source synchronized when changing formatting or lint rules.

Run repository scripts through Bun:

```sh
bun run format
bun run format:check
bun run lint
bun run lint:fix
bun run type-check
bun run test
bun run test:coverage
bun run build
bun run test:packed
bun run check
```

`bun run check` is the minimum completion gate. It includes Ultracite linting and formatting checks. Run `bun run build` when exports, declarations, build configuration, or package metadata change. Run `bun run test:coverage` when behavior changes.

## Code style and architecture

- Use strict TypeScript. Never introduce `any`; use `unknown`, narrowing, discriminated unions, generics, and schema-derived types.
- Preserve precise consumer inference. Avoid widening literals, unnecessary assertions, broad index signatures, or generic defaults that erase information.
- Prefer named options objects when a function has multiple inputs or is likely to grow. Do not churn a stable public signature solely to satisfy this preference.
- Prefer focused functions, composition, early returns, immutable values, and clear module boundaries.
- Use function declarations for named reusable functions and arrow functions for callbacks or concise local closures.
- Avoid hidden mutable global state and unnecessary allocation in streaming hot paths.
- Keep parser state transitions explicit. Preserve chunk-boundary correctness, incremental UTF-8 decoding, backpressure, cancellation, and useful error context.
- Do not add dependencies when a platform primitive or a small local implementation is sufficient.
- Follow the configured format: two spaces, double quotes, no semicolons, and a 100-column print width.

## Documentation and comments

- Add useful TSDoc block comments to public classes, functions, methods, exported types, and non-obvious internal boundaries.
- Describe behavior, invariants, streaming semantics, failure modes, and inference guarantees—not a restatement of the identifier.
- Include `@param`, `@returns`, `@throws`, `@typeParam`, or examples when they add information for consumers or maintainers.
- Avoid narration-style inline comments. Use an inline comment only for a subtle invariant, protocol detail, compatibility workaround, or actionable TODO with context.
- Update README or focused files under `docs/` when public behavior, configuration, performance tradeoffs, or compatibility changes.
- Keep examples executable, type-correct, and representative of the published API.

## Testing expectations

Behavior changes should include the narrowest regression test plus broader coverage where the risk warrants it. Exercise relevant combinations of:

- chunks split at every meaningful token and UTF-8 boundary;
- empty, partial, malformed, deeply nested, and very large JSON values;
- objects, arrays, primitives, nullable and optional fields, unions, records, and recursive schemas;
- snapshot policies, final snapshot behavior, cancellation, and parser errors;
- Zod 3, Zod 4, and Zod Mini;
- ESM, CommonJS, Node.js, Bun, and packed-package declaration consumption;
- sustained high-throughput streams and allocation-sensitive paths.

Performance tests must be deterministic enough to detect major regressions without asserting fragile machine-specific timings. Prefer reporting throughput, snapshot count, and peak-memory-relevant behavior; keep correctness assertions alongside benchmarks.

## Public API and releases

- Treat every export and emitted declaration as public API.
- Favor additive changes. Clearly document intentional breaking changes and add a changeset with the appropriate version bump.
- Keep `package.json` exports, `files`, runtime requirements, README installation guidance, and packed-consumer tests synchronized.
- Verify the packed tarball rather than assuming a successful source build proves publishability.

## Repository map

- `src/`: library source and public entry points
- `tests/`: unit, integration, benchmark, and packed-consumer verification
- `docs/`: focused design and behavior documentation
- `.changeset/`: release notes and version intent
- `.github/workflows/`: CI and release automation
3 changes: 3 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Claude Instructions

Read and follow [`AGENTS.md`](./AGENTS.md) as the repository's canonical agent and contributor guidance. Keep this file as a pointer so instructions do not drift between tools.
44 changes: 44 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Contributing to Schema Stream

Thanks for helping improve Schema Stream.

## Setup

Install the repository toolchain with [mise](https://mise.jdx.dev/) and install dependencies with Bun:

```sh
mise install
bun install
```

The repository develops with Bun 1.3.14, Node.js 24, and TypeScript 7. Consumers do not need TypeScript 7; the packed-package test verifies the published declarations with TypeScript 5.9.

## Development checks

Run the full local gate before opening a pull request:

```sh
bun run check
bun run build
```

For behavior changes, also inspect coverage:

```sh
bun run test:coverage
```

`bun run test:packed` builds and installs the package tarball into isolated ESM and CommonJS consumers. It verifies Zod 3, Zod 4, Zod Mini, OpenAI Agents SDK, Vercel AI SDK, and TypeScript 5.9 compatibility.

## Changes

- Include focused tests for fixes and new behavior.
- Preserve strong public type inference and supported runtime/schema compatibility.
- Add or update TSDoc for public APIs and document meaningful user-facing behavior.
- Add a changeset for changes that should appear in a release:

```sh
bun run changeset
```

See [`AGENTS.md`](./AGENTS.md) for the complete project conventions, testing matrix, and public-OSS privacy rules.
32 changes: 27 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,17 @@ for await (const partial of parser.iterate(response.body!)) {
across byte chunks, cancels the source when iteration ends early, and yields an independent snapshot
for every input chunk.

Snapshot cadence is opt-in and shared with `parse()`. Omitting the option retains one snapshot per
input chunk:

```typescript
parser.iterate(source, { snapshotPolicy: { mode: "value" } })
parser.iterate(source, { snapshotPolicy: { mode: "bytes", bytes: 256 * 1024 } })
parser.iterate(source, { snapshotPolicy: { mode: "final" } })
```

See [snapshot policies](./docs/snapshot-policies.md) for exact semantics and performance tradeoffs.

## OpenAI Agents SDK

Pass the Agents SDK text stream directly to `iterate()`:
Expand Down Expand Up @@ -180,18 +191,29 @@ for await (const bytes of snapshots) {
}
```

`parse()` accepts the same `snapshotPolicy` option as `iterate()`.

## Development

```bash
mise install
bun install
bun run type-check
bun run test:coverage
bun run build
bun run test:packed
bun run format
bun run lint
bun run check
```

`mise.toml` pins Bun 1.3.14 and the current Node 24 release. Maintainers type-check and emit
declarations with TypeScript 7.0.2. TypeScript is a development-only dependency, so installing
`schema-stream` does not install or require TypeScript 7.

Ultracite and Biome own formatting and linting. `bun run format` applies safe fixes, `bun run lint`
checks the repository without writing, and `bun run check` includes linting before type checks,
tests, and packed-consumer verification.

`test:packed` installs the generated tarball into clean consumers and verifies ESM, CommonJS,
Zod 4/Mini, Zod 3, OpenAI Agents SDK, and Vercel AI SDK compatibility without contacting a model.
Zod 4/Mini, Zod 3, OpenAI Agents SDK, and Vercel AI SDK compatibility with TypeScript 5.9 without
contacting a model. This protects the declaration surface from accidental TS7-only syntax.

## License

Expand Down
Loading