Add attribute discovery and filtering to world.analytics#2903
Conversation
🦋 Changeset detectedLatest commit: 716b6ee The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (5)1f47722Mon, 13 Jul 2026 20:41:46 GMT · run logs
fe54d70Mon, 13 Jul 2026 20:18:59 GMT · run logs
e19f7b3Mon, 13 Jul 2026 19:45:11 GMT · run logs
369fe23Mon, 13 Jul 2026 19:22:38 GMT · run logs
9d11b05Mon, 13 Jul 2026 18:39:32 GMT · run logs
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader) Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request. |
Not a derived requirement: the agent-runs UI filters by known constant values and reads per-run values via batch attribute fetches; it never enumerates distinct values for a key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
15c2be6 to
369fe23
Compare
| const world = await getWorld(); | ||
|
|
||
| // Which attribute keys exist, and on how many runs? | ||
| const keys = await world.analytics.attributes.list(); |
There was a problem hiding this comment.
World.analytics is optional, but this snippet dereferences it without the runtime check claimed by the @skip-typecheck annotation. It will throw in local/Postgres/custom Worlds where analytics is absent. Please feature-detect here (as the new Analytics reference page does) before calling attributes.list() or runs.list().
There was a problem hiding this comment.
Fixed in e19f7b3. The snippet now feature-detects: if (!world.analytics) { throw new Error(...) } before any analytics call, and the prose above it states the namespace is optional and absent on local/Postgres/custom Worlds. Since the guard makes the block genuinely type-safe, I also removed the @skip-typecheck annotation — the block is now typechecked by test:docs (skipped count dropped by one, all green).
The snippet dereferenced the optional analytics namespace without the runtime check its skip-typecheck annotation claimed, and would throw on local/Postgres/custom Worlds. Guard it and drop the annotation — the block is now genuinely typechecked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Behavior list still claimed a query API was only planned, directly contradicting the Searching and filtering section above it. State what is implemented: attributes are readable on run objects everywhere, and key discovery / key=value run filtering are available through the optional Analytics API. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Analytics page sits under workflow/runtime > World SDK, but neither the API reference index card, the workflow/runtime World SDK card, nor the World SDK overview mentioned analytics — making the new reference effectively undiscoverable from /docs/api-reference. Mention it at each level of the path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TooTallNate
left a comment
TooTallNate
left a comment
There was a problem hiding this comment.
Reviewed the API surface, wire encoding, schemas, and docs; ran the suites and the docs typecheck. Approving, with one small ask below.
What I verified:
- Wire encoding: the JSON-encoded
attributesparam is the right call (keys/values are arbitrary user strings that can contain=/,), the empty-object omission is handled and tested, and the both-or-neither window contract matches the existingruns.listconvention. The newAnalyticsAttributeKeySchemacoercions (z.coerce.number/z.coerce.date) match the backend's stringly-typed row output. - Semantics documented honestly: latest-write-wins matching (a run whose attribute moved
v1→v2no longer matchesv1) and the reserved-$-keys-readable-but-not-writable asymmetry are both spelled out in the interface docs and the guide, and both match the backend behavior. - Docs: the new
world/analytics.mdxreference page is accurate against the implementation (8-pair cap, return shapes, plan-window/pageInfosemantics), and it closes a real gap — the namespace was previously undocumented while the stale "in the future…" line sat in the attributes guide. Docs typecheck passes (1021) with the ambientworldglobal gaining the namespace. - Tests: world-vercel 238/238, world 73/73 locally; CI fully green.
One ask before merge: bump the changeset to minor. This adds new public API surface (analytics.attributes.list(), the attributes filter param, a new exported schema/type) — that's a feature, not a fix, and per the repo's changeset conventions the bump type is what carries semantic meaning through to any future backport. Trivial edit.
One note, no action needed: the Analytics interface gains attributes as a required property, which is a compile-time break for any out-of-tree implementer of the namespace. In-tree only world-vercel implements it and analytics stays optional on World, so this is fine on the beta line — just flagging that once the interface is considered stable, additions like this should probably land optional-first.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
No backport to This commit extends the To override, re-run the Backport to stable workflow manually via |
2 participants
Summary
Extends the optional
world.analyticsnamespace with attribute APIs:analytics.attributes.list(params?)— lists the distinct attribute keys observed on runs in the window, withrunCount,firstSeenAt,lastSeenAt. OptionalworkflowNamefilter andstartTime/endTimewindow; alphabetical keyset pagination. HitsGET /v2/analytics/attributes.analytics.runs.list({ attributes: { key: value } })— restricts the runs listing to runs whose latest attribute snapshot matches every provided pair (AND semantics, up to 8 pairs). Serialized as a single JSON-encodedattributesquery param because attribute keys/values are arbitrary user strings that may contain=or,. Reserved$-prefixed keys are allowed in read filters (they're rejected only on user writes).Docs
The
world.analyticsnamespace was previously undocumented. This PR adds a World SDK reference page (docs/.../world/analytics.mdx) covering the whole namespace — runs, attributes, steps/events/hooks/waits, lookback windows, andpageInfo— wires it into the World SDK index, and replaces the stale "in the future you'll be able to search and filter runs by attributes" line in the attributes guide with a filtering example. The docs-typecheck ambientworldglobal gains the analytics namespace so snippets typecheck.Notes
analyticsremains optional onWorldand onlyworld-vercelimplements it, so no changes to local/postgres worlds.AnalyticsAttributeKeySchemaexported from@workflow/worldalongside the existing analytics schemas.observability-upgrade-required.listValues(distinct values per key) API was considered and dropped — not a derived requirement.Testing
packages/world-vercel/src/analytics.test.tscovering the JSON filter serialization (incl. empty-object omission) and keys-listing query params.pnpm --filter @workflow/world-vercel test,packages/worldvitest, fullpnpm build,pnpm test:docs(1020 passed), and the docs link lint all green.🤖 Generated with Claude Code