Skip to content

feat: add Pi session support (v2 subagent dirs + v3 native format)#155

Open
adamjramirez wants to merge 6 commits intoobsessiondb:mainfrom
adamjramirez:adamjramirez/pi-session-support
Open

feat: add Pi session support (v2 subagent dirs + v3 native format)#155
adamjramirez wants to merge 6 commits intoobsessiondb:mainfrom
adamjramirez:adamjramirez/pi-session-support

Conversation

@adamjramirez
Copy link
Copy Markdown

Summary

Adds support for pi sessions that were previously invisible to Rudel. Closes #153.

Before: 60 sessions visible (Claude Code only)
After: 1,188 sessions visible (Claude Code + Pi v2 + Pi v3)

Two Pi storage formats

Format Location Structure Count
Pi v2 ~/.claude/projects/<project>/<uuid>/subagents/agent-*.jsonl Same JSONL as Claude Code ~282
Pi v3 ~/.pi/agent/sessions/--<encoded-cwd>--/<timestamp>_<uuid>.jsonl Different schema (own event types, different usage field names) ~846

Approach

New PiAdapter following the existing adapter pattern (like Codex):

  • Registered via registerScanOnlyAdapter() — shares source: "claude_code" without colliding in the adapter registry. No SourceSchema changes needed.
  • v2 sessions: Reads subagent files, concatenates into content field, populates subagents map
  • v3 sessions: Transforms pi's native JSONL format into Claude Code-compatible JSONL so the ClickHouse materialized view can extract analytics (timestamps, tokens, model, errors, etc.)

v3 content transformation

The MV parses content line-by-line expecting Claude Code format (type: "user"/"assistant" at line level, message.usage.input_tokens, etc.). Pi v3 uses different structures, so the adapter transforms on upload:

  • {type: "message", message: {role: "user"}}{type: "user", ...}
  • Usage: inputinput_tokens, outputoutput_tokens, cacheReadcache_read_input_tokens, cacheWritecache_creation_input_tokens
  • Includes uuid and sessionId so the web UI conversation parser works
  • Strips v3-only fields (api, provider, stopReason, cost, totalTokens) to reduce content size
  • Skips non-interaction lines (session, compaction, thinking_level_change, toolResult)

Changes

File Change
packages/agent-adapters/src/adapters/pi/index.ts New — Pi adapter (v2 + v3 scanning, content transform)
packages/agent-adapters/src/registry.ts registerScanOnlyAdapter() for shared-source adapters
packages/agent-adapters/src/index.ts Export + register Pi adapter
apps/cli/src/lib/session-resolver.ts Resolve pi sessions by UUID or path (v2 dirs + v3 files)
apps/cli/src/commands/upload.ts Route to Pi adapter in all upload paths (interactive, single, retry)
apps/cli/src/commands/dev/list-sessions.ts Show [Pi] label for pi sessions
apps/cli/src/commands/enable.ts Handle Pi's no-op hooks gracefully
apps/cli/src/__tests__/agents.test.ts 24 tests covering v2 + v3 discovery, transform, upload

Zero API/schema/migration changes.

Tested

  • bun run check-types
  • bun run lint
  • bun test — 24 pass, 0 fail ✅
  • Real uploads of v2 and v3 sessions to app.rudel.ai — sessions appear with correct analytics, timestamps, conversation view ✅

Follow-up suggestion

Currently pi sessions show as source: "claude_code" since we reuse the existing source to avoid schema changes. A natural follow-up would be adding "pi" to SourceSchema so the UI can distinguish which harness produced a session.

Support pi (https://github.com/mariozechner/pi-coding-agent) sessions
that were previously invisible to Rudel. Handles two pi storage formats:

Pi v2 (~282 sessions): subagent-only UUID dirs under ~/.claude/projects/
Pi v3 (~846 sessions): native format under ~/.pi/agent/sessions/

Approach:
- New PiAdapter in packages/agent-adapters/src/adapters/pi/
- Registered via registerScanOnlyAdapter() to share source 'claude_code'
  without colliding in the adapter registry
- v3 content transformed to Claude Code JSONL format so the ClickHouse
  materialized view can extract analytics (timestamps, tokens, model, etc.)
- Session resolver updated to find pi sessions by UUID or path
- Upload command routes to Pi adapter via isPiSession() detection
- Enable command gracefully handles Pi's no-op hooks

Closes obsessiondb#153
@KeKs0r
Copy link
Copy Markdown
Contributor

KeKs0r commented Mar 20, 2026

Thanks a lot for this. I would love to enable you to also use sessions from Pi in Rudel.
There are some design decions we took, that are not fully reflected here, mainly due to lack of proper documentation. (Added to our todo).
For new agents, we would rather have new tables + parsing pipeline, instead of doing the parsing during the ingestion. So instead of mapping it to the claude code strucutre, I think we would rather embrace the Pi format, take it raw, this will allow us later, if Pi adds feature, to potentially extract insights or things from it, that would have gotten lost.
I think we should have a new pi sessions table, based on the base table, and probably have a version column additinoally. so we can distuinghis v2 vs v3. and then have 2 mvs that extract them accordingly. The one for v2, would mirror the one from claude code, or be a copy of it. And the one for v3, would encompass the transforms that are currently done during ingestion.

Let me know if you need extra help, or which type of problems you run into during building. The docs and system are still a little bit rough around the edges.

@adamjramirez
Copy link
Copy Markdown
Author

Got it. I think that's a valid approach and I see where you're going with it.

Let me work on it in the background and I'll push the changes up when finished.

I'll reach out if anything pops up.

- Add 'pi' to SourceSchema (own source key, not sharing claude_code)
- Create rudel.pi_sessions table with version column (UInt8)
- Add pi_v2_session_analytics_mv: parses Claude Code format subagent JSONL
- Add pi_v3_session_analytics_mv: parses native Pi v3 JSONL format
- Both MVs write to shared session_analytics with source='pi'
- Pi adapter stores raw content (no transform at ingest time)
- Remove registerScanOnlyAdapter/transformV3Content (no longer needed)
- Add optional version field to IngestSessionInput schema
- Simplify CLI: getAdapter(source) works directly for Pi now

Per PR obsessiondb#155 feedback: embrace the Pi format, store raw, parse in MVs.
@adamjramirez
Copy link
Copy Markdown
Author

adamjramirez commented Mar 20, 2026

What changed:

  • New rudel.pi_sessions table with version column (UInt8) to distinguish v2 vs v3
  • source: "pi" — Pi is now a first-class source, not sharing claude_code
  • Raw storage — no transform at ingest time, content stored as-is
  • Two MVs into session_analytics:
    • pi_v2_session_analytics_mv (WHERE version=2) — mirrors the Claude Code MV logic since v2 subagent content is already in that format
    • pi_v3_session_analytics_mv (WHERE version=3) — parses native Pi format directly (nested message.usage.input/output/cacheRead/cacheWrite, type:"message" with message.role)
  • Cleaned up the adapter: removed transformV3Content, registerScanOnlyAdapter, and all the isPiSession dispatch hacks in the CLI (since getAdapter("pi") now works directly)
  • Error handling follows existing patternsbuildUploadRequest lets errors propagate to the caller's retry logic (matches Codex and Claude Code adapters) rather than swallowing them

Migration: 20260320124725_auto.sql — creates table + both MVs. 4 safe operations, table before MVs.

Tests: 23 pass, 70 assertions — covers v2/v3 raw storage, timestamp extraction, adapter isolation from Claude Code, error propagation, session discovery.

Let me know if there's anything else to adjust/

- Fix version=0 in error path → undefined (avoids Zod min(1) rejection)
- Remove Claude Code error patterns from v2 MV (isApiErrorMessage/is_error
  don't apply to Pi) — use toUInt32(0) like v3 MV
- Add test for error path (nonexistent transcript → empty content, no version)
- Extract resolveAdapterFromPath() to deduplicate upload.ts adapter resolution
- Export extractV3SessionId from adapter, remove duplicate in session-resolver
- Regenerate migration as single clean file (4 safe ops)
Match existing adapter pattern (Codex, Claude Code) — errors bubble
up to upload command's retry logic instead of being swallowed.

- Remove try/catch from PiAdapter.buildUploadRequest
- version is now always set (number, not number|undefined)
- Update test: assert rejects.toThrow() instead of empty content
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support pi (subagent-only) sessions — most sessions invisible to Rudel

2 participants