agent-librarian

agent-librarian v1.0 is a deterministic, local-first, provider-neutral CLI for cataloging local collections of agentic AI artifacts. It scans Markdown, YAML, and JSON files; classifies them with a framework-neutral taxonomy; extracts discoverability metadata; and reports likely overlap without modifying or executing the source collection.

v1.0 stabilizes deterministic local cataloging, generated outputs and schema validation, package adapter surfaces, offline presentation, and the source snapshot contract and conformance path. Governed external-source exports are supported only after they are local, reviewable snapshots. The synthetic SharePoint-style demo proves that bounded workflow without adding a live connector, authentication, provider SDK, or network behavior.

The repository's public examples are synthetic. Generated outputs inherit the sensitivity of their inputs and remain review aids, not certification or approval.

For a short project framing and demo-safe overview, see the Showcase Brief.

For a runnable five-minute demo path, see the Demo Walkthrough.

For a live presentation path, see the Forum Demo Runbook.

For a taxonomy-aligned view of the system, see the Architecture Map.

For a taxonomy-aligned catalog of the planned LLM interaction layer and deterministic CLI backend, see the Two-Layer Artifact Catalog.

For the planned LLM layer's identity, operating style, prompts, capability, tool boundary, and policies, see the Agent Layer Design Artifacts.

The bounded CLI contract for the planned interaction layer is documented in agent-librarian-cli.md.

The approval-gated catalog-review workflow is documented in catalog-review.md.

The planned LLM layer's refusal and redirect policy is documented in public-safety.md.

Static safe-scan eval cases for the planned LLM layer are documented in safe-scan-cases.md.

Runtime state and approval-log designs for the planned LLM layer are documented in state-and-approval-log.md.

The planned LLM layer's review summary contract is documented in review-summary.schema.json.

The optional local runtime wrapper prototype is documented in runtime-wrapper-prototype.md.

For guidance on safely adapting the tool to your own collection, see the Public-Safe Adoption Guide.

For the v0.6 presentation outputs, see the Presentation Demo Walkthrough.

The public stability boundary is defined in the v1.0 readiness and compatibility contract. The maintainer handoff is defined in the v1.0.0 release checklist.

For future external-source workflows, see the source snapshot contract. External sources must be exported to a local, reviewable snapshot before agent-librarian catalogs them. The same guide documents the local snapshot conformance check. The synthetic SharePoint-style end-to-end demo runs the complete public-safe workflow without credentials or network access. For an approval-gated private workflow with an already-created snapshot, see the Claude Code private-source snapshot runbook.

For post-forum planning and the next LLM-layer direction, see the v0.4 roadmap and forum feedback log.

Architecture direction

agent-librarian currently provides the deterministic CLI backend for cataloging, validating, and reviewing local agentic AI artifact collections. The project is evolving toward a two-layer architecture where a future LLM interaction layer can scope user intent, explain safety boundaries, propose bounded CLI commands, request approval before execution, and summarize generated outputs.

The LLM layer should orchestrate the workflow, not replace deterministic cataloging logic or invent results. CLI-generated files remain the source of truth and explicit review artifacts. The default CLI remains deterministic and does not call an LLM. The optional runtime wrapper prototype also does not call an LLM provider or network service. Only the explicit present --narrate path contacts a model provider.

Portable LLM agent packages

v0.5.0 adds portable package adapters for Claude Code, Codex, GPT, and ChatGPT Projects. The canonical agent contract remains framework-neutral under agent/; package adapters under packages/ adapt that contract for specific LLM workspaces.

See Portable Agent Packages and Agent Package Adapters for the shared package manifest and conformance foundation.

The first platform adapter is the Claude package, which demonstrates a functional Claude Code workflow using the approval-gated runtime wrapper and deterministic CLI evidence. See the Claude Code end-to-end demo and cross-platform demo guide.

The OpenAI package adds a command-capable Codex adapter and advisory GPT and ChatGPT Project adapters. See the Codex end-to-end demo and GPT/ChatGPT advisory demo.

Quickstart

python -m pip install -e ".[dev]"
agent-librarian --help
agent-librarian catalog examples/sample-collection --out examples/generated-catalog
agent-librarian catalog examples/sample-collection --out examples/generated-catalog --strict
agent-librarian validate examples/generated-catalog
pytest

The catalog command creates:

• index.json: machine-readable catalog entries and generation metadata.
• catalog.md: a human-readable inventory with warnings.
• overlap-report.json: duplicate and overlap candidates for human review.
• diagnostics.json: per-file parse status, warning codes, and sanitized errors.

The CLI parses only supported files under the input directory and records unsupported non-ignored files as skipped without reading their contents. It applies safe default excludes for Git metadata, environment and dependency folders, caches, build outputs, and generated catalogs. The selected output directory is also excluded whenever it is inside the input directory. It writes only the four generated files under --out.

Scan Include and Exclude Patterns

Narrow the inventoried files with repeated --include glob patterns and add custom relative path or name exclusions with --exclude:

agent-librarian catalog ./collection --out ./catalog \
  --include "**/*.md" \
  --include "**/*.yaml" \
  --exclude "private, scratch, tmp"

Include patterns narrow what appears in the scan and diagnostics. Repeated include patterns are combined with OR. Exclude values may be comma-separated or repeated, and custom excludes always add to the safe defaults.

Use include and exclude patterns to keep private folders, raw traces, memory/state snapshots, credentials, and generated outputs out of public catalogs. Generated catalogs from private collections should not be committed.

Normal mode continues when an individual file cannot be parsed, omits failed files from the catalog entries, records the failure in diagnostics.json, and exits 0 unless the command itself fails. Add --strict for CI or release validation:

agent-librarian catalog COLLECTION --out GENERATED --strict

Strict mode still writes outputs when practical, then exits non-zero if any file has a failed parse status. Partial and skipped files do not make strict mode fail.

Validate generated JSON against the bundled catalog schemas:

agent-librarian validate examples/generated-catalog

The command checks index.json, its embedded catalog entries, overlap-report.json, and diagnostics.json. It reports each file as passing or failing and exits non-zero for missing, malformed, or schema-invalid JSON. Validation is read-only and requires no network access.

Validation confirms generated JSON matches expected structure. It does not certify that the catalog is complete, safe, semantically correct, or free of private data.

After generating a catalog, summarize the review surfaces:

agent-librarian report examples/generated-catalog

The report reads existing generated outputs and summarizes diagnostics, warnings, and overlap candidates for human review. It does not rescan source files, modify generated outputs, or make artifact-management decisions.

Render the same generated facts as a self-contained offline HTML overview:

agent-librarian present examples/generated-catalog --out .tmp/present-demo

The command reads only index.json, diagnostics.json, and overlap-report.json from the catalog directory and writes .tmp/present-demo/overview.html. The result has no external CSS, JavaScript, network, provider, or LLM dependency and is deterministic for unchanged input JSON. It requires no API key.

Optionally add a grounded, model-authored summary for human review:

python -m pip install -e ".[narrate]"
$env:ANTHROPIC_API_KEY = "..."  # PowerShell; do not commit or print this value
agent-librarian present examples/generated-catalog --out .tmp/present-narrated --narrate

--narrate is an explicit opt-in. It requires ANTHROPIC_API_KEY; the Anthropic SDK is optional and installed through the narrate extra shown above. Use --model MODEL_ID to override the centralized default model. Narrated runs send only the three generated JSON documents named above to the Anthropic Messages API and write overview.html, narrative.md, and narrative-provenance.json. The model-authored section is a secondary review aid; deterministic facts remain visible and remain the source of truth. It is not a safety, privacy, correctness, completeness, approval, compliance, or publication-readiness certification. Without --narrate, no API key, provider dependency, or network access is required. All presentation outputs inherit the source catalog's sensitivity; public demos must use synthetic catalog data. Warnings and overlap candidates are prompts for human review, not decisions.

Committed synthetic examples are available under examples/generated-presentation and examples/generated-presentation-narrated. The narrated example is a static, stubbed demonstration and was created without a model or network call.

What It Catalogs

The classifier recognizes common artifact surfaces such as:

• SKILL.md capability modules
• AGENTS.md, CLAUDE.md, and agent manifests
• prompt Markdown files
• tool manifests
• MCP-style JSON manifests
• JSON Schemas

Internal classification uses the stable 14-bucket framework documented in taxonomy alignment. Framework and protocol names are optional hints, not replacement taxonomy categories.

Metadata and Warnings

Markdown frontmatter, headings, YAML fields, and JSON fields are normalized into catalog entries. Entries can include purpose, activation triggers, inputs, outputs, tool scope, side effects, dependencies, tags, ownership, version, related files, safety flags, discoverability warnings, and extraction confidence.

Warnings identify missing or weak metadata, unclear output contracts, broad tool scope, unclear side effects, missing examples or evals, unknown artifact types, and overlap candidates. Warnings are review aids, not correctness or safety certifications.

See Warnings and overlap for warning code meanings, suggested review actions, diagnostics interpretation, and the exact overlap scoring thresholds.

v0.6 Non-Goals

The v0.6 presentation layer does not provide:

• source crawling
• SharePoint or Microsoft 365 connectors
• MCP server behavior
• multi-provider abstraction
• safety or compliance certification
• publication approval
• automatic deduplication decisions
• automatic artifact edits
• a private-to-public sanitization guarantee

Public Safety

All committed examples are synthetic. Do not catalog or publish secrets, credentials, private endpoints, employer-specific workflows, real user data, live memory stores, raw traces, or unsanitized runtime state without an appropriate private handling and review process.

See public-safety guidance and the sample collection.

Development

python -m pip install -e ".[dev]"
pytest
git diff --check

See Developer workflow for local setup, validation commands, branch hygiene, Codex handoff expectations, and public-safety reminders.

The deterministic commands and default present path do not call external services and require no API keys. The explicit optional present --narrate path requires the narrate extra and ANTHROPIC_API_KEY.