README.md

scrape-docs

Centralized, scheduled documentation scraping for a collection of public Markdown repositories.

The scraper publishes each docset to scraped-docs-<name> as Markdown only. Downstream repositories contain only the generated documentation and can be used directly as OpenCode references.

Two docsets publish daily from .github/workflows/scrape-daily.yml: OpenCode V2 to birbprophet/scraped-docs-opencode-v2 and Effect v4 to birbprophet/scraped-docs-effect-v4. The remaining docsets are kept as reusable configuration and run on demand from .github/workflows/scrape.yml.

// opencode.jsonc

{
  // ...
  "references": {
    "foldkit-docs": {
      "repository": "mpsuesser/scraped-docs-foldkit",
      "branch": "main",
      "description": "Full scraped documentation for Foldkit"
    },
    "alchemy-docs": {
      "repository": "mpsuesser/scraped-docs-alchemy",
      "branch": "main",
      "description": "Full scraped documentation for Alchemy"
    }
  }
}

Docsets

• Alchemy
• Better Auth
• Drizzle
• Electron
• Ent (Go)
• Ent (TypeScript)
• Effect (v4)
• Foldkit
• LadybugDB
• OpenCode (V2)
• Pi
• PlanetScale
• ts-morph
• Turborepo

Commands

bun install
bun run typecheck
bun run test
bun run scrape-docs:foldkit
bun run scrape-docs:opencode-v2
bun run scrape-docs:effect-v4
bun run alchemy:plan

Alchemy declares the mpsuesser downstream repositories in alchemy.run.ts. The workflow uses repository-scoped SSH deploy keys stored in the encrypted DOCS_REPO_KEYS Actions secret.