Skip to content

Proxy federated SQL across multiple SQLite or DuckDB databases #3

@shah

Description

@shah

Implement a “serverless” way to execute federated SQL across one or more SQLite and/or DuckDB databases without spawning per-database servers.

The core idea is to treat the local/discovered databases as queryable sources and generate a temporary SQL script that:

  1. ATTACHes each discovered database under a deterministic schema name (based on pattern, filename, or catalog identity).
  2. Optionally applies guardrails (read-only mode, deny DDL/DML by default, allowlist statements) to make execution “proxy safe”.
  3. Executes a user-supplied query (or a file of queries) against the attached schemas, letting SQLite/DuckDB handle the cross-database proxying via ATTACH semantics through either direct SQL or an MCP or "Skills".

Scope to explore:

  • Discover databases (SQLite .db/.rssd.db/.sqlpage.db and DuckDB .duckdb) using existing discovery patterns.
  • Define schema naming rules to avoid collisions and keep names stable across runs (using same proxy prefixing pattern).
  • Generate an SQL file that contains the ATTACH statements plus the user’s query block.
  • Generate an MCP that can access the data via AI assistants
  • Support running the generated script through sqlite3 and/or duckdb CLI (or via the yard runtime) with consistent output modes (table/json/csv).
  • Restrict queries to specific tenants or other "groupings" ("scoping")

Add safety defaults:

  • Read-only by default (for SQLite, use immutable/query-only options where possible; for DuckDB, enforce read-only connection when feasible).
  • Block or warn on DDL/DML unless an explicit flag is provided.
  • Prevent writing temp files into the attached DB directories unless explicitly enabled.

Acceptance criteria:

  • Can discover multiple databases and attach them under predictable schema names using same identifiers as proxy pass rules.
  • Can use either naming or tokenization to group by tenants or other scopes
  • Can run a query that references one attached database, or joins across multiple attached schemas, without starting any server.
  • Generated SQL (ATTACH + query) can be emitted to a file for inspection/reproducibility.
  • Safety defaults prevent accidental modification of source databases unless explicitly overridden.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions