Skip to content

Commit 15d41a9

Browse files
committed
Remove parser dependency from DuckDB engine - use database validation only
DuckDB now uses a minimal pass-through parser instead of the TiDB parser. All SQL parsing, validation, and type checking happens directly in the DuckDB database via the analyzer. This ensures 100% compatibility with DuckDB syntax without maintaining a separate parser. Changes: - Removed TiDB parser dependency from parse.go - Parser now returns minimal TODO AST nodes - Deleted convert.go (AST conversion not needed) - Database handles all SQL validation via PREPARE/DESCRIBE - Updated documentation to reflect database-only validation approach Benefits: - No shared parser with MySQL/Dolphin engine - Perfect DuckDB syntax compatibility - Simpler codebase with fewer dependencies - All validation happens where it should: in the database The analyzer is now solely responsible for: - Parsing SQL via DuckDB's native parser - Validating queries against the schema - Extracting column and parameter type information - Normalizing DuckDB types to sqlc types 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent c9a899b commit 15d41a9

File tree

3 files changed

+44
-1956
lines changed

3 files changed

+44
-1956
lines changed

DUCKDB_SUPPORT.md

Lines changed: 26 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,11 @@ DuckDB support has been added to sqlc using a database-backed approach, similar
1111
### Core Components
1212

1313
1. **Parser** (`/internal/engine/duckdb/parse.go`)
14-
- Uses the TiDB parser (same as MySQL/Dolphin engine)
14+
- **Minimal pass-through parser** - does not parse SQL into AST
15+
- All parsing and validation happens in the database via the analyzer
1516
- Implements the `Parser` interface with `Parse()`, `CommentSyntax()`, and `IsReservedKeyword()` methods
1617
- Supports `--` and `/* */` comment styles (DuckDB standard)
18+
- Returns TODO AST nodes - actual parsing done by DuckDB database
1719

1820
2. **Catalog** (`/internal/engine/duckdb/catalog.go`)
1921
- Minimal catalog implementation
@@ -24,14 +26,11 @@ DuckDB support has been added to sqlc using a database-backed approach, similar
2426
- **REQUIRED** for DuckDB engine (not optional like PostgreSQL)
2527
- Connects to DuckDB database via `github.com/marcboeker/go-duckdb`
2628
- Uses PREPARE and DESCRIBE to analyze queries
29+
- Handles all SQL parsing and validation via the database
2730
- Queries column metadata from prepared statements
2831
- Normalizes DuckDB types to sqlc-compatible types
2932

30-
4. **AST Converter** (`/internal/engine/duckdb/convert.go`)
31-
- Copied from Dolphin/MySQL implementation
32-
- Converts TiDB parser AST to sqlc universal AST
33-
34-
5. **Reserved Keywords** (`/internal/engine/duckdb/reserved.go`)
33+
4. **Reserved Keywords** (`/internal/engine/duckdb/reserved.go`)
3534
- DuckDB reserved keywords based on official documentation
3635
- Includes LAMBDA (reserved as of DuckDB 1.3.0)
3736
- Can be queried from DuckDB using `SELECT * FROM duckdb_keywords()`
@@ -121,12 +120,15 @@ VALUES ($1, $2);
121120
## Key Differences from Other Engines
122121

123122
### vs PostgreSQL
124-
- **PostgreSQL**: Optional database analyzer, rich Go-based catalog with pg_catalog
125-
- **DuckDB**: Required database analyzer, minimal catalog
123+
- **PostgreSQL**: Optional database analyzer, rich Go-based catalog with pg_catalog, full AST parsing
124+
- **DuckDB**: Required database analyzer, minimal catalog, no AST parsing (database validates SQL)
126125

127126
### vs MySQL/SQLite
128-
- **MySQL/SQLite**: Go-based catalog with built-in functions
129-
- **DuckDB**: Database-backed only, no Go-based catalog
127+
- **MySQL/SQLite**: Go-based catalog with built-in functions, TiDB/ANTLR parser with full AST
128+
- **DuckDB**: Database-backed only, no Go-based catalog, minimal parser (database parses SQL)
129+
130+
### Unique Approach
131+
DuckDB is the only engine that doesn't parse SQL in Go. All SQL parsing, validation, and type checking happens directly in the DuckDB database. This ensures 100% compatibility with DuckDB's SQL syntax without needing to maintain a separate parser.
130132

131133
## Type Mapping
132134

@@ -205,27 +207,28 @@ duckdb engine requires database configuration
205207

206208
1. **Network dependency**: Requires network access to download go-duckdb initially
207209
2. **Parameter type inference**: DuckDB doesn't provide parameter types without execution, so parameters are typed as "any" by the analyzer
208-
3. **Parser limitations**: Uses TiDB parser which may not support all DuckDB-specific syntax (STRUCT, LIST, UNION types may require custom handling)
210+
3. **Database required**: Unlike other engines, DuckDB cannot generate code without a database connection (no offline mode)
209211

210212
## Future Enhancements
211213

212-
1. Improve parameter type inference
214+
1. Improve parameter type inference by analyzing query patterns
213215
2. Add support for DuckDB-specific types (STRUCT, LIST, UNION, MAP)
214-
3. Support DuckDB extensions
216+
3. Support DuckDB extensions and extension-specific functions
215217
4. Add DuckDB-specific selector for custom column handling
216218
5. Improve error messages with DuckDB-specific error codes
219+
6. Cache database connections for better performance
220+
7. Support managed databases via database manager
217221

218222
## Files Modified/Created
219223

220224
### Created:
221-
- `/internal/engine/duckdb/parse.go`
222-
- `/internal/engine/duckdb/catalog.go`
223-
- `/internal/engine/duckdb/convert.go`
224-
- `/internal/engine/duckdb/reserved.go`
225-
- `/internal/engine/duckdb/analyzer/analyze.go`
226-
- `/examples/duckdb/basic/schema/schema.sql`
227-
- `/examples/duckdb/basic/query/query.sql`
228-
- `/examples/duckdb/basic/sqlc.yaml`
225+
- `/internal/engine/duckdb/parse.go` - Minimal pass-through parser
226+
- `/internal/engine/duckdb/catalog.go` - Minimal catalog
227+
- `/internal/engine/duckdb/reserved.go` - Reserved keywords
228+
- `/internal/engine/duckdb/analyzer/analyze.go` - Database analyzer
229+
- `/examples/duckdb/basic/schema/schema.sql` - Example schema
230+
- `/examples/duckdb/basic/query/query.sql` - Example queries
231+
- `/examples/duckdb/basic/sqlc.yaml` - Example configuration
229232

230233
### Modified:
231234
- `/internal/config/config.go` - Added `EngineDuckDB` constant
@@ -234,8 +237,10 @@ duckdb engine requires database configuration
234237

235238
## Notes
236239

240+
- **No SQL parsing in Go**: DuckDB engine validates all SQL via the database, not in Go code
237241
- DuckDB uses "main" as the default schema (different from PostgreSQL's "public")
238242
- DuckDB uses "memory" as the default catalog name
239243
- Comment syntax supports only `--` and `/* */`, not `#`
240244
- Reserved keyword LAMBDA was added in DuckDB 1.3.0
241245
- Reserved keyword GRANT was removed in DuckDB 1.3.0
246+
- 100% compatibility with DuckDB syntax since the database itself parses SQL

0 commit comments

Comments
 (0)