Experiment: Port MySQL-on-SQLite to LALR(1) parser

[JanJakes]

Stacked on #429. This experiment ports the MySQL-on-SQLite driver end-to-end from the hand-written recursive parser to the LALR(1) parser, consumed as a proper Composer dependency. All driver tests pass: 543 tests / 7,172 assertions in mysql-on-sqlite, and 41 tests / 1,051 assertions in mysql-parser, including the 69,577-query MySQL server corpus pin. Net diff: +1,611 / −7,180 lines.

Composer-based package reuse

The wordpress/mysql-parser package gets a classmap Composer autoloader (the WordPress-style file naming rules out PSR-4) and exposes WP_MySQL_Parser::PARSE_TABLE_PATH, since the generated parse table is data the autoloader cannot cover. The wordpress/mysql-on-sqlite package requires it through a Composer path repository pointing at the monorepo sibling, so development has a single source of truth (a vendor symlink) and nothing is duplicated: the driver's old parser machinery — grammar, lexer, parse tree classes, and the native Rust parser fork bound to the old grammar contract — is removed entirely.

Parser changes the port surfaced

• Empty reductions produce no AST nodes. Empty optionals (opt_*) and Bison's mid-rule action rules ($@N) carry no information, so they no longer appear in the tree, and consumers see an optional clause only when it is present.
• WP_Parser_Node::get_flattened_child_nodes() iterates left-recursive grammar lists (list: list ',' item) as if they were flat.
• ANSI_QUOTES SQL mode in the lexer, with a driver-side parse retry. MySQL rejects double-quoted identifiers without ANSI_QUOTES, but WordPress relies on them (dbDelta can produce double-quoted index names) and the previous parser accepted them. The retry accepts these statements while preserving string-literal semantics for everything that parses without it.

Driver port

The statement dispatch, the query translation layer, and the information schema builder are re-keyed to the official sql_yacc.yy rule names and tree shapes. Multi-statement input is split on top-level ; separators, as the grammar parses a single statement (this is how MySQL clients split multi-statement input); the old create_parser()/next_query() API is replaced by parse_mysql_query().

The information schema builder was verified byte-exact against the old parser and builder over a DDL battery covering all supported data types, constraints, indexes, and table options. Multi-column ADD COLUMN (a INT, b INT) is now recorded correctly; the previous builder crashed on it.

Deployment and CI

The WordPress Docker environments install the driver's Composer dependencies and mount the package vendor directory and the parser package into the containers. The plugin zip build bundles the driver's production dependencies, resolving the path-repository symlink into a real, pruned copy of the parser package. The driver test workflow now also triggers on parser package changes, and the native parser extension jobs and scripts are removed (packages/php-ext-wp-mysql-parser is orphaned by this branch).

Testing

cd packages/mysql-on-sqlite && composer install && composer run test
cd packages/mysql-parser && composer install && composer run test
composer run build-sqlite-plugin-zip

[github-actions]

🤖 Lexer benchmark

Changes to lexer-related files were detected and triggered a benchmark:

Config	Base (QPS)	This PR (QPS)	Speedup
no JIT	74,170	55,056	0.74×
tracing JIT	152,834	114,823	0.75×

Note: Hosted runners are noisy, and absolute numbers vary. Treat the results with caution and verify them locally.

To reproduce locally:

cd packages/mysql-on-sqlite && composer run bench-lexer