dasSQLITE chunk 14c: schema rebuild via [struct_convert]

[borisbat]

Summary

Final piece of the migration trio (after 14a spine, 14b typed ALTER) — the rebuild path for schema changes SQLite's narrow ALTER vocabulary cannot express in place (PK/FK/CHECK changes, type narrowing).

User-side: SQLite's "12-step recipe" collapses to three lines.

[sql_table(name="users", legacy=true)]
struct UserV5 { @sql_primary_key Id : int; Name : string; Email : string }

[sql_table(name="users")]
struct User { @sql_primary_key Id : int; Name : string; Email : string; LastLoginAt : Option<int64> }

[struct_convert]
def m_v5_to_v6(old : UserV5; var dst : User) {}    // empty body — every field auto-derives

[sql_migration(version=6, description="add LastLoginAt")]
def migration_006(db : SqlRunner) {
    db |> convert_and_rename(type<UserV5>, type<User>)
}

What ships

• [sql_table(legacy=true)] — emits read-side helpers only; writes (create_table / insert / update / delete) fail at compile time as unresolved overloads.
• [struct_convert] function annotation — auto-derives field-by-field translation (same-name same-type, T → Option<T>, scalar primitive casts, @sql_renamed_from). Walks the user body via AstVisitor::preVisitExprField for ANY mention of dst.X to suppress the auto-derived assignment when the user takes control.
• db |> convert(type<S>, type<T> [, name]) — SELECT → converter → INSERT loop.
• db |> convert_and_rename(type<S>, type<T>) — full rebuild: CREATE staging, copy via converter, DROP original, RENAME.
• Runtime name : string overloads for create_table and insert — make_*_sql_fn factories refactored to twin overloads (1-arg wrapper forwards to 2-arg with the struct's own [sql_table(name=...)]).

Bonus: utils/hygiene

• New rule rule_collapse_blank_lines.das chained after the private-docstring trim — runs of 2+ blank lines collapse to one, CRLF round-trips, whitespace-only lines count as blank. 8 unit tests.
• Applied across sqlite_boost.das / sqlite_migrate.das / sqlite_linq.das (57 blank-line collapses + a handful of private //! strips).
• --comment-hygiene (STYLE014/STYLE015) cleanup on the three SQL files: trimmed multi-paragraph banner blocks and oversized private // ... comments to 1-line WHYs per CLAUDE.md.

Test plan

• dastest --test tests/dasSQLITE — 762/762 (interpreted)
• test_aot.exe -use-aot full suite — 7321/7321 (AOT)
• daslang utils/lint/main.das clean on all changed .das files
• --comment-hygiene clean on the three SQL daslib files
• MCP format_file clean on every changed .das
• Sphinx -W clean (zero warnings, zero errors) on the doc tree
• dasSQLITE migration tutorial 43 runs end-to-end with all 6 migrations applied; PART 5 demonstrates rebuild + surfaces the indexes-gone-after-rebuild gotcha

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR extends dasSQLITE migrations with a full “schema rebuild” path for changes that SQLite can’t express via in-place ALTER TABLE, centered around legacy structs, [struct_convert] auto-derived row translation, and convert_and_rename staging-table rebuilds. It also expands the hygiene toolchain with a rule to collapse consecutive blank lines and updates tutorial/docs plus adds targeted regression tests.

Changes:

• Add [struct_convert] annotation + convert / convert_and_rename helpers to support rebuild-style migrations.
• Introduce [sql_table(legacy=true)] read-only structs and runtime name:string overloads for staging-table create_table/insert.
• Add rule_collapse_blank_lines to hygiene rules, apply it in the rule chain, and add tests/docs/tutorial updates.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
utils/hygiene/test_hygiene.das	Adds unit tests for collapsing blank lines and rule chaining.
utils/hygiene/rules.das	Chains collapse_blank_lines after trim_private_docstrings.
utils/hygiene/rule_collapse_blank_lines.das	New hygiene rule to collapse runs of blank/whitespace-only lines while preserving CRLF.
tutorials/sql/43-migrations.das	Tutorial expanded with legacy struct + [struct_convert] + rebuild migration example and gotchas.
tests/dasSQLITE/migrate_91_full_rebuild_end_to_end.das	End-to-end test for rebuild flow preserving rows and performing translation overrides.
tests/dasSQLITE/migrate_90_sql_table_legacy_marker.das	Tests legacy structs remain readable and can be dropped.
tests/dasSQLITE/migrate_89_insert_name_override.das	Tests insert(row, name) writes into a staging table.
tests/dasSQLITE/migrate_88_create_table_name_override.das	Tests create_table(type<T>, name) emits correct schema + index naming for staging tables.
tests/dasSQLITE/migrate_87_struct_convert_override_via_mention.das	Tests override suppression via any dst.X mention in converter body.
tests/dasSQLITE/migrate_81_struct_convert_option_flip.das	Tests auto-derivation for T -> Option<T> wrapping behavior.
tests/dasSQLITE/migrate_80_struct_convert_trivial.das	Trivial rebuild round-trip test using auto-derived field copies.
tests/dasSQLITE/failed_struct_convert_option_to_t.das	Negative compile test: rejects Option<T> -> T without explicit fallback.
tests/dasSQLITE/failed_struct_convert_new_field_no_default.das	Negative compile test: rejects new non-optional field with no default and no override.
modules/dasSQLITE/daslib/sqlite_migrate.das	Implements [struct_convert] macro + convert / convert_and_rename rebuild primitives.
modules/dasSQLITE/daslib/sqlite_linq.das	Comment-hygiene edits (no behavioral changes apparent in shown diff).
modules/dasSQLITE/daslib/sqlite_boost.das	Adds legacy marker behavior, staging-table name overload plumbing, and refactors helper SQL factories.
doc/source/reference/tutorials/sql_43_migrations.rst	Documents rebuild workflow and updates “what does not ship” section accordingly.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

modules/dasSQLITE/daslib/sqlite_boost.das:2202

• is_option_field_type (and the nearby unwrap_option_payload_type) changed from private to public in the sqlite_boost public module, which expands the public API surface with very generic names. If these are intended as internal macro helpers, consider keeping them private (and duplicating/moving the minimal logic into sqlite_migrate) or renaming/prefixing them to avoid future namespace/API collisions.

def is_option_field_type(td : TypeDeclPtr) : bool {
    if (td == null) {
        return false
    }
    // Pre-instantiation form: typeMacro `$Option<T>` — what [sql_table] sees during structure_macro apply().

[Copilot]

Pull request overview

Copilot reviewed 26 out of 26 changed files in this pull request and generated 1 comment.