Merge pull request #51 from cmdscale/feature/rework_migration_code_generation

Rework migration code generation

Author: sebastian-ederer

.claude/CLAUDE.md

@@ -86,8 +86,8 @@ private class TestContext(string connectionString) : DbContext { }
 | Service Registration | `UseTimescaleDb()` configures all services | `reference/patterns.md` |
 | Convention System | `IEntityTypeAddedConvention` processes attributes | `reference/patterns.md` |
 | Dual Configuration | Annotations + Fluent API → same annotations | `reference/patterns.md` |
-| IFeatureDiffer | Per-feature differ with model extractor | `reference/patterns.md` |
-| Runtime vs Design-Time | `isDesignTime` parameter changes quote escaping | `reference/patterns.md` |
+| IFeatureDiffer | Per-feature differ with model extractor + `FeatureDiffContext` | `reference/patterns.md` |
+| Runtime vs Design-Time | `*SqlGenerator` (SQL) vs `*CSharpGenerator` (typed migration calls) | `reference/patterns.md` |
 | Column Name Resolution | Always use `StoreObjectIdentifier` + `GetColumnName()` | `reference/patterns.md` |
 
 ## Agent Workflow

.claude/agents/eftdb-bug-fixer.md

@@ -1,6 +1,6 @@
 ---
 name: eftdb-bug-fixer
-description: Use this agent when bugs are discovered in existing runtime or design-time code within the CmdScale.EntityFrameworkCore.TimescaleDB library. This includes:\n\n<example>\nContext: User discovers a bug in the HypertableDiffer.\nuser: "The HypertableDiffer is not detecting changes to chunk time interval"\nassistant: "I'll use the eftdb-bug-fixer agent to analyze and fix the HypertableDiffer issue."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: SQL generation is incorrect for reorder policies.\nuser: "The ReorderPolicyOperationGenerator is generating invalid SQL with wrong schema qualification"\nassistant: "I'll launch the eftdb-bug-fixer agent to fix the SQL generation bug in ReorderPolicyOperationGenerator."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: Scaffolding extractor query is failing.\nuser: "The ContinuousAggregateScaffoldingExtractor is throwing NullReferenceException when extracting aggregate functions"\nassistant: "Let me use the eftdb-bug-fixer agent to debug and fix the scaffolding extractor."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: Another agent reports a bug during its work.\nuser: "The eftdb-scaffold-support agent reported a mismatch between runtime annotations and scaffolding expectations"\nassistant: "I'll use the eftdb-bug-fixer agent to resolve the annotation mismatch issue reported by the scaffolding agent."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>
+description: Use this agent when bugs are discovered in existing runtime or design-time code within the CmdScale.EntityFrameworkCore.TimescaleDB library. This includes:\n\n<example>\nContext: User discovers a bug in the HypertableDiffer.\nuser: "The HypertableDiffer is not detecting changes to chunk time interval"\nassistant: "I'll use the eftdb-bug-fixer agent to analyze and fix the HypertableDiffer issue."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: SQL generation is incorrect for reorder policies.\nuser: "The ReorderPolicySqlGenerator is generating invalid SQL with wrong schema qualification"\nassistant: "I'll launch the eftdb-bug-fixer agent to fix the SQL generation bug in ReorderPolicySqlGenerator."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: Scaffolding extractor query is failing.\nuser: "The ContinuousAggregateScaffoldingExtractor is throwing NullReferenceException when extracting aggregate functions"\nassistant: "Let me use the eftdb-bug-fixer agent to debug and fix the scaffolding extractor."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>\n\n<example>\nContext: Another agent reports a bug during its work.\nuser: "The eftdb-scaffold-support agent reported a mismatch between runtime annotations and scaffolding expectations"\nassistant: "I'll use the eftdb-bug-fixer agent to resolve the annotation mismatch issue reported by the scaffolding agent."\n<uses Task tool to invoke eftdb-bug-fixer>\n</example>
 model: sonnet
 color: red
 ---
@@ -38,7 +38,9 @@ You are an elite debugging and code quality specialist for the CmdScale.EntityFr
    - Identify which component is affected:
      - Model Extractor (reads annotations from EF model)
      - Differ (compares models and generates operations)
-     - Operation Generator (generates SQL/C# code)
+     - SQL Generator (`Generators/[Feature]SqlGenerator.cs` — runtime SQL)
+     - C# Generator (`Design/Generators/[Feature]CSharpGenerator.cs` — typed migration calls)
+     - Migration Extensions (`MigrationExtensions/[Feature]MigrationExtensions.cs`)
      - Scaffolding Extractor (queries TimescaleDB catalog)
      - Scaffolding Applier (applies annotations to scaffolded model)
      - Convention (converts attributes to annotations)
@@ -72,10 +74,10 @@ Before fixing, understand WHY the bug exists:
    - Hard-coded column names instead of convention-aware resolution
 
 3. **SQL Generation Bugs:**
-   - Quote string not respected (`isDesignTime` parameter ignored)
+   - Identifiers not quoted via `SqlBuilderHelper` (`Regclass`/`QualifiedIdentifier`/`QuoteIdentifier`)
    - Schema qualification missing or incorrect
    - SQL syntax errors for specific TimescaleDB functions
-   - Parameter escaping issues
+   - Missing `suppressTransaction` for DDL that cannot run in a transaction (continuous aggregates)
 
 4. **Null Reference Issues:**
    - Missing null checks for optional properties
@@ -88,9 +90,9 @@ Before fixing, understand WHY the bug exists:
    - Type conversion issues (string vs long for intervals)
 
 6. **Design-Time vs Runtime Confusion:**
-   - Generator not handling `isDesignTime` parameter correctly
-   - Quote escaping wrong for C# string generation
-   - Operation registered in runtime but not in design-time generator
+   - Operation registered in the runtime `TimescaleDbMigrationsSqlGenerator` switch but not in the design-time `TimescaleCSharpMigrationOperationGenerator` switch (or vice versa)
+   - Missing `MigrationExtensions` method so generated migrations cannot call the operation
+   - Runtime SQL and design-time typed call producing inconsistent results
 
 ### Phase 3: Fix Implementation
 
@@ -160,12 +162,12 @@ string columnName = property.GetColumnName(storeIdentifier);
 ```
 
 ```csharp
-// Bug: Quote string not used in SQL generation
+// Bug: identifier not quoted via the helper
 // INCORRECT FIX - Hard-coded quotes
 string sql = $"SELECT * FROM \"{schema}\".\"{table}\"";
 
-// CORRECT FIX - Use quote string field and SqlBuilderHelper
-string qualifiedName = SqlBuilderHelper.GetQualifiedTableName(schema, table, _quoteString);
+// CORRECT FIX - Use SqlBuilderHelper
+string qualifiedName = SqlBuilderHelper.QualifiedIdentifier(table, schema);
 string sql = $"SELECT * FROM {qualifiedName}";
 ```
 
@@ -212,14 +214,11 @@ if (annotation.Value is not string expectedValue)
 
 ### For SQL Generation Issues:
 ```csharp
-// Verify quote string usage
-System.Diagnostics.Debug.WriteLine($"Quote string: '{_quoteString}'");
-System.Diagnostics.Debug.WriteLine($"Generated SQL: {sql}");
-
-// Check if isDesignTime is propagated correctly
-if (isDesignTime && !sql.Contains("\"\""))
+// Inspect the statements returned by the feature SqlGenerator
+List<string> statements = HypertableSqlGenerator.Generate(operation);
+foreach (string statement in statements)
 {
-    // Likely bug: design-time should have doubled quotes
+    System.Diagnostics.Debug.WriteLine($"Generated SQL: {statement}");
 }
 ```
 

.claude/agents/eftdb-feature-implementer.md

@@ -11,7 +11,7 @@ You are an elite Entity Framework Core migrations architect specializing in the
 
 **PROJECT SCOPE RESTRICTION**: You MUST NOT modify code in any project except:
 - CmdScale.EntityFrameworkCore.TimescaleDB (primary work area)
-- CmdScale.EntityFrameworkCore.TimescaleDB.Design (ONLY the TimescaleCSharpMigrationOperationGenerator.cs file)
+- CmdScale.EntityFrameworkCore.TimescaleDB.Design (the `Generators/[Feature]CSharpGenerator.cs` file and `TimescaleCSharpMigrationOperationGenerator.cs`)
 
 Any attempt to modify other projects should result in immediate rejection with explanation.
 
@@ -48,105 +48,58 @@ Implement the following components in this exact order:
 
 #### 2. Feature Differ (Internals/Features/[Feature]Differ.cs)
 
-- Implement `IFeatureDiffer` interface
-- Use the extractor to compare source and target models
-- Generate appropriate operations (Create, Alter, Drop) based on differences
-- Return `IEnumerable<MigrationOperation>` with proper priority values:
-  - Priority 0: Standard EF operations
-  - Priority 10: CreateHypertableOperation
-  - Priority 20: Reorder policies
-  - Priority 30: Create continuous aggregates
-  - Priority 40: Alter/Drop continuous aggregates
-  - Choose appropriate priority for your feature based on dependencies
-- Follow existing patterns from HypertableDiffer, ReorderPolicyDiffer, or ContinuousAggregateDiffer
+- Implement `IFeatureDiffer`: `IReadOnlyList<MigrationOperation> GetDifferences(IRelationalModel? source, IRelationalModel? target, FeatureDiffContext? context = null)`
+- Normalize `context ??= FeatureDiffContext.Empty;` and use it to resolve renames (`ResolveTable`, `ResolveColumn`, `ResolveIndex`) so a rename is not treated as drop-and-create
+- Use the extractor to compare source and target models, generating Create/Alter/Drop operations
+- Operation ordering is handled centrally by `GetOperationPriority()` (see step 3) — the differ does not set priorities itself
+- Follow existing patterns from HypertableDiffer, ReorderPolicyDiffer, RetentionPolicyDiffer, or ContinuousAggregateDiffer
 
 #### 3. Update TimescaleMigrationsModelDiffer (Internals/TimescaleMigrationsModelDiffer.cs)
 
-- Register your new differ in the constructor's `_featureDiffers` list
-- Ensure it's positioned correctly based on dependency order
-- No other changes needed to this file
+- Invoke your new differ in `GetDifferences()`, passing the shared `FeatureDiffContext`
+- Add a `case` for each new operation type in `GetOperationPriority()` (drops negative, adds/alters positive; pick values matching the feature's dependency order — see the priority table in `reference/architecture.md`)
 
-#### 4. Operation Generator (Generators/[Feature]OperationGenerator.cs)
+#### 4. Runtime SQL Generator (Generators/[Feature]SqlGenerator.cs)
 
-- Create a class that handles both SQL generation and C# code generation
-- **CRITICAL**: Constructor MUST have `isDesignTime` parameter with default value `false`:
-  ```csharp
-  public FeatureOperationGenerator(bool isDesignTime = false)
-  {
-      _quoteString = isDesignTime ? "\"\"" : "\"";
-  }
-  ```
-- Use `_quoteString` for all string literals in SQL generation
-- Implement these methods for each operation type:
-  - `Generate([Operation] operation, IModel? model, MigrationCommandListBuilder builder, bool isDesignTime)` - Runtime SQL
-  - `Generate([Operation] operation, CSharpMigrationOperationBuilder builder)` - Design-time C# code
-- Use `SqlBuilderHelper` static methods for table names, schema handling, and identifier quoting:
-  - `GetQualifiedTableName(schema, table, quoteString)` for fully qualified names
-  - `GetSchemaPrefix(schema, quoteString)` for schema prefixing
-  - Always pass your `_quoteString` to these methods
-- Follow SQL generation patterns from existing generators (HypertableOperationGenerator, ReorderPolicyOperationGenerator)
-
-#### 5. Update TimescaleDbMigrationsSqlGenerator (TimescaleDbMigrationsSqlGenerator.cs)
-
-- Add method to handle your operation type:
-  ```csharp
-  protected virtual void Generate([YourOperation] operation, IModel? model, MigrationCommandListBuilder builder)
-  {
-      var generator = new YourOperationGenerator(_isDesignTime);
-      generator.Generate(operation, model, builder, _isDesignTime);
-  }
-  ```
-- Store `isDesignTime` parameter in a field: `private readonly bool _isDesignTime;`
-- Pass it through to generators
+- Static class exposing `static List<string> Generate(XxxOperation operation)` per operation type, returning TimescaleDB SQL statements
+- Build identifiers with `SqlBuilderHelper.Regclass()`, `SqlBuilderHelper.QualifiedIdentifier()`, `SqlBuilderHelper.QuoteIdentifier()`
+- For policy scheduling SQL (`alter_job` clauses), reuse `PolicyJobSqlBuilder`
+- Follow existing generators (HypertableSqlGenerator, RetentionPolicySqlGenerator)
 
-#### 6. Update TimescaleCSharpMigrationOperationGenerator (Design Project - ONLY FILE ALLOWED)
+#### 5. Typed Migration Extensions (MigrationExtensions/[Feature]MigrationExtensions.cs)
 
-- Add C# code generation method:
-  ```csharp
-  protected virtual void Generate([YourOperation] operation, CSharpMigrationOperationBuilder builder)
-  {
-      var generator = new YourOperationGenerator(isDesignTime: true);
-      generator.Generate(operation, builder);
-  }
-  ```
-- This is the ONLY file in the Design project you may modify
+- Add extension methods on `MigrationBuilder` (declared in namespace `Microsoft.EntityFrameworkCore.Migrations`) that construct the operation and `migrationBuilder.Operations.Add(operation)`
+- Return an `OperationBuilder<XxxOperation>`
+- These are the methods generated migrations call (e.g. `migrationBuilder.CreateHypertable(...)`)
+
+#### 6. Register in TimescaleDbMigrationsSqlGenerator (TimescaleDbMigrationsSqlGenerator.cs)
+
+- Add a `case XxxOperation op:` to the `Generate` switch that calls `[Feature]SqlGenerator.Generate(op)` and assigns `statements`
+- Set `suppressTransaction = true` for operations whose DDL cannot run in a transaction (e.g. continuous-aggregate creation)
+
+#### 7. Design-Time C# Generator (Design/Generators/[Feature]CSharpGenerator.cs + register)
+
+- `Generate(XxxOperation operation, IndentedStringBuilder builder)` emits the typed `migrationBuilder.[Method](...)` call using `MigrationCallWriter` and `CSharpGeneratorHelper`
+- Emit a named `call.Arg("argName", code.Literal(...))` for each value, skipping defaults/empties
+- Register the operation type in the `switch` in `TimescaleCSharpMigrationOperationGenerator.cs`
 
 ## Critical Technical Requirements
 
-### Quote String Handling
+### Runtime vs Design-Time Split
 
-**This is ABSOLUTELY CRITICAL for runtime vs design-time duality:**
+The two paths are independent and consume the same operation types:
 
-- **Runtime Migrations** (`dotnet ef database update`):
-  - Quote string: `"` (single quote)
-  - Generates raw SQL that executes against database
-  
-- **Design-Time Migrations** (`dotnet ef migrations add`):
-  - Quote string: `""` (doubled quotes)
-  - Generates C# code with escaped strings for migration files
+- **Runtime** (`dotnet ef database update`): `TimescaleDbMigrationsSqlGenerator` → `[Feature]SqlGenerator.Generate(operation)` → SQL statements.
+- **Design-time** (`dotnet ef migrations add`): `TimescaleCSharpMigrationOperationGenerator` → `[Feature]CSharpGenerator.Generate(operation, builder)` → typed `migrationBuilder.[Method](...)` calls.
 
-**Implementation Pattern:**
-```csharp
-public class YourOperationGenerator
-{
-    private readonly string _quoteString;
-    
-    public YourOperationGenerator(bool isDesignTime = false)
-    {
-        _quoteString = isDesignTime ? "\"\"" : "\"";
-    }
-    
-    // Use _quoteString in all SQL generation
-    var tableName = SqlBuilderHelper.GetQualifiedTableName(schema, table, _quoteString);
-}
-```
+Generators carry no `isDesignTime` flag and do no quote-doubling.
 
 ### SqlBuilderHelper Usage
 
-ALWAYS use SqlBuilderHelper for:
-- Table name qualification: `GetQualifiedTableName(schema, table, quoteString)`
-- Schema prefixing: `GetSchemaPrefix(schema, quoteString)`
-- Identifier quoting: Methods handle this internally when you pass quoteString
+In `[Feature]SqlGenerator`, build identifiers with:
+- `SqlBuilderHelper.Regclass(table, schema)` → `'schema."table"'` (for `create_hypertable` and other regclass arguments)
+- `SqlBuilderHelper.QualifiedIdentifier(table, schema)` → `"schema"."table"` (for `ALTER TABLE` etc.)
+- `SqlBuilderHelper.QuoteIdentifier(column)` → `"column"`
 
 NEVER manually construct qualified names or handle quoting yourself.
 
@@ -243,10 +196,12 @@ Once the feature initializer completes, relaunch this agent to implement the mig
 Implemented Components:
 - Internals/Features/[Feature]/[Feature]ModelExtractor.cs
 - Internals/Features/[Feature]/[Feature]Differ.cs
-- Generators/[Feature]OperationGenerator.cs
-- Updated: Internals/TimescaleMigrationsModelDiffer.cs
-- Updated: TimescaleDbMigrationsSqlGenerator.cs
-- Updated: Design/TimescaleCSharpMigrationOperationGenerator.cs
+- Generators/[Feature]SqlGenerator.cs
+- MigrationExtensions/[Feature]MigrationExtensions.cs
+- Design/Generators/[Feature]CSharpGenerator.cs
+- Updated: Internals/TimescaleMigrationsModelDiffer.cs (differ invocation + GetOperationPriority cases)
+- Updated: TimescaleDbMigrationsSqlGenerator.cs (Generate switch case)
+- Updated: Design/TimescaleCSharpMigrationOperationGenerator.cs (Generate switch case)
 
 Operation Priority: [X] (rationale: [explanation])
 

.claude/agents/eftdb-feature-initializer.md

@@ -141,7 +141,7 @@ Created Files:
 
 NEXT STEPS:
 → Use eftdb-feature-implementer agent to implement migration logic
-   (Creates: Differ, ModelExtractor, OperationGenerator)
+   (Creates: Differ, ModelExtractor, SqlGenerator, MigrationExtensions, CSharpGenerator)
 
 → Then use eftdb-scaffold-support agent for db-first scaffolding
    (Creates: ScaffoldingExtractor, AnnotationApplier)

.claude/agents/eftdb-scaffold-support.md

@@ -16,7 +16,7 @@ You are ONLY permitted to work within:
 You are ABSOLUTELY FORBIDDEN from:
 - Modifying any files in other projects (Runtime, Tests, Example, etc.)
 - Fixing bugs you discover in other projects
-- Changing operation generators, differs, or migration code
+- Changing SQL/C# generators, migration extensions, differs, or migration code
 - Altering the core runtime library
 
 If you encounter bugs or missing functionality in other projects, you MUST:

.claude/agents/pr-code-reviewer.md

@@ -34,10 +34,12 @@ You are an expert code reviewer specializing in Entity Framework Core extensions
 
 4. **Critical Pattern Verification**
    - **StoreObjectIdentifier Usage**: Confirm `GetColumnName(storeIdentifier)` is used for column name resolution to support naming conventions
-   - **Quote Escaping**: Verify `isDesignTime` parameter is correctly passed to SQL generators
+   - **Generator Split**: Verify runtime SQL lives in `Generators/[Feature]SqlGenerator.cs` and design-time output in `Design/Generators/[Feature]CSharpGenerator.cs`; identifiers use `SqlBuilderHelper` (`Regclass`/`QualifiedIdentifier`/`QuoteIdentifier`)
+   - **Migration Extensions**: Confirm `MigrationExtensions/[Feature]MigrationExtensions.cs` adds the operation to `migrationBuilder.Operations`
+   - **Diff Context**: Verify differs accept `FeatureDiffContext` and resolve renames via it
    - **Annotation Storage**: Check that feature metadata uses centralized annotation constants
    - **Default Values**: Ensure `DefaultValues.cs` constants are referenced instead of hardcoded values
-   - **Continuous Aggregate Encoding**: Validate colon-delimited aggregate function strings follow the correct format
+   - **Continuous Aggregate Encoding**: Validate `ContinuousAggregateFunction` values and the colon-delimited annotation format follow the correct format
 
 5. **Project Structure Compliance**
    - Verify files are in correct namespaces and directories