[Schema Consistency] Schema Consistency Check - 2026-03-29

[github-actions[bot]]

Schema consistency analysis run for 2026-03-29. 3 inconsistencies found across engine configuration, removed features, and firewall config extraction.

Strategy used: Strategy-40 — Engine Configuration Dead Code + FirewallConfig Lifecycle Audit (NEW approach, day 88 mod 10 = 8)

---

Summary

#	Severity	Area	Finding
1	🔴 HIGH	Schema ↔ Parser + Code dead path	engine.firewall not in schema, and extracted but never consumed
2	🔴 HIGH	Schema ↔ Parser (removed feature)	error_patterns in schema/examples but removed from code
3	🟡 MEDIUM	Schema ↔ Parser (silent drop)	network.firewall.cleanup-script in schema but not extracted

---

Critical Issues

1. engine.firewall — doubly-dead configuration (HIGH)

Schema: All three object variants of engine_config (oneOf[1] with id, oneOf[2] with runtime, oneOf[3] with id+display-name) have additionalProperties: false. None include firewall as a property.

Code: ExtractEngineConfig() in pkg/workflow/engine.go:280–314 extracts engineObj["firewall"] into EngineConfig.Firewall, parsing sub-fields enabled, version, log-level, and cleanup-script. However, EngineConfig.Firewall is never read after being set — no consuming code exists anywhere in pkg/.

Impact: Users writing engine: { id: copilot, firewall: { version: v1.0.0 } } get a schema validation error (because additionalProperties: false), and even if schema validation were bypassed, the configuration would be silently ignored. The entire engine.firewall extraction branch in engine.go:280–314 is dead code.

Fix: Either add firewall to the engine_config.oneOf[1].properties in schema and wire EngineConfig.Firewall through to the compilation pipeline, or remove the extraction code and the Firewall field from EngineConfig.

Relevant code

// engine.go:280-314 — extraction happens here
if firewall, hasFirewall := engineObj["firewall"]; hasFirewall {
    if firewallObj, ok := firewall.(map[string]any); ok {
        firewallConfig := &FirewallConfig{}
        // ... populates firewallConfig ...
        config.Firewall = firewallConfig  // set but NEVER READ
    }
}

// EngineConfig struct — Firewall field defined but never consumed
type EngineConfig struct {
    // ...
    Firewall *FirewallConfig  // AWF firewall configuration
    // ...
}

Schema variants (all have additionalProperties: false):

• oneOf[1]: id, version, model, max-turns, max-continuations, concurrency, user-agent, command, env, error_patterns, config, agent, api-target, args
• oneOf[2]: runtime, provider
• oneOf[3]: id, display-name, description, runtime-id, provider, models, auth, options

---

2. error_patterns — removed from code but still in schema and examples (HIGH)

Schema: error_patterns is defined in engine_config.oneOf[1].properties as a valid array of error pattern objects (with id, pattern, level_group, message_group, description sub-fields).

Example workflow: pkg/cli/workflows/example-custom-error-patterns.md demonstrates the feature with sample patterns:

engine:
  id: copilot
  error_patterns:
    - pattern: 'CUSTOM_ERROR:\s+(.+)'
      level_group: 0
      message_group: 1

Code reality: ExtractEngineConfig() does not extract engineObj["error_patterns"]. The EngineConfig struct has no ErrorPatterns field. The feature was removed: pkg/cli/audit_report.go:293 says "// No error/warning extraction since error patterns have been removed" and pkg/cli/logs_report.go:922–924 says "// logErrorAggregator and related functions have been removed since error patterns are no longer supported".

Note: GetErrorPatterns mentioned in copilot_engine_tools.go:14 is a different concept (extracting built-in patterns from Copilot CLI logs), not user-defined error_patterns.

Impact: Users who configure error_patterns per the schema/example will have their configuration silently ignored. The example workflow ships dead configuration as a reference.

Fix: Remove error_patterns from engine_config.oneOf[1].properties in the schema. Archive or remove example-custom-error-patterns.md. Add a schema $comment noting the feature was removed if migration guidance is needed.

Evidence of removal

// pkg/cli/audit_report.go:293
// No error/warning extraction since error patterns have been removed

// pkg/cli/logs_report.go:922–924
// logErrorAggregator and related functions have been removed since error patterns are no longer supported
// buildCombinedErrorsSummary has been removed since error patterns are no longer supported

Schema still has (should be removed):

"error_patterns": {
  "type": "array",
  "description": "Custom error patterns for validating agent logs",
  "items": { ... }
}

---

Schema/Parser Mismatch

3. network.firewall.cleanup-script — in schema but not extracted (MEDIUM)

Schema: The object variant of network.firewall (network.oneOf[1].properties.firewall.oneOf[3]) defines these properties: args, version, log-level, ssl-bump, allow-urls, cleanup-script.

Code: extractFirewallConfig() in pkg/workflow/frontmatter_extraction_security.go:99–147 extracts from network.firewall: args, version, log-level, ssl-bump, allow-urls — but not cleanup-script. The CleanupScript field in FirewallConfig is only written from engine.firewall (which is itself dead code per finding #1) and never read in any compilation step.

Impact: Users who set network.firewall.cleanup-script according to the schema will have their setting silently dropped. The field is inert in all code paths.

Fix: Either implement cleanup-script extraction in extractFirewallConfig() and wire it through to the AWF invocation, or remove it from the network.firewall schema definition and the FirewallConfig struct.

Comparison

Schema (network.firewall object variant):

{
  "args": { ... },
  "version": { ... },
  "log-level": { ... },
  "ssl-bump": { ... },
  "allow-urls": { ... },
  "cleanup-script": { ... }   ← defined in schema
}

extractFirewallConfig() (frontmatter_extraction_security.go:99–147):

// extracts: args, version, log-level, ssl-bump, allow-urls
// MISSING: cleanup-script

FirewallConfig.CleanupScript: defined in struct, only written from dead engine.firewall extraction, never read in compilation.

---

Recommendations

1. engine.firewall (HIGH): Decide on intent — implement it (add to schema, wire to compilation) or remove dead extraction code from engine.go:280–314 and drop EngineConfig.Firewall.

2. error_patterns (HIGH): Remove from schema engine_config.oneOf[1].properties and retire/remove example-custom-error-patterns.md. The feature has been removed from the runtime.

3. network.firewall.cleanup-script (MEDIUM): Implement extraction in extractFirewallConfig() and wire CleanupScript to the AWF container invocation, or remove from schema and struct.

---

Strategy Performance

• Strategy: Engine Configuration Dead Code + FirewallConfig Lifecycle Audit (new approach)
• Findings: 3
• Effectiveness: HIGH — tracing struct field lifecycle (write-only vs. read) is a productive pattern for finding dead config paths
• Should Reuse: YES — lifecycle audits (track field from extraction → struct → consumption) are repeatable across other config types (SandboxConfig, RateLimitConfig, etc.)

Next Steps

• Fix or remove engine.firewall extraction dead code (engine.go:280–314, EngineConfig.Firewall field)
• Remove error_patterns from schema and retire example workflow
• Fix network.firewall.cleanup-script extraction or remove from schema
• Apply lifecycle audit pattern to other config structs (SandboxConfig, RateLimitConfig, QmdToolConfig) to find similar dead paths

References: §23701370406

AI generated by Schema Consistency Checker · history

• expires on Mar 30, 2026, 4:39 AM UTC

[pelikhan]

@copilot clean out all 3 entries from JSON schema

[pelikhan]

/plan in a single issue

[github-actions[bot]]

🚀 Plan Command has started processing this discussion comment