Repository Quality Improvement Report — Validation System Architecture Health (2026-03-23)

[github-actions[bot]]

🎯 Repository Quality Improvement Report — Validation System Architecture Health

Analysis Date: 2026-03-23
Focus Area: Validation System Architecture Health
Strategy Type: Custom
Custom Area: Yes — The gh-aw validator system has grown to 51+ validation files in pkg/workflow/ alone. This analysis targets two concrete, measurable architectural issues: (1) file size violations against the 300-line hard limit documented in AGENTS.md, and (2) test coverage gaps in validator files.

Executive Summary

The pkg/workflow/ package contains 51 non-test validation files with a total of ~8,700 lines of validation logic. Analysis reveals 8 files exceed the 300-line hard limit defined in AGENTS.md, and a large fraction of validators lack dedicated unit test files (relying solely on integration tests or being exercised only through compilation pipelines).

The most critical violations are safe_outputs_validation.go and safe_outputs_validation_config.go (both at 407 lines), dispatch_workflow_validation.go (363 lines), and tools_validation.go (359 lines). Additionally, permissions_validation.go (351 lines) and repository_features_validation.go (342 lines) exceed the limit. These files mix multiple distinct validation domains, making them harder to test, review, and maintain independently.

Full Analysis Report

Focus Area: Validation System Architecture Health

Current State Assessment

Metrics Collected:

Metric	Value	Status
Total validator source files	51	✅ Well-organized
Files exceeding 300-line hard limit	8	❌ Violates AGENTS.md
Largest file	407 lines (safe_outputs_validation.go)	❌
Validator files without dedicated unit test	~20	⚠️ Coverage gap
Validator test-to-source ratio (workflow pkg)	~163%	✅ High overall

Findings

Strengths

• 51 focused validator files shows clear domain separation intent
• Extensive integration test coverage (35+ safe-outputs test files)
• Most validators have well-documented function signatures and file-level comments
• AGENTS.md documents clear refactoring criteria (>300 lines → split)

Files Exceeding 300-Line Hard Limit

File	Lines	Primary Issue
safe_outputs_validation.go	407	Mixes network domain validation + target validation
safe_outputs_validation_config.go	407	Config struct is very large
dispatch_workflow_validation.go	363	Mixes file discovery utilities with validation logic
tools_validation.go	359	Mixes bash, GitHub, toolsets validation
permissions_validation.go	351	Mixes MCP permission validation + included workflow validation
repository_features_validation.go	342	Mixes repository API calls + feature checks
template_injection_validation.go	306	Single domain but very long
mcp_config_validation.go	300	At exact limit

Validator Files Without Dedicated Unit Test Files

expression_safety_validation.go (295 lines), expression_syntax_validation.go (236 lines), agent_validation.go (247 lines — only integration tests), docker_validation.go (175 lines), npm_validation.go (106 lines), pip_validation.go (192 lines), jobs_validation.go, strict_mode_env_validation.go (147 lines), strict_mode_network_validation.go (133 lines), strict_mode_permissions_validation.go (190 lines), runtime_validation.go (290 lines), lock_validation.go, firewall_validation.go, cache_validation.go, dispatch_repository_validation.go (107 lines), push_to_pull_request_branch_validation.go (119 lines), repo_memory_validation.go, template_validation.go, safe_outputs_validation.go (407 lines, has many domain tests but no safe_outputs_validation_test.go), sandbox_validation.go.

Detailed Analysis

tools_validation.go (359 lines) contains 7 distinct top-level functions spanning three domains: bash tool validation, GitHub read-only enforcement, GitHub tool config, GitHub guard policy, repos scope validation, repo pattern validation, and toolset cross-validation. The AGENTS.md guidance recommends {domain}_{subdomain}_validation.go naming. This file should become tools_bash_validation.go, tools_github_validation.go, and tools_toolsets_validation.go.

dispatch_workflow_validation.go (363 lines) has a primary validator function (validateDispatchWorkflow) plus 7 utility functions for file discovery (findWorkflowFile, extractWorkflowDispatchInputs, getCurrentWorkflowName, isPathWithinDir, etc.). The utility functions belong in a separate dispatch_workflow_helpers.go.

expression_safety_validation.go (295 lines) and expression_syntax_validation.go (236 lines)** have no dedicated test files. Template injection and expression safety are security-critical paths that deserve explicit unit test coverage rather than relying only on integration tests.

agent_validation.go (247 lines) only has integration tests (max_turns_validation_test.go). Unit tests would be faster and more precise for testing the agent validation functions.

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot coding agent execution. Please split these into individual work items for Claude to process. Each task is self-contained.

Improvement Tasks

Task 1: Split tools_validation.go by Domain

Priority: High
Estimated Effort: Medium
Focus Area: Code Organization / Validation Architecture

Description:
pkg/workflow/tools_validation.go (359 lines) violates the 300-line hard limit in AGENTS.md and mixes three distinct validation domains. Split it into three files following the {domain}_{subdomain}_validation.go naming convention.

Acceptance Criteria:

• tools_bash_validation.go contains validateBashToolConfig
• tools_github_validation.go contains validateGitHubReadOnly, validateGitHubToolConfig, validateGitHubGuardPolicy, validateReposScope, validateRepoPattern, isValidOwnerOrRepo
• tools_toolsets_validation.go contains ValidateGitHubToolsAgainstToolsets
• All existing tests pass: make test-unit
• No file exceeds 300 lines
• make fmt passes

Code Region: pkg/workflow/tools_validation.go

Split `pkg/workflow/tools_validation.go` (359 lines, violates the 300-line hard limit in AGENTS.md) into three files:

1. `pkg/workflow/tools_bash_validation.go` — move `validateBashToolConfig` here
2. `pkg/workflow/tools_github_validation.go` — move `validateGitHubReadOnly`, `validateGitHubToolConfig`, `validateGitHubGuardPolicy`, `validateReposScope`, `validateRepoPattern`, `isValidOwnerOrRepo` here  
3. `pkg/workflow/tools_toolsets_validation.go` — move `ValidateGitHubToolsAgainstToolsets` here

Delete the original `tools_validation.go` after moving all functions.

Ensure:
- All existing tests pass (`make test-unit`)
- No file exceeds 300 lines
- Run `make fmt` before committing
- No imports are broken (update imports in each new file as needed)

---

Task 2: Split dispatch_workflow_validation.go — Extract File Utilities

Priority: High
Estimated Effort: Medium
Focus Area: Code Organization / Validation Architecture

Description:
pkg/workflow/dispatch_workflow_validation.go (363 lines) exceeds the 300-line limit and mixes validation logic with file discovery utilities. The file utility functions (findWorkflowFile, isPathWithinDir, getCurrentWorkflowName, extractWorkflowDispatchInputs, extractMDWorkflowDispatchInputs, mdHasWorkflowDispatch, containsWorkflowDispatch) should be extracted to a helpers file.

Acceptance Criteria:

• dispatch_workflow_helpers.go created with file discovery/utility functions
• dispatch_workflow_validation.go retains only validateDispatchWorkflow and related validation logic
• Both files are under 300 lines
• All existing tests pass: make test-unit
• make fmt passes

Code Region: pkg/workflow/dispatch_workflow_validation.go

Refactor `pkg/workflow/dispatch_workflow_validation.go` (363 lines) to extract helper/utility functions into a new `dispatch_workflow_helpers.go` file.

Move these utility/helper functions to `pkg/workflow/dispatch_workflow_helpers.go`:
- `extractWorkflowDispatchInputs` (line 164)
- `getCurrentWorkflowName` (line 217)
- `isPathWithinDir` (line 226)
- `findWorkflowFileResult` struct and `findWorkflowFile` function (lines 248-296)
- `mdHasWorkflowDispatch` (line 301)
- `extractMDWorkflowDispatchInputs` (line 320)
- `containsWorkflowDispatch` (line 361)

Keep only `validateDispatchWorkflow` and its direct helpers in `dispatch_workflow_validation.go`.

Ensure:
- Both resulting files are under 300 lines
- All tests pass: `make test-unit`
- Run `make fmt` before committing

---

Task 3: Add Unit Tests for expression_safety_validation.go

Priority: Medium
Estimated Effort: Medium
Focus Area: Testing / Security

Description:
pkg/workflow/expression_safety_validation.go (295 lines) contains security-critical validation for expression safety but has no dedicated unit test file. This is a risk because expression safety directly impacts security. Add direct unit tests for the key validation functions.

Acceptance Criteria:

• pkg/workflow/expression_safety_validation_test.go created
• Build tag //go:build !integration added at top
• Tests cover the core expression safety validation functions
• Table-driven tests with descriptive names
• Both valid and invalid expression cases tested
• Tests use assert.* and require.* from github.com/stretchr/testify
• make fmt and make lint pass

Code Region: pkg/workflow/expression_safety_validation.go, new file pkg/workflow/expression_safety_validation_test.go

Create `pkg/workflow/expression_safety_validation_test.go` with direct unit tests for the expression safety validator in `pkg/workflow/expression_safety_validation.go`.

Requirements:
1. Add `//go:build !integration` as the FIRST LINE of the file (then a blank line, then package declaration)
2. Use table-driven tests with `t.Run()` and descriptive test names
3. Test the public-facing validation functions exported from the file
4. Also test `ValidateExpressionSafetyPublic` if present
5. Cover: valid expressions that should pass, dangerous expressions that should fail, edge cases (empty expressions, nested expressions)
6. Use `github.com/stretchr/testify/assert` and `github.com/stretchr/testify/require`
7. Add descriptive assertion messages

After creating the test file, run `make lint` to check for unused helpers or testifylint issues, then fix any found.

---

Task 4: Add Unit Tests for agent_validation.go

Priority: Medium
Estimated Effort: Small
Focus Area: Testing

Description:
pkg/workflow/agent_validation.go (247 lines) provides agent-specific validation including engine feature compatibility checks and security constraints on workflow triggers. It currently only has integration tests (max_turns_validation_test.go). Fast unit tests would improve the feedback cycle during development.

Acceptance Criteria:

• pkg/workflow/agent_validation_test.go created
• Build tag //go:build !integration at top
• Tests for validateMaxTurnsSupport with supported/unsupported engines
• Tests for validateMaxContinuationsSupport
• Tests for validateWebSearchSupport (warning vs error behavior)
• Table-driven test format
• make fmt and make lint pass

Code Region: pkg/workflow/agent_validation.go, new file pkg/workflow/agent_validation_test.go

Create `pkg/workflow/agent_validation_test.go` with unit tests for `pkg/workflow/agent_validation.go`.

Requirements:
1. Add `//go:build !integration` as the FIRST LINE (then blank line, then package declaration)
2. Write table-driven unit tests for:
   - `validateMaxTurnsSupport`: test with engines that support max-turns (e.g., copilot) and engines that don't
   - `validateMaxContinuationsSupport`: similar pattern
   - `validateWebSearchSupport`: verify it produces a warning (not error) for unsupported engines
3. Look at the function signatures in `agent_validation.go` and how engines are represented in the codebase to construct appropriate test inputs
4. Use `github.com/stretchr/testify/assert` and `require` — include assertion messages
5. After creating the test file, run `make lint` to check for issues (unused helpers, testifylint), then fix any found

Note: The existing integration tests in `max_turns_validation_test.go` use `//go:build integration` — these new tests should be unit tests (`//go:build !integration`) that don't require external resources.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-03-23	Validation System Architecture Health	Custom	Yes	8 files violating 300-line limit identified; 20 validators with missing unit tests identified

---

🎯 Recommendations

Immediate Actions (This Week)

1. Split tools_validation.go (Task 1) — Priority: High. Already over the documented limit, clean domain separation exists.
2. Split dispatch_workflow_validation.go (Task 2) — Priority: High. File discovery utilities are a clearly separate concern.

Short-term Actions (This Month)

1. Add unit tests for expression_safety_validation.go (Task 3) — Priority: Medium. Security-critical path deserves faster unit test feedback.
2. Add unit tests for agent_validation.go (Task 4) — Priority: Medium. Integration tests are slow; unit tests improve dev cycle.

Long-term Actions (This Quarter)

1. Systematic unit test coverage for all 20 validator files without dedicated unit tests — triage by file size and security sensitivity.
2. Consider a linting rule for file length in the CI to prevent future 300-line limit violations from being merged.

---

📈 Success Metrics

Track these metrics to measure improvement in Validation System Architecture Health:

• Files exceeding 300-line limit: 8 → 0 (target)
• Validators without unit tests: ~20 → < 10 (short-term target)
• Average validator file size: ~170 lines → < 150 lines (target)

---

Next Steps

1. Review and prioritize the 4 tasks above
2. Assign tasks to Copilot coding agent via planner agent
3. Track progress on improvement items
4. Re-evaluate this focus area in 30 days

---

References:

• §23439825818

Generated by Repository Quality Improvement Agent
Next analysis: 2026-03-24 — Focus area will be selected based on diversity algorithm

AI generated by Repository Quality Improvement Agent · history

• expires on Mar 24, 2026, 1:36 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Repository Quality Improvement Agent.

A newer discussion is available at Discussion #22700.