feat(workflows): automate GitHub backlog grooming - #2778
feat(workflows): automate GitHub backlog grooming#2778Chris Montazer (rezatnoMsirhC) wants to merge 32 commits into
Conversation
- add weekly advisory backlog assessment - persist cursor state in a validated tracker - add approved interactive writeback - cover continuation and documentation behavior 🧹 - Generated by Copilot
…t/2558-automated-backlog-grooming
- create the tracker when absent - replace and reopen the sole existing tracker - fail closed on ambiguous marker matches - align grooming policy and agent behavior - add tracker lifecycle and persistence tests 🔄 - Generated by Copilot
…t/2558-automated-backlog-grooming
- reconcile issues with code, PRs, commits, and releases - validate structured tracker reports and supersession lineage - expand publisher, continuation, and mutation-safety tests 🔍 - Generated by Copilot
…t/2558-automated-backlog-grooming
- align workflow and policy cursor semantics - cover priority-only assessment cohorts 🐛 - Generated by Copilot
- add deterministic manifest and read-only shard result contracts - bound aggregate AI credits and reusable worker permissions - align agent policy and focused regression coverage ⚙️ - Generated by Copilot
- pass the existing Copilot engine secret to the reusable worker - assert the secret boundary in the owning regression suite 🔐 - Generated by Copilot
- interpolate trusted caller inputs into the agent prompt - regenerate the gh-aw lock and assert candidate binding 🧭 - Generated by Copilot
- validate prior cursor as bounded proof provenance - include it in the immutable manifest and regression contract 🧭 - Generated by Copilot
- fan out deterministic read-only shard workers - validate complete artifacts and injected conflicts 🧪 - Generated by Copilot
- bind workflow and generated job concurrency to shard identity - assert canonical and compiled concurrency groups 🧭 - Generated by Copilot
- enforce measured sharding and aggregate credit ceilings - reject invalid fan-in before constructing a canonical aggregate - preserve manifest order and last-assessed cursor semantics 🧭 - Generated by Copilot
- orchestrate authenticated snapshot waves and checkpoint recovery - add fail-closed proof, reduction, and manual publication paths - document limits and cover contracts with focused tests 🧭 - Generated by Copilot
…t/2558-automated-backlog-grooming
🎨 - Generated by Copilot
- preserve role checks with a provenance-bound bot grant - use binary-safe artifact downloads and focused contracts 🔒 - Generated by Copilot
- persist authenticated reports on a durable history branch - deploy report history through the existing Docusaurus Pages workflow - schedule first-Monday monthly sweeps and retain manual recovery ⚙️ - Generated by Copilot
…t/2558-automated-backlog-grooming
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2778 +/- ##
==========================================
+ Coverage 82.98% 83.18% +0.19%
==========================================
Files 183 184 +1
Lines 33787 34378 +591
Branches 25 25
==========================================
+ Hits 28038 28597 +559
- Misses 5746 5778 +32
Partials 3 3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Eval Execution✅ Status: Passed — no merge-blocking failures (40 advisory assertion failure(s) present)
|
- preserve cursor, deferred-state, and comment safety contracts - restore docs migration and deterministic eval coverage - scope Poutine acknowledgement to the generated workflow 🔒 - Generated by Copilot
- align synthetic grooming prompts with read-only evidence retrieval - validate compiler-owned action pins through the gh-aw manifest 🔧 - Generated by Copilot
- provide exact report headers in the synthetic shard contract - require canonical run and issues JSON keys 🧪 - Generated by Copilot
- delete proof-only workflows, helper, and tests - retain production workflow documentation and contract checks 🧹 - Generated by Copilot
Katrien De Graeve (katriendg)
left a comment
Katrien De Graeve (katriendg)
left a comment
There was a problem hiding this comment.
The immutable sweep, provenance, and read-only assessment design is substantial, I do have some comments and suggestions. Please also separate reusable project-planning grooming behavior from workflow-only shard/publication mechanics and reconcile issue #2558 plus the PR description with the delivered orchestrator/worker/publisher architecture. Docusaurus should remain optional/TBC rather than a core dependency.
One thing I've reflected on, and may be visible in some comments is the reflection on shipping the sub-agent/instructions into the Plugins/Extension, while the full value becomes its integration with AW and the workflows, which are always HVE Core specific. I wonder if there is a version where we first test it within repo, and then add it to existing backlog manager as well as release the subagent. What do you think? Does it provide clear usage pattern without the workflows?
Finally, please upgrade the gh-aw CLI to v0.86.2 and run bare gh aw compile to regenerate and consistency-check ALL agentic Markdown workflows and their generated locks; backlog-groom.lock.yml is still v0.83.4 while the other six locks already report v0.86.2.
Additional non-inline comments:
- Reconcile issue #2558 with the delivered first-Monday orchestrator, reusable worker, publisher, report branch, tracker body replacement, and optional Pages output. Either update and approve its criteria for this architecture or split the publication work from the core assessment scope.
- Update the PR description so its summary, changed areas, validation evidence, and rollout notes describe the actual orchestrator/worker/publisher architecture and distinguish core tracker delivery from optional/TBC report-history and Docusaurus publication.
- invoke validation directly in the existing job - preserve canonical digests, schemas, outputs, and permissions - add focused Pester coverage and remove the composite action ♻️ - Generated by Copilot
- correct trust, ordering, deferral, and tracker freshness checks - isolate optional report publication behind an exact repository variable - separate reusable policy and compile generated locks with gh-aw v0.86.2 🔒 - Generated by Copilot
📐 - Generated by Copilot
🧪 - Generated by Copilot
I've separated the reusable grooming policy from the repository-specific automation. The shared instruction now owns workflow-neutral evidence gathering, similarity assessment, advisory dispositions, compact reporting, and approved handoff behavior. Shard inputs, cursor state, safe-output schema, deterministic digesting, artifacts, checkpoints, and publication remain in the repository-only worker and workflows. Backlog Manager now applies the reusable policy directly to ordinary issue inventory instead of dispatching the workflow-bound worker. Plugin, extension, and generated-documentation validation confirm that the worker is no longer distributed while the reusable instruction remains available.
That concern makes sense. In this PR, I've kept the workflow-bound worker repository-only and removed it from the plugin and extension. Backlog Manager now applies the reusable grooming instruction directly, which provides a workflow-independent usage pattern. It can assess ordinary issue inventory, produce evidence-backed advisory findings, and hand off only explicitly approved Update or Comment operations. The reusable instruction remains distributed, but I don't think the workflow-bound subagent yet demonstrates enough standalone value without the HVE Core workflows to justify releasing it. My preference is to test that worker within this repository first and distribute a standalone subagent only after we can demonstrate a clear workflow-neutral contract and usage pattern.
I've upgraded
Issue #2558 described a different execution and publication architecture. I've reconciled it with the delivered first-Monday orchestrator, repository-only worker, authenticated publisher, and compact tracker-body replacement as the core delivery path. Report history, the report branch, and Pages deployment are now documented as a separate, default-off capability enabled only when
I've updated the PR description to reflect the delivered orchestrator, repository-only worker, deterministic validation and reduction, authenticated publisher, and compact tracker-body architecture. It now distinguishes core tracker delivery from the default-off report-history and Docusaurus Pages path, and includes the current validation evidence and separate rollout requirements for each mode. |
|
Chris Montazer (@rezatnoMsirhC) - MONSTER ... love this ... will review tomorrow and thanks for the swing at this. |
Katrien De Graeve (katriendg)
left a comment
Katrien De Graeve (katriendg)
left a comment
There was a problem hiding this comment.
Thank you for the extensive follow-up, I really like the approach of first validating this and then reviewing what to expose in the plugin and distributions.
Most original comments are resolved, including scheduled activation, manual replay authentication, tracker freshness, deferred semantics, accessibility, distribution boundaries, issue scope, and the PR narrative. My reviewer found two blocking paths remain: cursor-prioritized wraparound order reaches the manifest but is still rejected by the worker and PowerShell validator, and optional Pages deployment trusts the mutable report-branch head without proving publisher provenance. Duplicate continuation dispatches also enter validation after omitting the manifest. I left smaller follow-ups for validator schema parity, PowerShell help, the core rollout checklist, and the stale gh-aw version in the runbook. The latter are smaller ones.
Really keen to see this merged in and learnings we get from the usage. Thanks for your work on this!
| } | ||
| if (!Array.isArray(orderedCandidateIds) || orderedCandidateIds.some( | ||
| (issue, index) => !Number.isInteger(issue) || issue <= 0 || | ||
| (index > 0 && issue <= orderedCandidateIds[index - 1]), |
There was a problem hiding this comment.
The planned sequence can legitimately wrap or put priority issues first, so this ascending-order check rejects valid input such as [105, 1, 2]. Please validate positive unique IDs while preserving their order, then compare cohort and report coverage as sets. For example:
const candidateSet = new Set(orderedCandidateIds);
const validCandidates = Array.isArray(orderedCandidateIds) &&
orderedCandidateIds.every((issue) => Number.isInteger(issue) && issue > 0) &&
candidateSet.size === orderedCandidateIds.length;
const cohortSet = new Set([...priorityCandidateIds, ...roundRobinCandidateIds]);
const sameMembers = cohortSet.size === candidateSet.size &&
[...cohortSet].every((issue) => candidateSet.has(issue));Apply the same order-versus-membership distinction in the PowerShell validator and regenerate the lock.
| } | ||
| await github.rest.git.getCommit({ ...context.repo, commit_sha: requestedRef }); | ||
| } | ||
| const authenticatedRef = requestedRef || reportRef.object.sha; |
There was a problem hiding this comment.
This proves only that the requested SHA equals the current mutable report-branch head; it does not prove that the authenticated publisher created that commit. A collaborator can update the branch and have an ordinary main docs push publish its subtree. Please consume and authenticate an immutable publisher artifact (workflow path, successful run, default-branch source SHA, artifact identity, and digest). At minimum, do not stage reports when report-ref is empty and enforce a ruleset that only the publisher identity can update this branch.
| - assess | ||
| if: >- | ||
| ${{ always() && needs.plan.result == 'success' && | ||
| needs.plan.outputs.mode != 'calendar-noop' && |
There was a problem hiding this comment.
An accepted duplicate checkpoint returns complete-noop without uploading a manifest, but this job still runs and downloads an empty artifact ID. Exclude that terminal mode here:
| needs.plan.outputs.mode != 'calendar-noop' && | |
| needs.plan.outputs.mode != 'calendar-noop' && | |
| needs.plan.outputs.mode != 'complete-noop' && |
| $Issue = $Row.GetProperty('issue') | ||
| $IssueId = if (Test-SafeJsonInteger -Element $Issue -Minimum 1) { $Issue.GetInt64() } else { -1 } | ||
| $AssessmentStatus = $Row.GetProperty('assessment_status').GetString() | ||
| if (-not (Test-ExactJsonKeys -Element $Row -Keys $RowKeys) -or |
There was a problem hiding this comment.
Exact keys alone do not reproduce the producer's row contract. This boundary still accepts shapes the worker rejects, including array-valued acceptance_signals and empty repository_evidence. Please mirror the canonical field types, bounds, enum/disposition relationships, and non-empty evidence requirements here, and make the validator fixtures valid producer outputs.
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| #region Functions | ||
| function ConvertTo-JavaScriptJsonString { |
There was a problem hiding this comment.
Repository PowerShell guidance requires block comment-based help for every function (ExportedOnly = $false, BlockComment = $true). Please add concise help to all eleven helpers. For example, this first one could start with:
| function ConvertTo-JavaScriptJsonString { | |
| <# | |
| .SYNOPSIS | |
| Converts a value to a JavaScript-compatible JSON string. | |
| .PARAMETER Value | |
| The string value to encode. | |
| .OUTPUTS | |
| System.String | |
| #> | |
| function ConvertTo-JavaScriptJsonString { |
|
|
||
| The workflow rejects capacity or AIC values that exceed safe integer | ||
| arithmetic. `max-parallel: 2` bounds concurrent model workers. gh-aw is pinned | ||
| at `v0.83.4`; its worker uses the Copilot engine, a 20-minute timeout, and at |
There was a problem hiding this comment.
All seven generated locks and the action lock now identify gh-aw v0.86.2.
| at `v0.83.4`; its worker uses the Copilot engine, a 20-minute timeout, and at | |
| at `v0.86.2`; its worker uses the Copilot engine, a 20-minute timeout, and at |
4 participants
Description
This PR added an advisory backlog-grooming system that assesses the complete open issue inventory in bounded, immutable waves while keeping model-facing jobs read-only. Digest-bound manifests and checkpoints authenticate continuation and recovery, and a separate publisher remains the only workflow with issue-write authority.
Orchestration and publication
backlog-grooming-reportsbranch./backlog-grooming/without replacing the production site root.Policy, validation, and distribution
Related Issue(s)
Related to #2558
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
hve-builderand addressed all actionable findings.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md).github/hooks/*/*.json)evals/)Other:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
None
Testing
Local validation
main.github/skills/project-planning/security-planning/references/data-classification.mdHosted proof artifacts
The hosted sweep produced aggregate digest
fca5892445cd51baad953ca8df4b16a2278478c96d2cd5289e6c138e0b36eecd. Report-history commit9d5341f73f7371aade32305be04849fe83a079f2was deployed by the successful Pages run.Manual exploratory testing was limited to the disposable repository and its rendered Pages surfaces. No production issues were mutated during validation.
Checklist
Required Checks
AI Artifact Contributions
hve-builderreview mode to review contributionhve-builderreviewRequired Local Checks
The following local-safe validation commands must pass before merging:
npm run validate:local(blocked by the unchanged link noted above)npm run validate:docsnpm run spell-checknpm run lint:md-links(blocked by the unchanged link noted above)Security Considerations
GHCP Membership Changes
.github/agents/github/backlog-grooming.agent.md.github/instructions/project-planning/github-backlog-grooming.instructions.mdnpm run plugin:syncregenerated the root manifest, andnpm run plugin:validatepassed after the merge.Additional Notes
The disposable repository used a standalone Pages deployment and root redirect because it had no Docusaurus site. Neither test-only surface is present in this PR. Production report history is staged only beneath
/backlog-grooming/in the existing Docusaurus deployment.