What happened
On PR #5689, the initial roadmap rewrite (commit b7537e0, Jul 28) deleted the entire "Foundation (April-May 2026)" section from docs/roadmap.md — 12 lines of historical milestone content (MVP date, Konflux repo count, dogfooding claim) that appeared nowhere else in the live docs. The review agent approved this commit with "Looks good to me" and did not flag the deletion. Human reviewer waynesun09 caught it as a medium-severity finding the next day, noting the section had survived two prior roadmap rewrites. The author agreed the content was worth preserving and created docs/archived-roadmap.md (732 lines) in response — a significant rework that would not have happened without human review. The review agent ran from the agents repo fullsend-ai/agents at commit 5e98b13.
What could go better
The review agent's docs sub-agent does not check whether a PR removes entire named sections or large blocks of content from living documents without mentioning the removal in the PR body. This is a different gap from fullsend#5401 (which covers citation/source link removal using markdown link counting). The Foundation section contained prose and facts, not citations — it would not trigger a link-removal heuristic. The detection mechanism here is structural: a named heading and its content block were deleted, the PR description did not mention this deletion, and the content was historical/archival in nature. Confidence is moderate-high that this pattern is generalizable — living documents like roadmaps, changelogs, and status pages routinely rotate content, and unintentional information loss is a recurring risk. Uncertainty: some section deletions are intentional cleanup, so the check should flag rather than block.
Proposed change
Add a content-deletion detection check to the review agent's docs dimension sub-agent (likely in the docs sub-agent definition under agents/ or the review orchestrator prompt in fullsend-ai/agents). The check should: (1) Identify when a PR removes an entire markdown section (heading + body, roughly 5+ lines) from a document in docs/. (2) Check whether the PR body mentions the removal (e.g., references the section name, says "removed", "deleted", or "archived"). (3) If the removal is unmentioned, emit an informational finding: "This PR removes the section '[heading]' (N lines) from [file]. If this content has historical value, consider archiving it. If the removal is intentional, mention it in the PR description." Severity should be informational or low — this is a quality signal to prompt the author, not a blocking defect.
Validation criteria
On the next 5 docs PRs in fullsend-ai/fullsend that delete an entire markdown section (heading + 5+ lines of content) from a living document under docs/, the review agent should emit an informational finding noting the deletion. False-positive rate should stay under 30% (i.e., at least 70% of flagged deletions should be cases where the author either forgot to mention the removal or where archival would genuinely add value). Measure over a 60-day window after implementation.
Generated by retro agent from fullsend-ai/fullsend#5689
What happened
On PR #5689, the initial roadmap rewrite (commit
b7537e0, Jul 28) deleted the entire "Foundation (April-May 2026)" section fromdocs/roadmap.md— 12 lines of historical milestone content (MVP date, Konflux repo count, dogfooding claim) that appeared nowhere else in the live docs. The review agent approved this commit with "Looks good to me" and did not flag the deletion. Human reviewer waynesun09 caught it as a medium-severity finding the next day, noting the section had survived two prior roadmap rewrites. The author agreed the content was worth preserving and createddocs/archived-roadmap.md(732 lines) in response — a significant rework that would not have happened without human review. The review agent ran from the agents repofullsend-ai/agentsat commit5e98b13.What could go better
The review agent's docs sub-agent does not check whether a PR removes entire named sections or large blocks of content from living documents without mentioning the removal in the PR body. This is a different gap from fullsend#5401 (which covers citation/source link removal using markdown link counting). The Foundation section contained prose and facts, not citations — it would not trigger a link-removal heuristic. The detection mechanism here is structural: a named heading and its content block were deleted, the PR description did not mention this deletion, and the content was historical/archival in nature. Confidence is moderate-high that this pattern is generalizable — living documents like roadmaps, changelogs, and status pages routinely rotate content, and unintentional information loss is a recurring risk. Uncertainty: some section deletions are intentional cleanup, so the check should flag rather than block.
Proposed change
Add a content-deletion detection check to the review agent's docs dimension sub-agent (likely in the docs sub-agent definition under
agents/or the review orchestrator prompt infullsend-ai/agents). The check should: (1) Identify when a PR removes an entire markdown section (heading + body, roughly 5+ lines) from a document indocs/. (2) Check whether the PR body mentions the removal (e.g., references the section name, says "removed", "deleted", or "archived"). (3) If the removal is unmentioned, emit an informational finding: "This PR removes the section '[heading]' (N lines) from [file]. If this content has historical value, consider archiving it. If the removal is intentional, mention it in the PR description." Severity should be informational or low — this is a quality signal to prompt the author, not a blocking defect.Validation criteria
On the next 5 docs PRs in fullsend-ai/fullsend that delete an entire markdown section (heading + 5+ lines of content) from a living document under docs/, the review agent should emit an informational finding noting the deletion. False-positive rate should stay under 30% (i.e., at least 70% of flagged deletions should be cases where the author either forgot to mention the removal or where archival would genuinely add value). Measure over a 60-day window after implementation.
Generated by retro agent from fullsend-ai/fullsend#5689