refactor: standardize skill documentation on references/ layout - #156
Conversation
rhartuv
changed the title
rhartuv
added
documentation
RedTanny
left a comment
RedTanny
left a comment
There was a problem hiding this comment.
all high issues resolve good work
new review found some medium and low issues please check
Request changes
Follow-up c871bf6 closed the previous High findings (automation INDEX/SOURCES, sre INDEX/README/catalog, docs-tree coverage, REBALANCE_MANUAL.md). Local Compass + both link validators pass on this head.
Three Medium items from the last merge bar are still open and belong in this PR.
| Severity | Location | Finding |
|---|---|---|
| Medium | rh-virt/SKILL_TEMPLATE.md:198-400 |
Still uses ../../docs/troubleshooting/…. New virt skills from this template will fail the new Compass check. |
| Medium | scripts/validate_skill_doc_links.py:123 |
(skill_dir / path).resolve() before is_symlink() kills chain detection. Regression vs main. skill_dir / normalized already handles ./references/. |
| Medium | scripts/validate_skills_tier1.py:40, validate_skills_tier2.py:690 |
Tier 1 still allows docs/; Tier 2 still suggests references/ or docs/. New skills can pass T1 and fail Compass. |
| Medium | Compass layout checks | No tests for leftover skills/*/docs/, nested references/references/, README docs/ links, or the .resolve() regression. |
| Low | ocp-admin/README.md, rh-virt/README.md |
Architecture trees still draw pack-level docs/ (those dirs are gone). |
| Low | rh-developer/.catalog/documentation_section.md:14 |
Still says “See docs/”. Files now live under skills/*/references/. |
| Low | rh-sre/skills/execution-summary/SKILL.md; rh-developer incident-triage, debug-rbac, debug-scc |
Plain-text docs/ leftovers (not markdown links, so CI misses them). |
Please land the three Mediums in this PR: template → references/troubleshooting/…, restore symlink-chain detection on the unresolved path, drop docs/ from Tier 1 / Tier 2.
RedTanny
left a comment
RedTanny
left a comment
There was a problem hiding this comment.
Approve
8f6d951 closed the remaining Mediums from the last review. Local Compass + both link validators + scripts/test_validate_compass_layout.py (6 tests) pass on this head. GitHub CI is green (Tier 1, Tier 2, codespell, MCP tool check).
| Previous finding | Status |
|---|---|
Medium rh-virt/SKILL_TEMPLATE.md ../../docs/troubleshooting/… |
Fixed — examples use references/troubleshooting/… |
Medium validate_skill_doc_links.py .resolve() before is_symlink() |
Fixed — chain check runs on the unresolved path; regression test covers it |
Medium Tier 1 allows docs/; Tier 2 suggests docs/ |
Fixed — Tier 1 errors on docs/; Tier 2 hint is references/ only |
| Medium no Compass layout tests | Fixed — leftover docs/, nested references/references/, forbidden docs/ links, README docs/ links, symlink-chain, Tier 1 rejection; wired into make validate / validate-structure |
| Low ocp-admin / rh-virt README trees | Fixed |
Low rh-developer catalog “See docs/” |
Fixed |
Low execution-summary + debug-rbac/scc/incident-triage plain-text docs/ |
Fixed |
Previous Highs from 5ed196f remain closed (automation INDEX/SOURCES, sre INDEX/README/catalog, docs-tree coverage, REBALANCE_MANUAL.md). No skills/*/docs/ or nested references/references/ remain.
Non-blocking leftovers if you want a cleanup commit later: ocp-admin/AGENTS.md still cites pack-level docs/*.md; shared debugging-patterns.md still says docs/python-s2i-entrypoints.md; virt troubleshooting INDEX still says docs/troubleshooting/ when suggesting a new category file; pack-level rh-virt/references/troubleshooting/.ai-index/ is leftover after the files moved under skills/*/references/.
r2dedios
left a comment
r2dedios
left a comment
There was a problem hiding this comment.
I noticed that the files common-issues.md and live-doc-lookup.md are repeated multiple times in rh-ai-engineer plugin. Why is that so? The rest of doc files are symlinks so I suppose this could be a mistake.
r2dedios
left a comment
r2dedios
left a comment
There was a problem hiding this comment.
Just a tiny mistake on a link. The rest looks good!
There was a problem hiding this comment.
| - [CLAUDE.md](../../../CLAUDE.md) — entity kinds, namespaces, reference formats |
This relative link is broken. It refers to .claude/CLAUDE.md instead of ./CLAUDE.md
3 participants
Refactor: Standardize skill documentation on
references/layout and consolidate Compass validationSummary
This PR migrates agentic pack skills from the legacy
docs/convention to the agent-plugins.orgreferences/layout, removes leftover duplicate directories, flattens accidentalreferences/references/nesting, and consolidates validation so regressions are caught in CI.Jira ticket
Motivation
Skill-local documentation must live under
references/, notdocs/. The olddocs/references/structure caused double-nestedreferences/references/folders after migration. Pack-leveldocs/duplicates also lingered afterreferences/was introduced.Changes
Documentation migration (all packs)
skills/<name>/docs/→skills/<name>/references/across 46+ skillsdocs/directories after migrationreferences/(not deleted):references/INDEX.md,SOURCES.md, and.ai-index/with paths to canonical skill filesreferences/; updatedINDEX.mdand.ai-index/JSON to point atskills/*/references/canonical filesdocs/trees (ocp-admin,rh-sre,rh-virt,rh-automation) where skill-localreferences/already held the contentdocs/...toreferences/...(including./references/inSKILL.md).catalogfragments still pointing atdocs/INDEX.md(rh-sre,rh-automation,rh-developer,ocp-admin,rh-virt)references/references/directories intoskills/<name>/references/rh-virt/skills/vm-rebalance/REBALANCE_MANUAL.md)docs/path examples in skills (execution-summary,incident-triage,debug-rbac,debug-scc) withskills/.../references/...orreferences/...formsValidation (single entry point + regression tests)
scripts/validate_compass_manifests.pyto enforce:skills/<name>/docs/references/references/nestingdocs/markdown linksreferences/symlinks targetingdocs/pathsscripts/validate_skill_references_layout.py(merged into compass validation)validate_skill_doc_links.py:references/...and./references/...REBALANCE_*.md, etc.) in addition toSKILL.md.resolve()(avoids false negatives on shared-pool symlinks)validate_docs_tree_links.pyto scan:skills/*/references/**/*.md*.mdunder each skill directoryreferences/**/*.mdand leftoverdocs/**/*.md.cataloglinks underreferences/,skills/, and staledocs/(migration regressions fail CI)validate_skills_tier1.py:docs/is no longer an allowed skill subdirectory (error, not warning)validate_skills_tier2.py: large-skill guidance suggestsreferences/onlyscripts/test_validate_compass_layout.pyregression tests (leftoverdocs/, nestedreferences/references/, forbiddendocs/links, READMEdocs/links, symlink-chain detection, Tier 1docs/rejection)make validateandmake validate-structureAuthoring standards
SKILL_DESIGN_PRINCIPLES.md— requiresreferences/, forbids nestedreferences/references/, documents symlink pool link rulescompass-manifest-maintenanceskill — workflow to scan, migrate, flatten, deletedocs/, fix symlinks, validate viamake validate-compass-manifestsagentic-contribution-skill— createsreferences/, notdocs/rh-virt/SKILL_TEMPLATE.md— troubleshooting examples usereferences/troubleshooting/...(new virt skills pass Compass layout checks).catalog/documentation_section.mdfragments, andcollection.yamlembedded_docpaths updatedNote for Reviewers:
Due to file relocation, this PR touches ~700 files. Most changes are path updates and moved files.
Test plan
make validatepasses locallyuv run python scripts/validate_skill_doc_links.py— passes (70 skills, 388 reference links)uv run python scripts/validate_docs_tree_links.py— passes (444 files)skills/*/docs/directories remainskills/*/references/references/directories remainname,description)name,description)