-
Notifications
You must be signed in to change notification settings - Fork 34
feat: add skill to author and maintain Compass manifests for agentic packs #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rhartuv
wants to merge
5
commits into
RHEcosystemAppEng:main
Choose a base branch
from
rhartuv:feat/compass-manifest-skill
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+457
−6
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
c625f31
feat: add skill to author and maintain Compass manifests for agentic …
rhartuv 89794ec
fix: removed the static mcp mapping tables
rhartuv 9a809a7
fix: remove pr references
rhartuv a3fe4b1
fix: add missing Compass manifest for ocp-admin/network-policy-archit…
rhartuv 445a70e
fix: fix validate skills tier2
rhartuv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,191 @@ | ||
| --- | ||
| name: compass-manifest-maintenance | ||
| description: | | ||
| Author and maintain Compass catalog-info.yaml manifests for agentic packs (skills, plugins, Locations, MCP inverse relations). Use when: | ||
| - Adding or updating a skill and its Compass manifest | ||
| - Adding a new pack to Compass registration | ||
| - MCP usage or orchestration changed in SKILL.md | ||
| - Auditing bidirectional dependsOn/dependencyOf drift against repo Compass conventions | ||
|
|
||
| File-based only: Read/Glob/Grep/Bash. For `.catalog/` marketplace metadata use create-collection. | ||
|
|
||
| NOT for: `.catalog/` metadata (use create-collection) or automated Compass registration. | ||
| model: inherit | ||
| color: magenta | ||
| license: Apache-2.0 | ||
| allowed-tools: Read Glob Grep Bash | ||
| --- | ||
|
|
||
| # /compass-manifest-maintenance Skill | ||
|
|
||
| **Audience:** Maintainers updating `catalog-info.yaml` manifests for Red Hat Compass (Backstage) registration. | ||
|
|
||
| **Goal:** Keep Compass manifests aligned with golden sources (`SKILL.md`, `mcps.json`, `AGENTS.md`) | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| **Required MCP Servers:** None — file-based skill. Do not use Compass MCP tools (`validate-entity`, `register-entity`, `get-catalog-entity`, `query-entity-graph`). | ||
|
|
||
| - Repository root as cwd. | ||
| - Read [CLAUDE.md](../../CLAUDE.md) Compass / Backstage Manifests section. | ||
| - Read [references/relationship-rules.md](references/relationship-rules.md) and [references/mcp-mapping.md](references/mcp-mapping.md). | ||
| - Templates: [assets/](assets/). | ||
|
|
||
| **Verification:** | ||
| ```bash | ||
| test -f CLAUDE.md && echo "✓ repo root" || echo "✗ wrong directory" | ||
| ``` | ||
|
|
||
| **Human Notification Protocol:** If drift audit finds violations, report file path and fix per workflow §1. | ||
|
|
||
| **Security:** Never display or expose credentials or token values. | ||
|
|
||
| ## When to Use This Skill | ||
|
|
||
| **Use when:** | ||
| - New skill needs `skills/<name>/catalog-info.yaml` and Location / inverse updates. | ||
| - Skill `allowed-tools` or orchestration changed → MCP or skill `dependsOn` must change. | ||
| - New pack registered in Compass (plugin, Location, root index, `system.yaml`). | ||
| - Drift audit: skill on disk without manifest, stale `dependencyOf`, or dangling entity refs. | ||
| - After `agentic-contribution-skill` creates a skill (run before opening PR). | ||
|
|
||
| **Do not use when:** `.catalog/collection.yaml` (use **create-collection** instead), generic Compass platform tasks, or automated Compass registration. | ||
|
|
||
| ## Workflow | ||
|
|
||
| **MCP Tool:** None — file-based; uses Read, Glob, Grep, Bash only (see `allowed-tools` in frontmatter). | ||
|
|
||
| **Parameters:** N/A — no MCP tools; inputs are `<pack>`, `<skill-name>`, and on-disk manifest paths. | ||
|
|
||
| ### 1. Add or update a skill manifest | ||
|
|
||
| 1. **Resolve** `<pack>` and `<skill-name>` — confirm `<pack>/skills/<skill-name>/SKILL.md` exists. | ||
|
|
||
| 2. **Read golden sources** (precedence): | ||
| - `SKILL.md` frontmatter: `name`, `description`, `allowed-tools` | ||
| - `SKILL.md` body: `Required MCP Servers`, `/skill-name` invocations, Dependencies, validator prerequisites | ||
| - `<pack>/mcps.json` — server keys (map via [mcp-mapping.md](references/mcp-mapping.md)) | ||
| - `<pack>/AGENTS.md` — tags/disciplines hints only (not orchestration inference) | ||
|
|
||
| 3. **Derive `dependsOn`** for the skill manifest: | ||
| - Always: `airesource:ai5-marketplace/<pack>` | ||
| - MCP: only servers the skill **actually uses** (tool prefixes + body); see [mcp-mapping.md](references/mcp-mapping.md) | ||
| - Other skills: orchestration or documented `/other-skill` invocations in `SKILL.md` | ||
| - Script-only skills: no `mcpserver:` entries | ||
|
|
||
| 4. **Write** `<pack>/skills/<skill-name>/catalog-info.yaml` from [assets/skill-catalog-info.yaml](assets/skill-catalog-info.yaml): | ||
| - `namespace: ai5-marketplace` | ||
| - `labels.distribution: external` | ||
| - `agents: []` | ||
| - `lifecycle: beta` | ||
| - `owner: group:redhat/ai5-marketplace` | ||
| - `backstage.io/source-location` → GitHub `main` branch SKILL.md URL | ||
| - `dependencyOf`: list orchestrators that `dependsOn` this skill (scan pack or update when editing orchestrator) | ||
|
|
||
| 5. **Update inverse manifests** ([relationship-rules.md](references/relationship-rules.md)): | ||
| - `<pack>/catalog-info.yaml` — `./skills/<skill-name>/catalog-info.yaml` in `spec.targets` | ||
| - `<pack>/<pack>-plugin.yaml` — `dependencyOf: airesource:ai5-marketplace/<skill-name>` | ||
| - Each `mcpserver:` in skill `dependsOn` → matching `mcps/*.yaml` `dependencyOf` | ||
| - Each skill in skill `dependsOn` → that skill's `dependencyOf` includes orchestrator | ||
|
|
||
| 6. **Reconcile plugin MCP deps** — plugin `dependsOn` = union of all `mcpserver:` refs across pack skill manifests. | ||
|
|
||
| ### 2. Add a new pack (Compass registration) | ||
|
|
||
| 1. Create `<pack>/<pack>-plugin.yaml` from [assets/plugin-catalog-info.yaml](assets/plugin-catalog-info.yaml). | ||
| 2. Create `<pack>/catalog-info.yaml` from [assets/pack-location.yaml](assets/pack-location.yaml). | ||
| 3. Add `./<pack>/catalog-info.yaml` to root `catalog-info.yaml`. | ||
| 4. Add `airesource:ai5-marketplace/<pack>` to `system.yaml` `spec.dependencyOf`. | ||
| 5. Run **create-collection** for `<pack>/.catalog/` separately. | ||
|
|
||
| ### 3. Drift / compliance audit | ||
|
|
||
| Run checks with `Glob`/`Grep`/`Bash`: | ||
|
|
||
| | Check | Rule | | ||
| |-------|------| | ||
| | Roster parity | Every `skills/*/SKILL.md` has `catalog-info.yaml` and pack Location target | | ||
| | Inverse parity | Plugin `dependencyOf` = full skill set; each skill `dependsOn` includes plugin | | ||
| | MCP inverse | Each skill `mcpserver:` in `dependsOn` → MCP `dependencyOf` includes skill | | ||
| | Skill inverse | Each skill→skill `dependsOn` → target `dependencyOf` includes source | | ||
| | Dangling refs | Every ref resolves to on-disk manifest or documented canonical MCP | | ||
| | Forbidden | No `partOf`/`hasPart` on AiResource/MCPServer; no redundant `dependsOn: system:default/agentic-plugins` on plugins | | ||
| | Namespace | Refs use `ai5-marketplace` except `mcpserver:redhat/*` and `default/agentic-plugins` | | ||
|
|
||
| Report violations with file path and fix per workflow §1. Do not weaken checks. | ||
|
|
||
| ### 4. Validate (file-based) | ||
|
|
||
| 1. Compare YAML against [assets/](assets/) and a known-good manifest in the same pack. | ||
| 2. Re-run drift audit (§3). | ||
| 3. Tier 1 on this skill if edited: `uv run python scripts/validate_skills_tier1.py .claude/skills/compass-manifest-maintenance/SKILL.md`. | ||
| 4. Post-merge: maintainer may register in Compass UI manually (out of scope for this skill). | ||
|
|
||
| **Error Handling:** | ||
| - If skill on disk has no manifest → create from `assets/skill-catalog-info.yaml` and add Location target. | ||
| - If inverse `dependencyOf` missing on plugin or MCP → update per [relationship-rules.md](references/relationship-rules.md). | ||
| - If MCP ref cannot be resolved → grep pack manifests; do not invent new `mcpserver:` refs. | ||
|
|
||
| ## Self-review checklist | ||
|
|
||
| - [ ] Skill manifest matches repo Compass conventions (`agents: []`, `labels.distribution: external`, `namespace: ai5-marketplace`, `owner: group:redhat/ai5-marketplace`). | ||
| - [ ] MCP deps derived from `SKILL.md` usage, not copied from sibling skills. | ||
| - [ ] Every new `dependsOn` has matching `dependencyOf` on the target entity. | ||
| - [ ] Pack Location lists every skill manifest path. | ||
| - [ ] Plugin `dependencyOf` lists every skill in the pack. | ||
| - [ ] No `partOf`/`hasPart` on custom kinds. | ||
|
|
||
| ## Dependencies | ||
|
|
||
| ### Required MCP Servers | ||
|
|
||
| None — file-based manifest maintenance only. | ||
|
|
||
| ### Required MCP Tools | ||
|
|
||
| None — uses Read, Glob, Grep, Bash. | ||
|
|
||
| ### Related Skills | ||
|
|
||
| - **create-collection** — for `.catalog/` marketplace metadata (not Compass manifests) | ||
| - **agentic-contribution-skill** — run before this skill when creating a new skill | ||
|
|
||
| ### Reference Documentation | ||
|
|
||
| - [CLAUDE.md](../../CLAUDE.md) — entity kinds, namespaces, reference formats | ||
| - [references/relationship-rules.md](references/relationship-rules.md) | ||
| - [references/mcp-mapping.md](references/mcp-mapping.md) | ||
|
|
||
| ## Common Issues | ||
|
|
||
| - **COMPASS-1288 drift** — updated skill `dependsOn` but forgot plugin or MCP `dependencyOf`. | ||
| - **Blanket MCP deps** — declaring every MCP in `mcps.json` instead of per-skill `allowed-tools` usage. | ||
| - **Orchestration gaps** — `remediation`-style skills missing skill→skill edges or inverse `dependencyOf` on depended skills. | ||
| - **Canonical vs owned** — Lightspeed and Security use `mcpserver:redhat/...`; do not register duplicates in `mcps/`. | ||
| - **Unregistered packs** — `rh-developer`, `rh-ai-engineer`, `rh-automation` exist on disk but are not in root Location until explicitly added. | ||
|
|
||
| ## Example usage | ||
|
|
||
| ```bash | ||
| # Roster: skills on disk missing from Location (example: rh-sre) | ||
| comm -23 \ | ||
| <(find rh-sre/skills -name SKILL.md | sed 's|.*/skills/||;s|/SKILL.md||' | sort) \ | ||
| <(grep -oP 'skills/\K[^/]+(?=/catalog-info)' rh-sre/catalog-info.yaml | sort) | ||
|
|
||
| # Tier 1 lint for this maintenance skill | ||
| uv run python scripts/validate_skills_tier1.py .claude/skills/compass-manifest-maintenance/SKILL.md | ||
|
|
||
| # Full repo validation (no Compass CI yet) | ||
| make validate | ||
| ``` | ||
|
|
||
| Audit all four registered packs: | ||
|
|
||
| ```bash | ||
| for pack in ocp-admin rh-sre rh-virt rh-basic; do | ||
| echo "=== $pack ===" | ||
| comm -23 \ | ||
| <(find "$pack/skills" -name SKILL.md 2>/dev/null | sed 's|.*/skills/||;s|/SKILL.md||' | sort) \ | ||
| <(grep -oP 'skills/\K[^/]+(?=/catalog-info)' "$pack/catalog-info.yaml" 2>/dev/null | sort) | ||
| done | ||
| ``` | ||
20 changes: 20 additions & 0 deletions
20
.claude/skills/compass-manifest-maintenance/assets/pack-location.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Template: <pack>/catalog-info.yaml (Location) | ||
| apiVersion: backstage.io/v1alpha1 | ||
| kind: Location | ||
| metadata: | ||
| name: <pack> | ||
| title: <Pack Title> Agentic Pack | ||
| description: > | ||
| Points to all AI skill catalog entities in the <pack> agentic pack. | ||
| annotations: | ||
| agentic-plugins.redhat.com/version: "1.0" | ||
| agentic-plugins.redhat.com/repo: "agentic-plugins" | ||
| agentic.redhat.com/lola-module: "true" | ||
| agentic.redhat.com/lola-module-name: <pack> | ||
| agentic.redhat.com/repository: https://github.com/RHEcosystemAppEng/agentic-plugins | ||
| agentic.redhat.com/path: <pack> | ||
| spec: | ||
| targets: | ||
| - ./<pack>-plugin.yaml | ||
| - ./skills/<skill-name>/catalog-info.yaml | ||
| # List every skill manifest under skills/ |
42 changes: 42 additions & 0 deletions
42
.claude/skills/compass-manifest-maintenance/assets/plugin-catalog-info.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Template: <pack>/<pack>-plugin.yaml | ||
| # Replace <pack>, title, description, tags, disciplines, categories, dependencyOf, dependsOn (MCP union). | ||
| apiVersion: backstage.io/v1alpha1 | ||
| kind: AiResource | ||
| metadata: | ||
| name: <pack> | ||
| namespace: ai5-marketplace | ||
| title: <Pack Title> | ||
| description: > | ||
| <Pack description from README or plugin manifest.> | ||
| tags: | ||
| - ai-skill | ||
| - <pack-tag> | ||
| annotations: | ||
| agentic-plugins.redhat.com/version: "1.0" | ||
| agentic-plugins.redhat.com/repo: "agentic-plugins" | ||
| agentic.redhat.com/lola-module: "true" | ||
| agentic.redhat.com/lola-module-name: <pack> | ||
| agentic.redhat.com/repository: https://github.com/RHEcosystemAppEng/agentic-plugins | ||
| agentic.redhat.com/path: <pack> | ||
| github.com/project-slug: RHEcosystemAppEng/agentic-plugins | ||
| links: | ||
| - url: https://github.com/RHEcosystemAppEng/agentic-plugins | ||
| title: Source Repository | ||
| icon: github | ||
| spec: | ||
| type: plugin | ||
| lifecycle: development | ||
| owner: group:redhat/ai5-marketplace | ||
| system: default/agentic-plugins | ||
| disciplines: | ||
| - <discipline> | ||
| categories: | ||
| - <category> | ||
| agents: [] | ||
| dependsOn: | ||
| # Union of mcpserver refs used by any skill in this pack (omit if none) | ||
| # - mcpserver:ai5-marketplace/<owned-mcp> | ||
| # - mcpserver:redhat/<canonical-mcp> | ||
| dependencyOf: | ||
| - airesource:ai5-marketplace/<skill-name> | ||
| # Every skill in the pack — bidirectional with skill dependsOn plugin |
36 changes: 36 additions & 0 deletions
36
.claude/skills/compass-manifest-maintenance/assets/skill-catalog-info.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # Template: skills/<skill-name>/catalog-info.yaml | ||
| # Replace <pack>, <skill-name>, title, description, tags, disciplines, categories, dependsOn. | ||
| apiVersion: backstage.io/v1alpha1 | ||
| kind: AiResource | ||
| metadata: | ||
| name: <skill-name> | ||
| namespace: ai5-marketplace | ||
| title: <Skill Title> | ||
| description: > | ||
| <Multi-line description from SKILL.md frontmatter description, trimmed for catalog.> | ||
| labels: | ||
| distribution: external | ||
| annotations: | ||
| backstage.io/source-location: >- | ||
| url:https://github.com/RHEcosystemAppEng/agentic-plugins/blob/main/<pack>/skills/<skill-name>/SKILL.md | ||
| tags: | ||
| - ai-skill | ||
| - <domain-tag> | ||
| links: | ||
| - url: https://github.com/RHEcosystemAppEng/agentic-plugins | ||
| title: Source Repository | ||
| icon: github | ||
| spec: | ||
| type: skill | ||
| lifecycle: beta | ||
| owner: group:redhat/ai5-marketplace | ||
| disciplines: | ||
| - <discipline> | ||
| categories: | ||
| - <category> | ||
| agents: [] | ||
| dependsOn: | ||
| - airesource:ai5-marketplace/<pack> | ||
| # Add mcpserver: and airesource: (other skills) only when SKILL.md uses them | ||
| dependencyOf: [] | ||
| # dependencyOf: list orchestration skills that dependsOn this skill (COMPASS-1288) |
57 changes: 57 additions & 0 deletions
57
.claude/skills/compass-manifest-maintenance/references/mcp-mapping.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| # Deriving MCP `dependsOn` for skill manifests | ||
|
|
||
| Map **only** MCP servers the skill **actually uses**. Never copy a sibling manifest and never declare every key in `<pack>/mcps.json`. | ||
|
|
||
| ## Step 1 — Detect MCP usage in the skill | ||
|
|
||
| From `<pack>/skills/<skill>/SKILL.md`: | ||
|
|
||
| - `allowed-tools` in frontmatter (tool names / prefixes) | ||
| - `Required MCP Servers` / `Required MCP Tools` in the body | ||
| - Workflow steps that name an MCP server key or tool | ||
|
|
||
| If none of the above reference MCP tools → **no** `mcpserver:` in `dependsOn` (script-only, `WebFetch`, or public API skills). | ||
|
|
||
| ## Step 2 — Match usage to `mcps.json` keys | ||
|
|
||
| Read `<pack>/mcps.json`. Note which **server keys** the skill documents (e.g. `openshift-administration`, `lightspeed-mcp`, `aap-mcp-job-management`). | ||
|
|
||
| When `allowed-tools` lists tools but SKILL.md does not name a key, infer the server from context in the skill body (which MCP block documents those tools). | ||
|
|
||
| ## Step 3 — Resolve each key to a Compass entity ref | ||
|
|
||
| For each MCP the skill uses: | ||
|
|
||
| ### A. Owned MCP (manifest in this repo) | ||
|
|
||
| 1. List `mcps/*.yaml` (exclude `catalog-info.yaml`). | ||
| 2. Open manifests and read `metadata.name` and `metadata.namespace` (typically `ai5-marketplace`). | ||
| 3. Entity ref: `mcpserver:<namespace>/<metadata.name>`. | ||
| 4. **Deduplication:** Several `mcps.json` keys may share one Compass entity (same image/technology). Compare `mcps.json` entries (image URL, upstream repo in manifest `links`) to pick the correct owned manifest — do not create one ref per key if they map to the same `mcps/*.yaml`. | ||
| 5. Confirm the skill’s tools appear in that manifest’s `spec.primitives` (or match the server described in SKILL.md) when ambiguous. | ||
|
|
||
| ### B. Canonical MCP (registered upstream, not in `mcps/`) | ||
|
|
||
| If the pack references an MCP that **no** file under `mcps/` covers: | ||
|
|
||
| 1. Search existing skill manifests in the same pack (or repo) for the same `mcps.json` key or the same upstream product. | ||
| 2. Reuse the `mcpserver:` ref already used there (often `mcpserver:redhat/<name>`). | ||
| 3. Do **not** add a duplicate MCPServer manifest in `mcps/`. | ||
|
|
||
| When unsure, grep the repo for the `mcps.json` key or tool prefix in other `catalog-info.yaml` files under the same pack. | ||
|
|
||
| ## Step 4 — Update inverse relations | ||
|
|
||
| For each `mcpserver:` added to the skill’s `dependsOn`: | ||
|
|
||
| - Add `airesource:ai5-marketplace/<skill-name>` to that MCP manifest’s `spec.dependencyOf` (owned MCPs only — canonical MCPs are maintained by their owners). | ||
|
|
||
| ## Step 5 — Plugin MCP union | ||
|
|
||
| The pack plugin’s `dependsOn` MCP list = **union** of all `mcpserver:` refs across skill manifests in that pack — not the full set of keys in `mcps.json`. | ||
|
|
||
| ## Sanity checks | ||
|
|
||
| - Fewer MCP refs than keys in `mcps.json` is normal (per-skill usage). | ||
| - More MCP refs than the skill uses is wrong (blanket template deps). | ||
| - Canonical refs (`mcpserver:redhat/...`) do not get new files under `mcps/`. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.