Skip to content

feat: dynamic skill loading tools#345

Open
RMartires wants to merge 5 commits intousestrix:mainfrom
RMartires:feat/dynamic-skill-load
Open

feat: dynamic skill loading tools#345
RMartires wants to merge 5 commits intousestrix:mainfrom
RMartires:feat/dynamic-skill-load

Conversation

@RMartires
Copy link
Contributor

Issue Link: #280

added load_skills and list_skills tools + added context in the system prompt so strix can dynamically choose to load a new skill when required

this will allow for the agent to add in exact context for a vulnerability type right before acting

below check out strix calling the new tools in action

e.g.
image
image

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 5, 2026

Greptile Summary

This PR introduces two new agent tools — list_skills and load_skills — that enable the Strix agent to dynamically discover and load specialized knowledge packages at runtime. The implementation integrates cleanly with the existing tool registry and skills infrastructure, with comprehensive examples in the updated system prompt.

However, three issues require attention:

  1. Category/path format validation bug: The XML schema documents that load_skills accepts category-prefixed paths like "vulnerabilities/sql_injection", but validate_skill_names only recognizes bare skill stems. Path-formatted inputs are silently rejected before the actual load_skills_content function (which can resolve them) ever receives them.

  2. Duplicate "BLACK-BOX PHASE 1" headings: The new skill-loading instruction was inserted between the original "BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING):" heading and its bullet points, then a second "BLACK-BOX - PHASE 1:" heading was added before those bullets. This creates two differently-named Phase 1 sections that may confuse the model.

  3. Missing error key on validation failure: When all requested skills fail validation, the response includes "success": False but lacks an "error" key, unlike every other failure path in the module. Downstream code expecting a consistent response shape may raise KeyError.

Confidence Score: 2/5

  • The core functionality works for bare skill names, but the documented category/path format is silently broken, and response inconsistency could break error handling.
  • Three verified issues reduce confidence: (1) the path-format feature documented in the schema is non-functional due to validation rejecting it; (2) inconsistent response shape with missing error key breaks downstream error handling; (3) confusing prompt structure with duplicate headings. None are blocking for basic bare-name loading, but they represent bugs in the documented feature set and error handling patterns that should be fixed before this is considered production-ready.
  • strix/tools/skills/skills_actions.py — validation and response shape issues; strix/agents/StrixAgent/system_prompt.jinja — duplicate heading structure.

Last reviewed commit: a157086

Comment on lines +150 to +152
validation = validate_skill_names(skill_list)
invalid_skills = validation.get("invalid", [])
valid_skills = validation.get("valid", [])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documented input format for load_skills allows category-prefixed paths (e.g., "vulnerabilities/sql_injection"), as stated in the schema on line 84:

You can specify skills by name only (e.g., "sql_injection") or with category path (e.g., "vulnerabilities/sql_injection")

However, validate_skill_names (in strix/skills/__init__.py line 43) only validates against bare skill stems returned by get_all_skill_names(). Any path-formatted input is silently rejected and placed in invalid_skills before the actual load_skills_content function (which can resolve paths) ever sees it.

To fix: Normalize path-format names before validation, or update validate_skill_names to accept both forms:

def _bare_name(s: str) -> str:
    return s.split("/")[-1]

validation = validate_skill_names([_bare_name(s) for s in skill_list])
invalid_bare = set(validation.get("invalid", []))
valid_skills = [s for s in skill_list if _bare_name(s) not in invalid_bare]
invalid_skills = [s for s in skill_list if _bare_name(s) in invalid_bare]
Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/tools/skills/skills_actions.py
Line: 150-152

Comment:
The documented input format for `load_skills` allows category-prefixed paths (e.g., `"vulnerabilities/sql_injection"`), as stated in the schema on line 84:

> You can specify skills by name only (e.g., "sql_injection") or with category path (e.g., "vulnerabilities/sql_injection")

However, `validate_skill_names` (in `strix/skills/__init__.py` line 43) only validates against bare skill stems returned by `get_all_skill_names()`. Any path-formatted input is silently rejected and placed in `invalid_skills` before the actual `load_skills_content` function (which can resolve paths) ever sees it.

**To fix:** Normalize path-format names before validation, or update `validate_skill_names` to accept both forms:

```python
def _bare_name(s: str) -> str:
    return s.split("/")[-1]

validation = validate_skill_names([_bare_name(s) for s in skill_list])
invalid_bare = set(validation.get("invalid", []))
valid_skills = [s for s in skill_list if _bare_name(s) not in invalid_bare]
invalid_skills = [s for s in skill_list if _bare_name(s) in invalid_bare]
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +192 to +196
BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING):

When spawning testing agents: call list_skills first to get valid skill names, then create_agent with skills param matching recon findings.

BLACK-BOX - PHASE 1:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The insertion of the skill-loading instruction splits the "BLACK-BOX TESTING - PHASE 1" heading into two separate headings with different names:

  • BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING): (line 192)
  • BLACK-BOX - PHASE 1: (line 196)

This creates a confusing structure that may cause the model to treat them as distinct phases. Merge the new instruction into the existing section:

BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING):
When spawning testing agents: call list_skills first to get valid skill names, then create_agent with skills param matching recon findings.
- COMPLETE full reconnaissance: subdomain enumeration, port scanning, service detection
- MAP entire attack surface: ...
Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/agents/StrixAgent/system_prompt.jinja
Line: 192-196

Comment:
The insertion of the skill-loading instruction splits the "BLACK-BOX TESTING - PHASE 1" heading into two separate headings with different names:
- `BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING):` (line 192)
- `BLACK-BOX - PHASE 1:` (line 196)

This creates a confusing structure that may cause the model to treat them as distinct phases. Merge the new instruction into the existing section:

```
BLACK-BOX TESTING - PHASE 1 (RECON & MAPPING):
When spawning testing agents: call list_skills first to get valid skill names, then create_agent with skills param matching recon findings.
- COMPLETE full reconnaissance: subdomain enumeration, port scanning, service detection
- MAP entire attack surface: ...
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +170 to +176
return {
"success": len(loaded_content) > 0,
"loaded_skills": loaded_content,
"loaded_count": len(loaded_content),
"invalid_skills": invalid_skills,
"warnings": warnings,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When all requested skills are invalid (i.e., all fail validate_skill_names), this function returns {"success": False, ...} without an "error" key. This breaks the response consistency pattern — every other failure path in this file and in list_skills includes an "error" key.

Downstream code that checks result["error"] will raise KeyError. While a warning is added to the warnings list, callers expect a dedicated error key for failure responses.

Fix: Add an error key when success is False and no skills loaded:

result: dict[str, Any] = {
    "success": len(loaded_content) > 0,
    "loaded_skills": loaded_content,
    "loaded_count": len(loaded_content),
    "invalid_skills": invalid_skills,
    "warnings": warnings,
}
if not result["success"] and not loaded_content:
    result["error"] = (
        "No skills could be loaded. "
        + (warnings[0] if warnings else "Check skill names with list_skills.")
    )
return result
Prompt To Fix With AI
This is a comment left during a code review.
Path: strix/tools/skills/skills_actions.py
Line: 170-176

Comment:
When all requested skills are invalid (i.e., all fail `validate_skill_names`), this function returns `{"success": False, ...}` without an `"error"` key. This breaks the response consistency pattern — every other failure path in this file and in `list_skills` includes an `"error"` key.

Downstream code that checks `result["error"]` will raise `KeyError`. While a warning is added to the `warnings` list, callers expect a dedicated error key for failure responses.

**Fix:** Add an error key when success is False and no skills loaded:

```python
result: dict[str, Any] = {
    "success": len(loaded_content) > 0,
    "loaded_skills": loaded_content,
    "loaded_count": len(loaded_content),
    "invalid_skills": invalid_skills,
    "warnings": warnings,
}
if not result["success"] and not loaded_content:
    result["error"] = (
        "No skills could be loaded. "
        + (warnings[0] if warnings else "Check skill names with list_skills.")
    )
return result
```

How can I resolve this? If you propose a fix, please make it concise.

- load skill validation
- added error response for load skills when no valid skills are passed
@RMartires
Copy link
Contributor Author

@0xallam have pushed an enchanement for #280 (load_skills tool)
do check it when possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant