Skip to content

Conversation

@sti0
Copy link
Contributor

@sti0 sti0 commented Jan 4, 2026

Enforce AgentFactory Usage and Improve Trait Inference

Summary

This PR implements mandatory AgentFactory execution for custom agents and significantly improves trait inference accuracy. The changes ensure that custom agents are properly composed with correct voice mappings and that users receive immediate feedback on invalid trait selections.

Problem Statement

Before this PR:

  1. ❌ AI could bypass AgentFactory by manually composing prompts from Traits.yaml
  2. ❌ No validation or guidance when invalid traits were specified
  3. ❌ Trait inference produced false matches (e.g., "POSIX compliance" → Legal Analyst)
  4. ❌ Explicit --traits were merged with inferred traits, defeating precision
  5. ❌ No clear guidance on when to use explicit vs. inferred traits

Result: Inconsistent agent composition, wrong voice assignments, poor user experience.

Solution Overview

This PR addresses all issues through four complementary improvements:

1. Constitutional Enforcement (SKILL.md)

Added mandatory AgentFactory execution rules at the top of the skill:

## MANDATORY: AgentFactory Execution (Constitutional Rule)

**BEFORE launching ANY custom agent, you MUST execute AgentFactory.ts via Bash.**

### Validation Checklist
- [ ] I executed AgentFactory.ts via Bash for EACH agent
- [ ] I captured the JSON output (prompt + voice_id)
- [ ] I am using the factory's prompt field verbatim
- [ ] I am using subagent_type: "general-purpose"
- [ ] Each agent has DIFFERENT trait combinations

Impact: AI cannot claim to create "custom agents" without executing AgentFactory.

2. Error Recovery Protocol (SKILL.md)

Added step-by-step recovery guidance for invalid trait errors:

### Error Recovery Protocol

If AgentFactory returns "Unknown traits" error:
1. DO NOT guess again
2. Read the error output (now shows all available traits)
3. Select valid traits from EXPERTISE, PERSONALITY, APPROACH
4. Retry with correct traits

Impact: Users immediately see valid options instead of guessing blindly.

3. Enhanced Error Messages (AgentFactory.ts v1.1.0)

Before:

Error: Unknown traits: typescript, bash

After:

Error: Unknown traits: typescript, bash

Available traits:
  EXPERTISE:   security, legal, finance, medical, technical, research, creative, business, data, communications
  PERSONALITY: skeptical, enthusiastic, cautious, bold, analytical, creative, empathetic, contrarian, pragmatic, meticulous
  APPROACH:    thorough, rapid, systematic, exploratory, comparative, synthesizing, adversarial, consultative

Run with --list to see full trait descriptions

Impact: Users get immediate actionable feedback without a second command.

4. Expanded Technical Keywords (Traits.yaml)

Added 20+ programming-related keywords to technical expertise:

technical:
  keywords:
    # Original:
    - code, architecture, system, implementation, debug, technical
    - API, database, infrastructure, software

    # NEW:
    - script, bash, shell, POSIX
    - TypeScript, JavaScript, Python, Java, Rust, Go
    - error handling, development, transform, conversion
    - CLI, function, class, module

Impact: Technical tasks now correctly infer technical instead of legal or research.

5. Explicit Trait Recommendations (CreateCustomAgent.md)

Added Step 1.5 with clear comparison:

Approach Precision Example
Explicit High - you control exactly which traits --traits "technical,meticulous,systematic"
Inference ⚠️ Low - keywords may match wrong expertise --task "..." (might infer wrong traits)

When to use explicit traits:

  • Technical/programming tasks (avoid false matches with legal/medical)
  • When you need specific personality/approach combinations
  • When voice diversity matters

Impact: Users understand when to prioritize explicit trait selection.

6. Fixed Trait Merging Bug (AgentFactory.ts)

Before (v1.0.0):

if (values.traits) {
  traitKeys = values.traits.split(",").map(t => t.trim().toLowerCase());
}
if (values.task) {
  const inferred = inferTraitsFromTask(values.task, traits);
  traitKeys = [...new Set([...traitKeys, ...inferred])]; // MERGES!
}

Result: --traits "technical,pragmatic,rapid" --task "POSIX compliance"[technical, pragmatic, rapid, legal, analytical, thorough]

After (v1.1.0):

if (values.traits) {
  traitKeys = values.traits.split(",").map(t => t.trim().toLowerCase());
} else if (values.task) {  // <-- Changed to else if
  const inferred = inferTraitsFromTask(values.task, traits);
  traitKeys = [...new Set(inferred)];
}

Result: --traits "technical,pragmatic,rapid" --task "POSIX compliance"[technical, pragmatic, rapid]

Impact: Explicit traits now fully override inference as intended.

Changes Summary

File Changes Lines
SKILL.md Added constitutional rule, validation checklist, error recovery protocol +70
CreateCustomAgent.md Added Step 1.5 explicit trait guidance with comparison table +67
AgentFactory.ts Improved error messages, fixed trait merging, bumped to v1.1.0 +21
Traits.yaml Expanded technical keywords from 10 to 30+ +22

Total: +180 lines, 4 files modified

Testing

Test 1: Invalid Traits Error Message

Command:

bun run AgentFactory.ts --traits "typescript,bash" --task "Test"

Before (v1.0.0):

Error: Unknown traits: typescript, bash

After (v1.1.0):

Error: Unknown traits: typescript, bash

Available traits:
  EXPERTISE:   security, legal, finance, medical, technical, research, ...
  PERSONALITY: skeptical, enthusiastic, cautious, bold, analytical, ...
  APPROACH:    thorough, rapid, systematic, exploratory, comparative, ...

Run with --list to see full trait descriptions

Pass: Users now see all valid options immediately.

Test 2: Trait Inference Accuracy

Command:

bun run AgentFactory.ts --task "TypeScript code development" --output summary

Before (v1.0.0):

COMPOSED AGENT: Research Specialist Analytical Thorough
Traits: research, analytical, thorough  ❌ Wrong!

After (v1.1.0):

COMPOSED AGENT: Technical Specialist Analytical Thorough
Traits: technical, analytical, thorough  ✅ Correct!

Pass: Technical keywords now match technical expertise.

Test 3: Explicit Trait Override

Command:

bun run AgentFactory.ts --traits "technical,pragmatic,rapid" --task "POSIX compliance" --output summary

Before (v1.0.0):

Traits: technical, pragmatic, rapid, legal, analytical, thorough  ❌ Merged!

After (v1.1.0):

Traits: technical, pragmatic, rapid  ✅ Override works!

Pass: Explicit traits fully override inference.

Breaking Changes

None. All changes are additive or improvements to existing functionality.

Migration Guide

No migration needed. Users will benefit from:

  • Better error messages automatically
  • More accurate trait inference automatically
  • Explicit trait recommendations in documentation

Documentation Updates

  • ✅ SKILL.md: Added constitutional rule and error recovery protocol
  • ✅ CreateCustomAgent.md: Added explicit trait recommendation guidance
  • ✅ AgentFactory.ts: Updated version to 1.1.0 with changelog

Checklist

  • Code changes implemented
  • Documentation updated
  • Version bumped (AgentFactory 1.0.0 → 1.1.0)
  • Manual testing completed
  • All changes work in both pack source and installed location
  • Commit message follows conventional commits format
  • Co-authored by Claude as per project standards

Related Issues

Closes: #312

Notes

This PR significantly improves the custom agent creation workflow by:

  1. Preventing bypasses of AgentFactory (constitutional enforcement)
  2. Providing immediate, actionable feedback on errors
  3. Improving trait inference accuracy for technical tasks
  4. Fixing a subtle but important trait merging bug
  5. Guiding users toward explicit trait selection for precision

The changes maintain backward compatibility while dramatically improving the user experience and correctness of agent composition.


🤖 Generated with Claude Code

This commit implements mandatory AgentFactory execution for custom agents
and improves trait inference accuracy.

Changes:
- Add constitutional rule requiring AgentFactory.ts execution for all custom agents
- Add validation checklist and error recovery protocol to SKILL.md
- Expand technical expertise keywords (bash, TypeScript, Python, POSIX, etc.)
- Improve error messages to show available traits on invalid input
- Fix trait merging bug: explicit --traits now override inference instead of merging
- Add explicit trait recommendation guidance in CreateCustomAgent.md
- Bump AgentFactory version to 1.1.0

Impact:
- Prevents manual prompt composition bypassing voice mapping
- Reduces false trait matches (e.g., "POSIX compliance" → legal analyst)
- Gives users immediate feedback on invalid trait names
- Encourages explicit trait selection for precision

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
@danielmiessler danielmiessler merged commit 5408c76 into danielmiessler:main Jan 4, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DEBUG: Agent Factory violation

2 participants