feat: add intelligent collaboration examples and evals

[smota]

Epic: #115

Background & Problem Statement

Harness leverage and collective intelligence will only reduce cognitive load if humans and agents can see clear examples and if evals prevent behavioral drift. The current eval scaffold exists, but collaboration modes, role-agent packages, evidence schemas, and runtime patterns need examples and validation fixtures.

Proposed Solution

Add human-facing docs, agent-facing examples, and eval suites that prove the intelligent collaboration model remains simple, safe, and useful.

Requirements

• Add examples for each collaboration mode.
• Add human-readable docs explaining what users see vs what harnesses may do behind the scenes.
• Add agent-facing examples for role-agent packages and collaboration evidence.
• Add eval fixtures for mode selection, role-agent outputs, bounded loops, sub-agent evidence, and PR readiness auditing.
• Add benchmark/report format for cognitive-load reduction checks.
• Document full implementation/test/validation/doc loop.

Technical Design

Suggested examples:

• single-agent low-risk docs change;
• advisory review for standard feature;
• architect council for ambiguous design;
• parallel discovery for broad repo scan;
• tester/developer bounded loop for validation failures;
• isolated worktree spike for risky approach;
• PR readiness evidence auditor.

Suggested eval categories:

• mode selection from issue metadata;
• smallest-sufficient-collaboration enforcement;
• role-agent contract completeness;
• bounded-loop stop condition;
• sub-agent evidence completeness;
• no false multi-agent/council claim;
• human-gated high-assurance behavior.

Feature Tracking

• Add human guide for intelligent collaboration.
• Add agent guide for using collaboration modes.
• Add examples under docs/examples/.
• Add eval fixtures under agents/evals/datasets/.
• Add eval suite specs under agents/evals/suites/.
• Add benchmark/report template.
• Add docs validation/link checks where feasible.

Acceptance criteria

• A human can explain AgentFlow intelligent collaboration in under 60 seconds.
• An agent can choose and evidence a collaboration mode from examples.
• Evals cover all collaboration modes and false-claim guardrails.
• Docs show implementation, testing, validation, and documentation loops.
• Examples preserve the principle: extra AI intelligence, low cognitive load.

Test plan

• pnpm test.
• pnpm test:workflow.
• pnpm format:check.
• Run new eval command if implemented; otherwise validate fixtures structurally.
• Link/path spot checks for examples.

Workflow classification

• Profile: standard
• Risk: low
• Effort: high
• Change surfaces: docs, examples, evals
• Review: self-review acceptable; human editorial review useful.

Skill-creator eval, benchmark, and learning refinement

Examples/evals must follow the skill-creator lifecycle: create, eval, improve, benchmark, learn, and optional local-first telemetry.

Eval case format

Each eval case should include:

• scenario;
• prompt;
• expected behavior;
• must include;
• must not include;
• pass criteria.

Additional requirements

• Add eval fixtures for unnecessary sub-agent use and unnecessary tool use.
• Add eval fixtures for context efficiency / compact durable evidence.
• Add eval fixtures for progressive disclosure: SKILL.md short, details in references/templates/checklists/examples.
• Add feedback.jsonl template for local learning artifacts.
• Add improvement-plan template.
• Add benchmark-report template.
• Document that feedback/telemetry is local-first and opt-in, with no raw prompts, secrets, customer data, or full transcripts by default.

Benchmark dimensions

• task completion;
• instruction following;
• output quality;
• evidence quality;
• guardrail/boundary behavior;
• unnecessary tool/sub-agent use;
• context footprint;
• regressions.

Acceptance criteria additions

• Evals prove collaboration modes improve decision quality without increasing visible cognitive load.
• Benchmarks compare single-agent, advisory, and council for quality, evidence, and context footprint.
• Improvement workflow requires reviewing failed evals or feedback.jsonl before changing workflow-skill behavior.

[github-actions]

Integrated into development by PR #116: #116

Merge commit: 5d08ddbe7e3cefe8fe59d9421a86cea781514495
Release/promotion to main is tracked separately.

[github-actions]

Integrated into development by PR #117: #117

Merge commit: 0a19c0a24cb234e21f9f48dd7034f0a4a7ad55bd
Release/promotion to main is tracked separately.

[github-actions]

Integrated into development by PR #118: #118

Merge commit: 6cd23f4289fff1ffd01dfca4605711030dd333f0
Release/promotion to main is tracked separately.