Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
- **Phase 3 prompt-injection defense — extend `_wrap_user_input` across GenAI callers** (Issue #1007, follow-up to #960): Adopts the Phase 2 wrapping helper across 8 additional `GenAIAnalyzer` callers (10 sites total), giving the codebase uniform prompt-injection defense at every LLM integration point. New helper `_safe_wrap(text)` in `plugins/autonomous-dev/hooks/genai_utils.py` simplifies adoption (single-line invocation, never raises). Modified callers — Pattern A (kwarg substitution): `complexity_assessor.py` (`feature_description`), `scope_detector.py` (`issue_text`), `issue_scope_detector.py` (`issue_text`), `alignment_assessor.py` (2 sites: `dependencies_sample`/`config_files`/`readme_content`; enum-constrained scalars like `primary_language` left unwrapped), `genai_refactor_analyzer.py` (5 sites covering `doc_content`/`source_content`/`test_source`/`source_under_test`/`function_source`/`references_summary`/`original_analysis`), `security_scan.py` (`line`+`variable_name`; `secret_type` regex-catalog constant unwrapped), `auto_fix_docs.py` (`item_name`; `item_type` constrained literal unwrapped). Pattern B (.format pre-substitution): `feature_completion_detector.py` wraps both `feature` and `evidence` at `.format()` time since `analyzer.analyze()` receives the already-formatted prompt. Audit verified 4 false positives (`error_analyzer.py`, `codebase_analyzer.py`, `enforce_prunable_threshold.py`, `align_project_retrofit.py`) do NOT call `GenAIAnalyzer` and are NOT modified — locked by a parametrized regression test. 26 new test functions: 4 in `tests/unit/hooks/test_genai_utils_safe_wrap.py` (helper guarantees: wraps, escapes, coerces non-string, never raises), 21 in `tests/unit/lib/test_phase3_wrap_adoption.py` (8 wrap-adoption tests parametrized over multi-site files, plus 4 false-positive lock tests), and 1 existing-test adjustment in `tests/unit/lib/test_scope_detector_genai.py` (assertion updated from raw to wrapped form). `docs/INTENT-CLASSIFICATION.md` and `CHANGELOG.md` updated.

### Added
- **Intent classifier Phase 2 — classifier-gated plan-critic + research skip** (Issue #961): New library `plugins/autonomous-dev/lib/pipeline_intent_gates.py` exposes two pure-predicate decision functions — `should_skip_web_research()` and `should_skip_plan_critic()` — that the `/implement` coordinator consults to skip non-floor pipeline steps for low-risk intents. Activation is opt-in via `INTENT_CLASSIFIER_ENABLED=true`; default behavior (env var unset/false) is byte-identical to the pre-#961 pipeline. **Web research skip** (STEP 3.6, new): when intent ∈ {`config`, `doc`, `refactor`} with confidence ≥ 0.85, the `researcher` (Sonnet, web) agent is omitted from STEP 4; `researcher-local` (Haiku, codebase) always runs. **Plan-critic skip** (STEP 5.5a.1, new): when intent ∈ {`refactor`, `config`, `doc`, `typo`} with confidence ≥ 0.85 AND `predicted_file_count` ≤ 3, plan-critic invocation (5.5b) is bypassed; STEP 5.5c structural validation (file paths, acceptance criteria, testing strategy) ALWAYS still runs. Both gates honor the `--strict` token in ARGUMENTS as a forced full-pipeline override. Fail-closed on classifier import error, exception, or AMBIGUOUS verdict — no skip path can fire without a high-confidence non-security classification. `pipeline_completion_state.py` extended with `record_web_research_skipped()` / `get_web_research_skipped()` and a backward-compatible `reason` kwarg on `record_plan_critic_skipped()`; `get_plan_critic_skipped()` reads both legacy bool and new dict (`{"skipped": True, "reason": ...}`) shapes. The classifier itself (Phase 1, #971) is unchanged. 41 new test functions in `tests/unit/test_implement_command_routing.py` (markdown structure, env-var truthiness, web-research and plan-critic gate logic across all skip/no-skip permutations, state-persistence round-trip, legacy bool back-compat). `tests/unit/test_implement_command_structure.py` line-cap bumped 575 → 1650 to accommodate the new STEP 3.6 / STEP 5.5a.1 sections.

- **Intent classifier Phase E — enforcement cutover** (Issue #999): Completes the intent classifier track started in Phase 1 (#971). When `INTENT_CLASSIFIER_ENFORCE=true` (default `false`), three pipeline-gating hooks consult the per-session artifact written by Phase D and skip non-floor checks for low-risk intent classes (`doc`, `config`, `typo`, `status_query`, `conversation`). Hard-floor hooks (Phase C registry) always fire regardless of intent class. New libraries: `lib/hook_stdin.py` — one-shot cached stdin reader providing `read_stdin_once()` and `extract_session_id()` so Phase E hooks can read the PreToolUse payload without exhausting the stream across multiple call sites; `lib/enforcement_decision.py` — pure policy layer with 8-rule priority chain (`should_skip_enforcement()`) — zero filesystem side effects, never raises, fail-safe direction is always enforce. Modified hooks: `unified_pre_tool.py` (5 wrap sites via `_phase_e_skip()` helper), `plan_gate.py`, `plan_mode_exit_detector.py`. New telemetry decision shape `"mode_skip"` added to `hook_telemetry.py` `VALID_DECISION_SHAPES` — emitted on skip path to `.claude/logs/hook-blocks.jsonl` for observability. Reader-side API added to `session_mode.py`: `read_session_mode(session_id)` and `should_pipeline_enforce(intent_class)`. Single env var rollback: unset `INTENT_CLASSIFIER_ENFORCE` or set to any non-"true" value to revert all Phase E gating. `docs/INTENT-CLASSIFICATION.md` updated: Phase E roadmap row, Phase E section, `INTENT_CLASSIFIER_ENFORCE` behavior, related files table. `docs/LIBRARIES.md` gains entries for `hook_stdin.py` and `enforcement_decision.py`; `session_mode.py` and `hook_telemetry.py` entries updated. `docs/HOOKS.md` and `docs/HOOK-REGISTRY.md` updated with Phase E notes. 48 new test functions: 13 in `tests/unit/lib/test_enforcement_decision.py`, 10 in `tests/unit/lib/test_hook_stdin.py`, 12 in `tests/unit/lib/test_session_mode_reader.py`, 10 in `tests/unit/hooks/test_phase_e_integration.py`, 3 in `tests/integration/test_enforcement_mode_cutover.py`.

- **Intent classifier Phase 2 — prompt-injection defense** (Issue #960): Hardens `lib/intent_classifier.py` against prompt-injection attacks on the LLM classification path (OWASP LLM01:2025). User prompts are now wrapped in `<user_input>…</user_input>` XML delimiters via a new `_wrap_user_input(text)` helper in `hooks/genai_utils.py` before being substituted into `_LLM_PROMPT_TEMPLATE`; `html.escape(text, quote=False)` encodes `&`, `<`, and `>` so an attacker cannot inject structural tokens that escape the delimiter. Apostrophes and double-quotes are preserved (`quote=False`) because they are common in legitimate prompts. A module-load guard (`_validate_template_integrity`) raises `RuntimeError` (not `assert` — survives `python -O`) at import time if the template no longer contains `<user_input>`; refusing to load is safer than silently degrading to a vulnerable state. `_wrap_user_input` lives in `genai_utils.py` for cross-codebase reuse — other `GenAIAnalyzer` callers can adopt the same defense via follow-up issues. The security regex pre-gate and fail-open contract are unchanged; this fix only affects the LLM path for non-security-keyword prompts. 8 new tests in `tests/unit/lib/test_intent_classifier.py` (`TestPromptInjectionResistance`): injection-escape verification, RuntimeError guard, `_wrap_user_input` round-trip, LLM path uses wrapped form, and fallback behavior when `_wrap_user_input` is unavailable. `docs/INTENT-CLASSIFICATION.md` updated: Phase 2 roadmap row, new Architecture section 5 (prompt-injection defense), test count corrected. `docs/LIBRARIES.md` `intent_classifier.py` entry updated with Phase 2 description and corrected test count. `docs/HOOKS.md` `genai_utils.py` entry updated to document `_wrap_user_input`. Prerequisite before `INTENT_CLASSIFIER_ENFORCE=true` rollout.
Expand Down
75 changes: 71 additions & 4 deletions plugins/autonomous-dev/commands/implement.md
Original file line number Diff line number Diff line change
Expand Up @@ -502,15 +502,48 @@ Otherwise: proceed to STEP 4.
- ❌ The change touches security-sensitive files or topics (authentication, encryption, tokens, secrets, sso, oauth, rbac, permission, session, jwt)
- ❌ More than 3 files are referenced in the description

### STEP 3.6: Web Research Classifier Gate (Phase 2 — opt-in)

**Progress**: Output step banner only when the gate fires (STEP 3.6/15 — Web Research Classifier Gate). Skip silently when `INTENT_CLASSIFIER_ENABLED` is unset/false.

**Activation**: Only runs when `INTENT_CLASSIFIER_ENABLED=true` is set in the environment. Default behavior (env var unset/false) is unchanged — both researchers run as before.

**Decision logic** (delegated to pure predicate):

```python
from pipeline_intent_gates import should_skip_web_research
from pipeline_completion_state import record_web_research_skipped

skip, reason = should_skip_web_research(
feature_description=FEATURE_DESCRIPTION,
arguments=ARGUMENTS,
)
if skip:
record_web_research_skipped(SESSION_ID, issue_number=ISSUE_NUM, reason=reason)
print(f"Web research: SKIPPED ({reason}) — researcher-local still runs")
```

**When skipped**:
- STEP 4 dispatches **only** `researcher-local` (Haiku) — the `researcher` (Sonnet, web) agent is omitted.
- The merged research consists of researcher-local output alone.

**When NOT skipped** (any of these): classifier disabled, `--strict` in ARGUMENTS, intent not in {CONFIG, DOC, REFACTOR}, confidence < 0.85, classifier failed open or returned AMBIGUOUS.

**FORBIDDEN** — You MUST NOT:
- Skip BOTH researchers (researcher-local always runs)
- Skip when intent is SECURITY_CRITICAL, IMPLEMENT, TEST, or any other class outside the explicit allow-list
- Skip when `--strict` appears in ARGUMENTS

### STEP 4: Parallel Research (2 agents)

**Progress**: Output step banner (STEP 4/15 — Research, Agents: researcher-local (Haiku), researcher (Sonnet)). Output agent completions after each returns.

**Pre-dispatch**: Follow the Pre-Dispatch Ordering Protocol (above) for each agent before invoking.

Invoke TWO agents in PARALLEL (single message, both Agent tool calls):
1. **Agent**(subagent_type="researcher-local", model="haiku") — "Search codebase for patterns related to: {feature}. Output JSON with findings and sources."
2. **Agent**(subagent_type="researcher", model="sonnet") — "Research best practices for: {feature}. MUST use WebSearch. Output JSON with findings, sources, security considerations."
Invoke researchers in PARALLEL (single message). The set depends on STEP 3.6:

- **Always**: **Agent**(subagent_type="researcher-local", model="haiku") — "Search codebase for patterns related to: {feature}. Output JSON with findings and sources."
- **Unless STEP 3.6 set `web_research_skipped`**: **Agent**(subagent_type="researcher", model="sonnet") — "Research best practices for: {feature}. MUST use WebSearch. Output JSON with findings, sources, security considerations."

Validation: If web researcher shows 0 tool uses, retry. Merge both outputs. Persist research via `save_merged_research()`.

Expand Down Expand Up @@ -556,6 +589,40 @@ record_plan_critic_skipped(SESSION_ID, issue_number=ISSUE_NUM)

If no matching file with "Verdict: PROCEED" is found, proceed to 5.5b.

#### 5.5a.1 — Classifier-Gated Skip (Phase 2 — opt-in)

**Activation**: Only runs when `INTENT_CLASSIFIER_ENABLED=true`. Default (env var unset/false) is unchanged — proceed straight to 5.5b.

**Decision logic** (delegated to pure predicate):

```python
from pipeline_intent_gates import should_skip_plan_critic
from pipeline_completion_state import record_plan_critic_skipped

skip, reason = should_skip_plan_critic(
feature_description=FEATURE_DESCRIPTION,
arguments=ARGUMENTS,
)
if skip:
record_plan_critic_skipped(SESSION_ID, issue_number=ISSUE_NUM, reason=reason)
print(f"Plan validation: SKIPPED ({reason}) — proceeding to 5.5c structural validation")
# Skip 5.5b (plan-critic invocation). 5.5c structural validation STILL RUNS.
```

**Skip conditions** (ALL must hold): classifier enabled, no `--strict`, intent in {REFACTOR, CONFIG, DOC, TYPO}, confidence >= 0.85, predicted_file_count <= 3, classifier did not fail open.

**When skipped**:
- 5.5b (plan-critic agent invocation) is bypassed.
- 5.5c (structural validation — file paths, acceptance criteria, testing strategy) **always still runs**.

**When NOT skipped**: proceed to 5.5b as before.

**FORBIDDEN** — You MUST NOT:
- Skip 5.5c structural validation under any circumstance (the existing 5.5d FORBIDDEN list still applies in full)
- Skip when `--strict` is in ARGUMENTS
- Skip when intent is IMPLEMENT, SECURITY_CRITICAL, TEST, or any class outside {REFACTOR, CONFIG, DOC, TYPO}
- Skip when predicted_file_count > 3 (the issue may look small but the classifier predicts otherwise)

#### 5.5b — Budget Plan-Critic Invocation

**When no pre-validated plan exists**, invoke the plan-critic agent with a constrained budget:
Expand Down Expand Up @@ -599,7 +666,7 @@ If any requirement is missing:

- ❌ You MUST NOT accept a plan that contains 0 file paths (structural validation always blocks this)
- ❌ You MUST NOT accept a plan that has no acceptance criteria section
- ❌ You MUST NOT skip plan-critic when no pre-validated plan file exists in `.claude/plans/`
- ❌ You MUST NOT skip plan-critic when no pre-validated plan file exists in `.claude/plans/` AND the 5.5a.1 classifier gate did NOT fire (classifier disabled, low-confidence, --strict, or intent outside allow-list)
- ❌ You MUST NOT skip structural validation for any reason (it always runs, even with a pre-validated plan)

### STEP 6: Generate Acceptance Tests (default mode only)
Expand Down
1 change: 1 addition & 0 deletions plugins/autonomous-dev/config/install_manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@
"plugins/autonomous-dev/lib/permission_classifier.py",
"plugins/autonomous-dev/lib/pipeline_completion_state.py",
"plugins/autonomous-dev/lib/pipeline_efficiency_analyzer.py",
"plugins/autonomous-dev/lib/pipeline_intent_gates.py",
"plugins/autonomous-dev/lib/pipeline_intent_validator.py",
"plugins/autonomous-dev/lib/pipeline_state.py",
"plugins/autonomous-dev/lib/pipeline_timing_analyzer.py",
Expand Down
70 changes: 64 additions & 6 deletions plugins/autonomous-dev/lib/pipeline_completion_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -779,22 +779,24 @@ def record_plan_critic_skipped(
session_id: str,
*,
issue_number: int = 0,
reason: str = "pre_validated",
) -> None:
"""Record that plan-critic was skipped for a given session/issue.

Called by the coordinator at STEP 5.5a when a pre-validated plan
is found in `.claude/plans/`, bypassing plan-critic invocation.
Called by the coordinator at STEP 5.5a when a pre-validated plan is
found, OR at STEP 5.5a.1 when the Phase 2 classifier gate fires.

Args:
session_id: The pipeline session identifier.
issue_number: The issue number (0 for non-batch).
reason: Audit string. "pre_validated" (5.5a default) or "classifier" (Phase 2).

Issues: #878
Issues: #878, #961
"""
state = _ensure_state(session_id)
plan_critic_skipped = state.setdefault("plan_critic_skipped", {})
issue_key = str(issue_number)
plan_critic_skipped[issue_key] = True
plan_critic_skipped[issue_key] = {"skipped": True, "reason": reason}
_write_state(session_id, state)


Expand All @@ -812,14 +814,70 @@ def get_plan_critic_skipped(
Returns:
True if plan-critic was recorded as skipped, False otherwise.

Issues: #878
Issues: #878, #961
"""
state = _read_state(session_id)
if not state:
return False
plan_critic_skipped = state.get("plan_critic_skipped", {})
issue_key = str(issue_number)
return bool(plan_critic_skipped.get(issue_key, False))
entry = plan_critic_skipped.get(issue_key, False)
if isinstance(entry, dict):
return bool(entry.get("skipped", False))
return bool(entry)


def record_web_research_skipped(
session_id: str,
*,
issue_number: int = 0,
reason: str = "classifier",
) -> None:
"""Record that web research (researcher agent) was skipped at STEP 4.

Distinct from `record_research_skipped` (STEP 3.5 fully-specified path) so
Phase 5 telemetry can attribute the skip to the classifier vs the
pre-existing fully-specified gate.

Args:
session_id: The pipeline session identifier.
issue_number: The issue number (0 for non-batch).
reason: Audit string from pipeline_intent_gates.should_skip_web_research.

Issues: #961
"""
state = _ensure_state(session_id)
web_research_skipped = state.setdefault("web_research_skipped", {})
issue_key = str(issue_number)
web_research_skipped[issue_key] = {"skipped": True, "reason": reason}
_write_state(session_id, state)


def get_web_research_skipped(
session_id: str,
*,
issue_number: int = 0,
) -> bool:
"""Check if web research was skipped (Phase 2 classifier gate).

Args:
session_id: The pipeline session identifier.
issue_number: The issue number (0 for non-batch).

Returns:
True if recorded as skipped, False otherwise.

Issues: #961
"""
state = _read_state(session_id)
if not state:
return False
web_research_skipped = state.get("web_research_skipped", {})
issue_key = str(issue_number)
entry = web_research_skipped.get(issue_key, False)
if isinstance(entry, dict):
return bool(entry.get("skipped", False))
return bool(entry)


def verify_pipeline_agent_completions(
Expand Down
Loading
Loading