feat: Add AI-powered exploration workflow with map-reduce pattern

[plusplusoneplusplus]

Summary

This PR implements an intelligent AI-powered exploration workflow system with map-reduce pattern, enabling automated codebase exploration through task decomposition, parallel AI investigation, and result aggregation.

Key Features

🤖 AI-Powered Task Decomposition

• AI Split Operation: Uses Claude (via CLIExecutor) to intelligently analyze high-level goals and generate focused exploration sub-tasks
• Configurable task count limits with min_tasks and max_tasks parameters
• Fallback mechanism for graceful degradation when AI fails
• Generates structured JSON with reasoning and task definitions

🔍 Parallel AI Exploration

• Exploration Operation: Integrated with ExploreAgent for actual AI investigation
• Supports 5 exploration types: question, implementation, structure, usage, flow
• Stores findings in session files (JSON format) for persistence and incremental processing
• Proper error handling with failed status reporting

📊 Result Aggregation

• Summarize Operation: Aggregates findings from session files or direct input
• Three summary formats:
  • detailed: Complete findings with all metadata
  • concise: Truncated results for quick overview
  • structured: Grouped by exploration type
• Supports output to JSON files for reporting

🔧 Deterministic Alternative

• Split Operation: Traditional algorithmic splitting (no AI)
• Four strategies: by_items, by_count, by_chunk_size, custom
• Useful for predetermined task decomposition

Implementation Details

New Operations

• plugins/automation/workflows/steps/operations/ai_split.py - AI task decomposition
• plugins/automation/workflows/steps/operations/exploration.py - AI exploration with ExploreAgent
• plugins/automation/workflows/steps/operations/summarize.py - Result aggregation
• plugins/automation/workflows/steps/operations/split.py - Deterministic splitting

Test Coverage

• 35 tests total, all passing ✅
• Fast execution (~0.6 seconds)
• Mocked AI calls for deterministic testing
• Comprehensive coverage:
  • test_mapreduce_operations.py (20 tests): Split, exploration, summarize operations
  • test_ai_split_operation.py (15 tests): AI split validation, execution, error handling

Workflow Examples

• ai_exploration_workflow.yaml: AI-driven exploration (AI decides how to split)
• exploration_mapreduce.yaml: Manual task specification

Usage Example

workflow:
  name: "ai-codebase-exploration"
  inputs:
    goal: {type: string, required: true}
    codebase_path: {type: string, required: true}

  steps:
    # Step 1: AI intelligently splits the goal
    - id: ai_split
      type: transform
      config:
        operation: ai_split
        model: haiku
        max_tasks: 8
      inputs:
        goal: "{{ inputs.goal }}"
        codebase_path: "{{ inputs.codebase_path }}"

    # Step 2: Explore each task in parallel (future)
    - id: explore_tasks
      type: transform
      config:
        operation: explore
        exploration_type: question
        session_dir: .mcp_sessions
      inputs:
        task: "{{ task }}"
        codebase_path: "{{ inputs.codebase_path }}"

    # Step 3: Aggregate all findings
    - id: summarize
      type: transform
      config:
        operation: summarize
        summary_format: structured
        output_file: summary.json
      inputs:
        session_id: "{{ context.execution_id }}"

Documentation

• Updated plugins/automation/workflows/README.md with:
  • Map-reduce exploration operations section
  • Usage examples for all new operations
  • Architecture diagram updates
  • Implementation status

Testing

All tests pass with mocked AI calls:

uv run pytest plugins/automation/tests/test_mapreduce_operations.py -v  # 20 tests
uv run pytest plugins/automation/tests/test_ai_split_operation.py -v    # 15 tests

Architecture

The system follows the existing workflow operation pattern:

• Pluggable operations via OperationRegistry
• Validation before execution
• Async execution model
• Proper error handling

Breaking Changes

None - this is purely additive functionality.

Future Enhancements

• True parallel execution of exploration tasks (currently sequential in workflow)
• Resume capabilities from session files
• Progressive summarization for long-running explorations
• Additional exploration types (e.g., dependency analysis, architecture mapping)

🤖 Generated with Claude Code