Add iree_evo package for evolutionary optimization of IREE compiler

[Copilot]

Implements an autonomous optimization system for IREE using evolutionary strategies via openevolve. Targets Integer-Only Requantization (fusing float dequant/requant into integer math).

Package Structure (experimental/iree_evo/)

• knowledge_base.py — Valid IREE flags by backend (llvm-cpu, cuda, rocm) and Transform Dialect ops
• slicer.py — MLIR analysis: extracts compute ops, tensor shapes, quantization patterns; parses compilation errors with context for LLM feedback
• verification.py — LIT test generation with CHECK/CHECK-NOT patterns per optimization strategy
• evaluator.py — 5-phase evaluation pipeline: compile → structural verify → correctness → benchmark; returns fitness scores
• prompts.py — System prompts for Planner (strategy selection) and Coder (flag/script generation) agents
• main.py — CLI entry point with mock LLM functions for standalone testing

Usage

from iree_evo import IREEEvaluator, KnowledgeBase, MLIRSlicer

# Analyze MLIR
summary = MLIRSlicer.extract_summary(mlir_content)
# {'compute_ops': ['linalg.matmul'], 'quantization_ops': ['arith.sitofp', 'arith.mulf'], ...}

# Get optimization flags
flags = KnowledgeBase.get_optimization_strategy_flags("IntegerRequantization")

# Evaluate configuration
evaluator = IREEEvaluator(baseline_mlir_path="model.mlir", target_backend="llvm-cpu")
score = evaluator.evaluate(config_string)  # Returns 1000/latency_ms or negative on failure

Evaluation Scoring

Outcome	Score
Compilation failure	-100
Structural verification failure	-50
Correctness failure	-10
Success	1000.0 / mean_latency_ms

Compilation failures capture debug dumps via --mlir-print-ir-after-all and parse error context for LLM feedback loops.

Original prompt

IMPLEMENTATION TASK: IREE-EVO SYSTEM

Context:
We are building an autonomous optimization system for the IREE compiler using the openevolve library. The system will use Evolutionary Strategies to optimize compilation flags and Transform Dialect scripts.

Objective:
Generate the complete Python codebase for the iree_evo package. The system must optimize for the specific task of "Integer-Only Requantization" (fusing float dequant/requant operations into integer math).

Dependencies:

• openevolve (assume installed)
• iree.compiler and iree.runtime
• numpy
• lit (llvm-lit)

---

REQUIRED FILES

Please generate the following Python files. Ensure all code is robust, uses subprocess for external calls, and handles errors gracefully.

1. iree_evo/knowledge_base.py

• Class KnowledgeBase:
  • Method get_valid_flags(backend: str): Returns a list of valid IREE flags for a given backend (e.g., llvm-cpu, cuda).
  • Method get_transform_ops(): Returns a list of valid MLIR Transform Dialect operations (e.g., transform.structured.tile).
  • Note: For now, you can hardcode a subset of flags relevant to quantization and tiling.

2. iree_evo/slicer.py

• Class MLIRSlicer:
  • Method extract_summary(mlir_content: str): Returns a JSON dict containing:
    • compute_ops: List of ops like linalg.matmul, linalg.conv_2d.
    • tensor_shapes: List of shapes found in the IR.
  • Method parse_compilation_error(stderr: str):
    • Input: The standard error output from a failed iree-compile.
    • Logic: Use Regex to find the error message (e.g., error: ...) and extract the specific line number and 5 lines of surrounding context.
    • Output: A clean string summarizing the error for the LLM.

3. iree_evo/verification.py

• Class LitGen:
  • Method generate_structural_test(strategy: str, original_mlir: str) -> str:
    • Creates a .mlir file content ready for llvm-lit.
    • Logic:
      • If strategy == "IntegerRequantization":
        • Add // CHECK-NOT: arith.sitofp
        • Add // CHECK-NOT: arith.mulf
        • Add // CHECK: arith.muli
        • Add // CHECK: arith.shrsi
  • Method run_lit(test_filepath: str) -> bool: Runs llvm-lit and returns True if passed.

4. iree_evo/evaluator.py

• Class IREEEvaluator (Inherits from openevolve.evaluator.Evaluator):
  • Constructor: Accepts baseline_mlir_path, target_backend.
  • Method evaluate(self, individual: str) -> float:
    • The individual string contains the compiler flags and/or transform script.
    • Phase 1: Write: Save individual to flags.txt.
    • Phase 2: Compile (The "Build" Check):
      • Run iree-compile with the flags.
      • CRITICAL: If compilation FAILS:
        • Re-run with --mlir-print-ir-after-all to capture the debug dump.
        • Use MLIRSlicer.parse_compilation_error to get the error details.
        • raise a CompilationError (or return score -100) containing the debug info so OpenEvolve can feed it back to the LLM.
    • Phase 3: Structural Verification:
      • Use LitGen to create a test file.
      • Run llvm-lit. If fail, return score -50.
    • Phase 4: Correctness:
      • Run iree-run-module on the artifact.
      • Compare outputs vs baseline using numpy.allclose. If fail, return score -10.
    • Phase 5: Benchmark:
      • Run iree-benchmark-module.
      • Parse mean latency.
      • Return 1000.0 / mean_latency_ms.

5. iree_evo/prompts.py

• Define the System Prompts for the agents.
• PLANNER_PROMPT:
  • Role: IREE Architect.
  • Task: Analyze MLIR and choose a strategy from the Optimization Menu (Tile, Vectorize, Quantize, etc.).
• CODER_PROMPT:
  • Role: IREE Compiler Engineer.
  • Task: Generate the specific flags/scripts.
  • Constraint: Must use the provided "Math" for Integer Requantization ($M \approx \text{multiplier} \cdot 2^{-n}$).

6. iree_evo/main.py

• Initialize the IREEEvaluator.
• Initialize openevolve.Controller.
• Step 1: Run the Planner LLM (mock or real API call) to decide on "Integer Requantization".
• Step 2: Start the OpenEvolve loop to mutate the flags/scripts to achieve that goal.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.