feat: add TelecomBackup benchmark — power-backup scheduling for telecom sites (PowerSystems)

[zzy2007zzy2007]

这个基准聚焦电信基础设施的停电备电调度：区域内基站由电池供电，停电时需要在"任意时刻 LTE 良好覆盖栅格比例 ≥ 80%"的约束下，为每个电源制定时序开关策略，最大化区域总备电时长。

任务内容

• 目标：最大化备电时长（分钟），评分 = 8 个固定实例的平均备电时长
• 实例：8 个确定性实例（20-40 站点、6-12 电源），功耗参数随实例提供（p_silent=0.05 / p_work_base=3.0 / p_work_coef=3.0），生成器保证"错峰调度 ≥ 全程开启×25%"的每实例可复现 headroom（实测 +28%~+119%，平均 +54%）
• baseline：全程开启（176.2 分钟）——朴素但正常，可被错峰明显超越
• 参考求解器：verification/ref_solver.py（多路休息轮换，271.2 分钟，+54%）
• 时间预算档：300s / 60s / 10s（模拟原始课题的三档求解时限）

验证方法

# 单元测试（34 个：simulator / validator / evaluator / sandbox 端到端）
python -m unittest discover -s verification -p "test_*.py"

# baseline / 参考求解器
python verification/evaluate.py baseline/solver.py          # 176.25
python verification/evaluate.py verification/ref_solver.py  # 参考解（直接 simulate 打分）

# 生成实例（防硬编码）
python verification/evaluate.py baseline/solver.py --generate-seed <SEED>

Agent 实测（deepseek-v4-flash，3 轮/框架，全部由保存程序直跑验证）：

框架	3 次运行	mean ± std
openevolve（25 迭代）	414.4 / 298.1 / 357.5	356.7 ± 47.5
ShinkaEvolve（15 代）	312.5 / 357.5 / 325.6	331.9 ± 18.9
AB-MCTS（15 迭代）	266.9 / 208.8 / 227.5	234.4 ± 24.2

泛化证据（固定 8 + 生成 8，seed=42）：openevolve 410.9 / ShinkaEvolve 319.4 / AB-MCTS 280.3——agent 的解可泛化到未见过的生成实例。完整 run ID 见 baseline/result_log.txt。

Docker 隔离：纯标准库评测，最小镜像即可；Linux/WSL 实测通过（task.runtime.isolation_mode=docker）。

评测完整性

• 沙箱只复制评测所需文件（evaluate.py / simulator.py / validator.py / data/instances），参考求解器与生成器不进沙箱且被静态检查禁止引用
• EVOLVE-BLOCK 固定区归一化比对（行尾/换行差异不计）、跨规模确定性探针、FRONTIER_*/TELECOM_EVAL_* 环境变量剥离
• 防硬编码依赖 TELECOM_EVAL_GENERATE_SEED（评测时现场生成未见实例），威胁模型已在 README 如实说明
• simulator.py 有意作为白盒计分器开放给候选使用（constraints 允许）

备注

• ShinkaEvolve 的一个 25 代 run 曾产生更高分但非确定性（时间型模拟退火），被确定性探针拒绝——只计入确定性程序
• 框架侧（frontier_eval/algorithms/shinkaevolve）与 vendored ShinkaEvolve 的版本对齐属于框架环境问题，与基准无关

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new engineering benchmark task named TelecomBackup under the PowerSystems category. It also updates the global task registry to include a VehicleRouting (CVRP) entry. The TelecomBackup task involves time-sequenced scheduling of telecom power supplies to maximize backup time during outages while maintaining a minimum LTE coverage threshold.
• Modified File Structure & Modifications:
  • TASK_DETAILS.md & TASK_DETAILS_zh-CN.md: Registered VehicleRouting/CVRP and updated the task list.
  • benchmarks/PowerSystems/README.md & README_zh-CN.md: Added entry for TelecomBackup with quick-run commands.
  • benchmarks/PowerSystems/TelecomBackup/: Created a complete benchmark suite including:
    • baseline/solver.py: A template solver with EVOLVE-BLOCK markers for agents.
    • verification/: Contains the core simulation logic (simulator.py), instance generator (generator.py), and a comprehensive test suite (test_*.py).
    • frontier_eval/: Framework integration files (metadata, evaluator entry points).
    • Task.md & README.md: Detailed technical specifications and scoring rules in English and Chinese.

2. AI Content Analysis

• Estimated AI Component: 25%
• Reasoning & Evidence:
  • Boilerplate Framework Code: Files like frontier_eval/run_eval.py and frontier_eval/evaluator.py follow a highly standardized pattern seen in other tasks within this repository, likely generated or adapted via AI using existing templates.
  • Documentation: The READMEs are exceptionally well-structured and detailed. While the technical content is domain-specific, the formatting and "Notes" sections exhibit the exhaustive and polite tone typical of AI-assisted technical writing.
  • Domain Logic: The core simulation logic (RSRP calculations, battery depletion, load migration) appears to be human-engineered or heavily guided, as it requires specific domain knowledge of telecom infrastructure that generic AI models often struggle to synthesize without precise prompts.

3. Engineering & Economic Assessment

• Engineering Reality Check: Production-Grade. This is not a "toy example." It addresses a non-trivial optimization problem involving:
  • Complex Constraints: LTE coverage (RSRP > -105 dBm) and battery capacity.
  • Dynamic Load Migration: Closing one site increases the load on neighbors, a realistic physical phenomenon.
  • Anti-Cheating Mechanisms: Includes runtime-generated instances and environment stripping to prevent hardcoding solutions.
• Economic Value: High. Energy efficiency and service reliability are critical OPEX (Operating Expense) drivers for telecom operators. Algorithms that extend backup time by 50-100% (as shown in the reference scores) have direct commercial value in disaster recovery and grid stability scenarios.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: PowerSystems/TelecomBackup
  • Execution & Dependencies: The README.md provides explicit commands for both local execution and Docker-based isolation. It correctly identifies the need for PYTHONUTF8=1 on Windows to avoid encoding issues.
• Documentation Quality: Excellent. The documentation includes a clear "Threat Model," scoring breakdown, and reference benchmarks for different LLM agents (OpenEvolve, AB-MCTS, etc.). No significant grammatical errors were detected.
• Organizational Structure: Logical and Scalable. The separation of baseline, verification, and frontier_eval follows the established repository standard, making it easy for new developers to navigate.

5. Security & Privacy Check

• Sensitive Files: Clean. The .gitignore correctly excludes __pycache__ and *.pyc. No .env or API keys were found.
• Absolute Paths: None detected. The code uses Path(__file__).resolve() and relative pathing for file discovery.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 在 PowerSystems 类别下引入了一个名为 TelecomBackup（电信备电调度）的新工程基准任务。同时更新了全局任务注册表，增加了 VehicleRouting (CVRP) 条目。TelecomBackup 任务要求对电信电源进行时序开关调度，以在保持最低 LTE 覆盖阈值的前提下，最大化停电期间的区域总备电时长。
• 修改的文件结构与变更摘要:
  • TASK_DETAILS.md & TASK_DETAILS_zh-CN.md: 注册了 VehicleRouting/CVRP 并更新了任务列表。
  • benchmarks/PowerSystems/README.md & README_zh-CN.md: 增加了 TelecomBackup 的条目及快速运行命令。
  • benchmarks/PowerSystems/TelecomBackup/: 创建了完整的基准测试套件，包括：
    • baseline/solver.py: 带有 EVOLVE-BLOCK 标记的求解器模板，供 Agent 修改。
    • verification/: 包含核心仿真逻辑 (simulator.py)、实例生成器 (generator.py) 和全面的测试套件 (test_*.py)。
    • frontier_eval/: 框架集成文件（元数据、评测入口）。
    • Task.md & README.md: 中英文双语的详细技术规范和评分规则。

2. AI 成分分析

• 预估 AI 含量: 25%
• 判断依据与证据:
  • 模板化框架代码: 如 frontier_eval/run_eval.py 和 frontier_eval/evaluator.py 遵循了该仓库中其他任务的高度标准化模式，很可能是通过 AI 利用现有模板生成或改编的。
  • 文档编写: README 文档结构极其严整且详细。虽然技术内容具有领域特定性，但其格式和“Notes”部分的详尽程度表现出典型的 AI 辅助技术写作风格。
  • 领域逻辑: 核心仿真逻辑（RSRP 计算、电池耗尽、负载迁移）看起来是人工设计或深度引导的，因为这需要电信基础设施的特定领域知识，通用 AI 模型在没有精确提示的情况下很难合成此类逻辑。

3. 工程与经济评估

• 工程现实检验: 生产级。这并非“玩具示例”。它解决了一个非平凡的优化问题，涉及：
  • 复杂约束: LTE 覆盖范围 (RSRP > -105 dBm) 和电池容量。
  • 动态负载迁移: 关闭一个站点会增加邻近站点的负载，这反映了真实的物理现象。
  • 防作弊机制: 包含运行时生成实例和环境变量剥离，以防止硬编码解法。
• 经济价值: 高。能源效率和业务可靠性是电信运营商的关键运营成本 (OPEX) 驱动因素。能够将备电时长延长 50-100% 的算法（如参考分数所示）在灾难恢复和电网稳定性场景中具有直接的商业价值。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是。
  • task_name: PowerSystems/TelecomBackup
  • 运行与依赖: README.md 提供了本地运行和基于 Docker 隔离运行的明确命令。正确指出了在 Windows 上需要设置 PYTHONUTF8=1 以避免编码问题。
• 文档质量: 优秀。文档包含清晰的“威胁模型”、评分细则以及不同 LLM Agent（OpenEvolve, AB-MCTS 等）的基准参考分数。未发现明显的语法错误。
• 组织结构: 逻辑清晰且具备可扩展性。baseline、verification 和 frontier_eval 的分离遵循了既定的仓库标准，便于新开发者理解。

5. 安全与隐私检查

• 敏感文件: 未发现异常。.gitignore 正确排除了 __pycache__ 和 *.pyc。未发现 .env 或 API 密钥。
• 绝对路径: 未检测到。代码使用 Path(__file__).resolve() 和相对路径进行文件定位。

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new benchmark named TelecomBackup under the PowerSystems category. The task involves time-sequenced scheduling of telecom power supplies to maximize backup duration during outages while maintaining a minimum LTE coverage constraint (RSRP > -105 dBm for at least 80% of the area).
• Modified File Structure & Modifications:
  • benchmarks/PowerSystems/README.md & README_zh-CN.md: Added entries for the new TelecomBackup task, including quick-run commands and descriptions.
  • benchmarks/PowerSystems/TelecomBackup/README.md: Comprehensive documentation covering layout, requirements, Docker usage, and anti-hardcoding measures.
  • benchmarks/PowerSystems/TelecomBackup/Task.md: Detailed technical specifications, scoring rules, and reference performance metrics in Chinese.
  • benchmarks/PowerSystems/TelecomBackup/baseline/solver.py: A template solver implementing a naive "always-on" strategy, providing the EVOLVE-BLOCK for agents to modify.
  • benchmarks/PowerSystems/TelecomBackup/verification/: A robust verification suite including a simulator (simulator.py), instance generator (generator.py), evaluator (evaluate.py), and integrity validator (validator.py).
  • benchmarks/PowerSystems/TelecomBackup/frontier_eval/: Metadata and scripts (e.g., evaluator.py, run_eval.py) required for integration with the frontier_eval framework.

2. AI Content Analysis

• Estimated AI Component: 15-20%
• Reasoning & Evidence: The overall architecture and boilerplate (especially in frontier_eval/run_eval.py and evaluator.py) follow standard patterns likely assisted by AI. However, the core domain logic—specifically the RSRP coverage calculations, battery depletion models in simulator.py, and the highly specific calibration of power parameters (p_silent, p_work_base)—exhibits deep domain expertise. The inclusion of result_log.txt with specific run IDs and performance deltas (+135% for specific agents) indicates significant human-led empirical testing and validation that goes beyond generic AI generation.

3. Engineering & Economic Assessment

• Engineering Reality Check: This is a high-quality, production-grade engineering problem. It moves beyond "toy" examples by incorporating:
  • Realistic Constraints: Signal propagation (path loss models), battery discharge curves, and load migration.
  • Anti-Cheating Mechanisms: Runtime instance generation via TELECOM_EVAL_GENERATE_SEED and environment variable stripping to prevent hardcoding.
  • Edge Case Handling: The simulator accounts for battery exhaustion and coverage drops, penalizing invalid schedules with zero scores.
• Economic Value: High. Telecom energy management is a critical operational cost. The benchmark demonstrates that optimized scheduling can improve backup time by over 100% compared to naive strategies, which translates directly to improved network resilience and reduced operational risk during power grid failures.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: PowerSystems/TelecomBackup
  • Execution & Dependencies: The README.md clearly documents the execution commands for both local and Docker environments. It specifies Python >= 3.10 and confirms the use of the standard library to minimize dependency hell.
• Documentation Quality: Excellent. The documentation is dual-language, provides clear scoring tiers (Base, Advanced, Challenge), and includes a detailed "Integrity / threat model" section. No significant grammatical errors or formatting inconsistencies were detected in the provided diff.
• Organizational Structure: The structure is logical and modular. Separating verification logic from baseline code and frontier_eval metadata ensures scalability and ease of maintenance.

5. Security & Privacy Check

• Sensitive Files: Clean. The .gitignore correctly excludes __pycache__ and *.pyc. No .env files or API keys were found.
• Absolute Paths: None detected. The code uses Path(__file__).resolve() and relative path joining, ensuring portability across different environments.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 在 PowerSystems 类别下引入了一个名为 TelecomBackup 的新基准测试。该任务涉及电信电源的时序调度，旨在停电期间最大化备电时长，同时满足最低 LTE 覆盖约束（至少 80% 的区域 RSRP > -105 dBm）。
• 修改的文件结构与变更摘要:
  • benchmarks/PowerSystems/README.md & README_zh-CN.md: 增加了 TelecomBackup 任务的条目，包括快速运行命令和任务简介。
  • benchmarks/PowerSystems/TelecomBackup/README.md: 详尽的文档，涵盖布局、需求、Docker 使用以及防硬编码措施。
  • benchmarks/PowerSystems/TelecomBackup/Task.md: 详细的技术规范、评分规则和参考性能指标（中文）。
  • benchmarks/PowerSystems/TelecomBackup/baseline/solver.py: 模板求解器，实现了朴素的“全程开启”策略，并为 Agent 提供了可修改的 EVOLVE-BLOCK 区域。
  • benchmarks/PowerSystems/TelecomBackup/verification/: 健壮的验证套件，包括模拟器 (simulator.py)、实例生成器 (generator.py)、评估器 (evaluate.py) 和完整性校验器 (validator.py)。
  • benchmarks/PowerSystems/TelecomBackup/frontier_eval/: 与 frontier_eval 框架集成所需的元数据和脚本（如 evaluator.py, run_eval.py）。

2. AI 成分分析

• 预估 AI 含量: 15-20%
• 判断依据与证据: 整体架构和模板代码（特别是 frontier_eval/run_eval.py 和 evaluator.py）遵循了可能由 AI 辅助的标准模式。然而，核心领域逻辑——特别是 simulator.py 中的 RSRP 覆盖计算、电池耗尽模型，以及功耗参数（p_silent, p_work_base）的高度特定校准——展示了深厚的领域专业知识。result_log.txt 中包含的具体运行 ID 和性能增量（特定 Agent 提升 +135%）表明了大量由人工主导的实证测试和验证，超出了通用 AI 生成的范畴。

3. 工程与经济评估

• 工程现实检验: 这是一个高质量、生产级别的工程问题。它通过以下方式超越了“玩具级”示例：
  • 现实约束: 引入了信号传播（路径损耗模型）、电池放电曲线和负载迁移。
  • 防作弊机制: 通过 TELECOM_EVAL_GENERATE_SEED 进行运行时实例生成，并剥离环境变量以防止硬编码。
  • 边缘情况处理: 模拟器考虑了电池耗尽和覆盖率下降，对无效调度给予零分处罚。
• 经济价值: 高。电信能源管理是一项关键的运营成本。该基准测试表明，与朴素策略相比，优化调度可以将备电时间提高 100% 以上，这直接转化为电网故障期间网络韧性的增强和运营风险的降低。

4. Quality Assurance

• 验证与测试:
  • frontier_eval 集成: 是
  • task_name: PowerSystems/TelecomBackup
  • 运行与依赖: README.md 清晰地记录了本地和 Docker 环境的运行命令。它指定了 Python >= 3.10 并确认使用标准库以减少依赖冲突。
• 文档质量: 优秀。文档采用双语编写，提供了清晰的评分分级（基础、高级、挑战），并包含详细的“完整性/威胁模型”章节。在提供的 diff 中未检测到明显的语法错误或格式不一致。
• 组织结构: 结构逻辑清晰且模块化。将 verification 逻辑与 baseline 代码和 frontier_eval 元数据分离，确保了可扩展性和维护性。

5. 安全与隐私检查

• 敏感文件: 未发现异常。.gitignore 正确排除了 __pycache__ 和 *.pyc。未发现 .env 文件或 API 密钥。
• 绝对路径: 未检测到。代码使用 Path(__file__).resolve() 和相对路径拼接，确保了在不同环境下的可移植性。