feat: add TelecomBackup benchmark — power-backup scheduling for telecom sites (PowerSystems)

benchmarks/PowerSystems/README.md

@@ -9,3 +9,7 @@ Current tasks emphasize realistic operational constraints, economic objectives,
   - Unified benchmark: `task=unified task.benchmark=PowerSystems/EV2GymSmartCharging`
   - Quick run: `python -m frontier_eval task=unified task.benchmark=PowerSystems/EV2GymSmartCharging task.runtime.env_name=frontier-eval-driver algorithm.iterations=0`
   - Description: upstream-aligned EV smart charging with transformer constraints in the real `EV2Gym` simulator
+- `TelecomBackup`
+  - Unified benchmark: `task=unified task.benchmark=PowerSystems/TelecomBackup`
+  - Quick run: `python -m frontier_eval task=unified task.benchmark=PowerSystems/TelecomBackup algorithm.iterations=0`
+  - Description: time-sequenced on/off scheduling of telecom backup power supplies to maximize outage backup time while keeping LTE coverage >= 80%

benchmarks/PowerSystems/README_zh-CN.md

@@ -9,3 +9,7 @@
   - `frontier_eval` 任务：`task=unified task.benchmark=PowerSystems/EV2GymSmartCharging`
   - 快速运行：`python -m frontier_eval task=unified task.benchmark=PowerSystems/EV2GymSmartCharging task.runtime.env_name=frontier-eval-driver algorithm.iterations=0`
   - 简介：在真实上游 `EV2Gym` 模拟器中进行、与上游数据对齐的 EV 智能充电与变压器约束优化
+- `TelecomBackup`
+  - `frontier_eval` 任务：`task=unified task.benchmark=PowerSystems/TelecomBackup`
+  - 快速运行：`python -m frontier_eval task=unified task.benchmark=PowerSystems/TelecomBackup algorithm.iterations=0`
+  - 简介：电信备电电源的时序开关调度，在停电时最大化备电时长同时保持 LTE 覆盖 ≥ 80%

benchmarks/PowerSystems/TelecomBackup/.gitignore

@@ -0,0 +1,2 @@
+__pycache__/
+*.pyc

benchmarks/PowerSystems/TelecomBackup/README.md

@@ -0,0 +1,164 @@
+# TelecomBackup: Power-Backup Scheduling for Telecom Sites (Frontier-Eng Benchmark)
+
+An **original** Frontier-Engineering benchmark: given a region of telecom sites powered by
+batteries, a solver must produce a **time-sequenced on/off schedule for every power supply** that
+maximizes the region's total backup time while keeping good LTE coverage (RSRP > -105 dBm) above
+80% at every moment. The power-consumption parameters live in each instance and are calibrated so
+that stagger/rotation scheduling has clear, reproducible headroom over the naive always-on strategy
+(see "Scoring").
+
+The full game rules and evaluation semantics are in [Task.md](./Task.md) (Chinese).
+
+## Layout
+
+```
+benchmarks/PowerSystems/TelecomBackup/
+├── baseline/solver.py          # Candidate solver (EVOLVE-BLOCK region is the only editable part)
+├── verification/
+│   ├── generator.py            # Fixed-seed instance generator
+│   ├── simulator.py            # Scoring simulator (coverage/power/battery simulation)
+│   ├── evaluate.py             # Evaluation entry (subprocess + time budget + scoring)
+│   ├── validator.py            # Integrity checks (static + env stripping + determinism)
+│   ├── ref_solver.py           # Reference heuristic (rest-rotation) — documented "good" score
+│   ├── test_simulator.py       # Unit tests: simulator correctness
+│   ├── test_validator.py       # Unit tests: integrity checks / env stripping / determinism
+│   ├── test_evaluator.py       # Unit tests: end-to-end evaluation behavior
+│   ├── data/instances/         # 8 fixed instances (seed-fixed, reproducible)
+│   └── requirements.txt
+├── frontier_eval/              # UnifiedTask metadata (ConnectFour/AntGame pattern)
+├── Task.md                     # Task rules, interface, scoring, reference scores
+└── README.md
+```
+
+## Requirements
+
+- Python >= 3.10, standard library only (no third-party dependencies).
+- Runtime is pure-Python simulation; each instance evaluation takes well under a second for
+  the baseline solver.
+
+## Run
+
+```powershell
+# Score a solver on the fixed 8-instance set (default 60s time budget per instance)
+python verification/evaluate.py baseline/solver.py
+
+# Add runtime-generated instances (anti-hardcoding)
+python verification/evaluate.py baseline/solver.py --generate-seed <SEED>
+
+# Tighter budget (challenge tier: 10s)
+python verification/evaluate.py baseline/solver.py --time-budget 10
+```
+
+### Docker
+
+The evaluator is pure stdlib, so a minimal `python` image suffices. Build it and
+use the unified runtime's `isolation_mode=docker`:
+
+```bash
+# Build (inside the TelecomBackup directory)
+docker build -t telecombackup-benchmark -f verification/docker/Dockerfile .
+
+# From the repo root
+python -m frontier_eval task=unified task.benchmark=PowerSystems/TelecomBackup algorithm.iterations=0 \
+  task.runtime.isolation_mode=docker task.runtime.docker_image=telecombackup-benchmark
+```
+
+> Docker isolation is validated on Linux / WSL. `frontier_eval/eval_command.txt`
+> injects the host-benchmark path via the `{benchmark_source}` placeholder, so
+> scoring works without framework changes; if the container user cannot write
+> the evaluation sandbox, set `task.runtime.docker_user=<host uid>:<host gid>`
+> (e.g. `1000:1000`). On Windows hosts the unified docker path is blocked by a
+> framework path bug (`Path.resolve()` rewrites container paths to drive
+> paths) — run docker mode under WSL instead.
+
+## Tests
+
+```powershell
+# From the TelecomBackup task directory (stdlib unittest, no dependencies)
+python -m unittest discover -s verification -p "test_*.py"
+```
+
+34 tests across four modules (simulator / validator / evaluator / sandbox): simulator correctness (manual golden cases, interval
+normalization, battery depletion, coverage constraint, determinism), validator integrity
+(EVOLVE-BLOCK / forbidden references / absolute paths / per-instance hardcoding / env
+stripping / determinism probe), and evaluator behavior (scoring, malformed/timeout/preflight
+handling, runtime generation, reproducibility).
+
+To run inside the Frontier-Eng framework (unified task):
+
+```powershell
+# Windows: point the unified runtime at the venv python (WSL bash can't run Windows exes,
+# so also use an MSYS2/Git bash instead of the default `bash`). Set PYTHONUTF8=1 to avoid
+# GBK decoding crashes in some framework libs, and raise the LLM timeout for thinking models.
+$env:PYTHONUTF8 = "1"
+$env:FRONTIER_EVAL_UNIFIED_PYTHON = "<repo>\.venvs\frontier-eval-driver\Scripts\python.exe"
+$env:TELECOM_EVAL_GENERATE_SEED = "<SEED>"  # runtime-generated instances (anti-hardcoding); 勿用固定值
+python -m frontier_eval task=unified task.benchmark=PowerSystems/TelecomBackup algorithm.iterations=0 "task.runtime.shell=<path to Git Bash>" llm.timeout=300
+```
+
+Note: evaluation spawns solver subprocesses with a time budget; if running very slow solvers,
+increase `FRONTIER_EVAL_EVALUATOR_TIMEOUT_S` accordingly (e.g. 1200).
+
+## Integrity / threat model
+
+- **Runtime-generated instances**: with `TELECOM_EVAL_GENERATE_SEED` set, the evaluator
+  generates fresh instances at evaluation time (temp dir, never in the repo/sandbox), so a
+  candidate cannot pre-position solutions for them.
+- **Candidate env stripping**: candidate subprocesses get `FRONTIER_*` / `TELECOM_EVAL_*`
+  variables stripped (see `verification/validator.py`), closing the host-env side channel.
+- **Static checks**: EVOLVE-BLOCK markers + fixed-region byte diff vs the initial baseline,
+  forbidden imports of evaluation/generation modules, absolute paths, per-instance hardcoding,
+  plus a determinism probe (two runs must match). Any violation scores 0.
+- Honest note: in process mode the candidate has host filesystem access (framework-wide
+  limitation); this benchmark relies on the layered defenses above. `verification/simulator.py`
+  is intentionally exposed as a white-box scorer for candidate-side search.
+- **Sandbox scope** (design trade-off): the 8 fixed instances and the evaluator/validator
+  sources are visible to the candidate during evolution (they are needed for scoring and the
+  simulator is intentionally usable). Anti-hardcoding therefore relies on
+  `TELECOM_EVAL_GENERATE_SEED` (fresh instances at evaluation time — set it, do not use a
+  fixed seed); the name-keyed hardcoding check is best-effort (array-index dispatch can evade
+  it, as in any static check). The reference solver (`ref_solver.py`) and the generator are
+  **not** copied into the sandbox and are additionally forbidden by the validator.
+
+## Scoring
+
+- Instances = 8 fixed (N = 20..40 sites, K = 6..12 power supplies, each instance carries its
+  power-consumption params `p_silent` / `p_work_base` / `p_work_coef`) + runtime-generated when
+  `TELECOM_EVAL_GENERATE_SEED` is set; score = mean backup time (minutes).
+- Malformed output / out-of-range intervals / crash / timeout ⇒ 0 points for that instance.
+- **Power calibration**: instances are generated with `p_silent=0.05`, `p_work_base=3.0`,
+  `p_work_coef=3.0` (silent is cheap, working is expensive), and the generator accepts only
+  instances where a multi-rest stagger (rest 1..3 supplies at a time) beats "always-on" by ≥ 25%
+  per instance — so the scheduling problem has large, reproducible headroom (verified: +28%..+119%,
+  avg +54%).
+- Reference scores (measured on the fixed 8 instances, deepseek-v4-flash agents; agent scores are
+  **verified by directly evaluating the saved programs** from the task directory — candidate
+  solvers resolve `verification/simulator.py` relative to their own location, so re-evaluating a
+  saved program from an arbitrary path silently degrades it to the always-on fallback). **The
+  published agent scores are on the fixed 8 instances only** (no `TELECOM_EVAL_GENERATE_SEED`
+  was set for those runs); set the seed to additionally score fresh instances, which is the
+  recommended anti-hardcoding configuration:
+  - baseline (always-on, no scheduling): **176.2** minutes
+  - agent (openevolve, 25 iterations, best saved program): **414.4** minutes (+135%; run `20260816_130700`); on fixed 8 + 8 generated (seed 42): **410.9**
+  - agent (ShinkaEvolve, 15 generations, best generation program): **312.5** minutes (+77%; run `20260816_214014`, gen 3); on fixed 8 + 8 generated (seed 42): **319.4**
+  - agent (AB-MCTS, 15 iterations, best saved program): **266.9** minutes (+52%; run `20260816_220646`); on fixed 8 + 8 generated (seed 42): **280.3**
+  - reference heuristic (`verification/ref_solver.py`, multi-rest rotation): **271.2** minutes (+54%)
+  - **multi-run statistics** (3 runs per framework, best valid saved program per run):
+    - openevolve (25 iterations): 414.4 / 298.1 / 357.5 → **mean 356.7 ± 47.5**
+    - ShinkaEvolve (15 generations): 312.5 / 357.5 / 325.6 → **mean 331.9 ± 18.9**
+    - AB-MCTS (15 iterations): 266.9 / 208.8 / 227.5 → **mean 234.4 ± 24.2**
+  - horizon: 480 minutes (upper bound if coverage never fails)
+  - note: with 5-15 iterations agents mostly plateau at the baseline; more iterations let all
+    three frameworks discover stagger/coverage-driven schedules that beat it (openevolve even
+    found a near-horizon minimal-covering-subset schedule, surpassing the reference heuristic).
+    ShinkaEvolve's 25-generation run produced a higher-scoring but **non-deterministic** program
+    (time-budgeted simulated annealing) that fails the determinism probe — only deterministic
+    programs count (312.5 from the 15-generation run is the valid best).
+
+## Time Budget Tiers (from the original problem)
+
+| Tier | --time-budget | Notes |
+|---|---|---|
+| Base | 300 s | T+1: 10-min solve |
+| Advanced (default) | 60 s | T+2: 1-min solve |
+| Challenge | 10 s | T+3: 10-s solve |

benchmarks/PowerSystems/TelecomBackup/Task.md

@@ -0,0 +1,126 @@
+# 区域备电栅格级优化（TelecomBackup）
+
+## 任务概述
+
+现网某片区域停电时，需要基于各基站（站点）的备电电量、功耗与覆盖关系，设置**每个电源的时序开关策略**，
+在"任意时刻良好覆盖栅格比例 ≥ 80%"的约束下，**最大化区域总备电时长**。
+
+本质是一个受限的调度/优化问题：覆盖约束让问题解空间巨大，因此要求在给定的求解时间预算内
+给出尽可能优的开关调度。
+
+## 实例输入
+
+评测会传入一个实例 JSON（路径作为命令行参数），包含：
+
+| 字段                           | 含义                                    |
+| ---------------------------- | ------------------------------------- |
+| `grid`                       | 区域 200m×200m，栅格化为 nx×ny（默认 20×20=400） |
+| `sites`                      | 站点坐标列表（N 个）                           |
+| `groups`                     | 电源分组：每个电源管理一组站点（K 个电源，每组共享一块电池）       |
+| `battery`                    | 每块电池电量（kWh）                           |
+| `demand`                     | 每个栅格的需求（相对负载单位，0\~1）                  |
+| `pt_dbm / n_exp / threshold` | 覆盖参数：Pt=20dBm、路径损耗指数 n=6、阈值 -105dBm   |
+| `coverage_ratio`             | 覆盖约束比例 0.8                            |
+| `delta_min`                  | 时隙长度 5 分钟                             |
+| `horizon`                    | 时隙总数 96（=8 小时规划期）                     |
+| `site_cap`                   | 站点额定容量（归一化负载用）                        |
+
+## 规则（评分模拟器语义）
+
+- **覆盖**：站点 s 对栅格 g 的电平 `P(g,s) = pt_dbm - 10*n_exp*log10(d(g,s)+1)`（dBm，d 为米）。
+  栅格接入**最强存活站点**（存活 = 所属电源开启且电池未耗尽）；栅格"良好" ⟺ 覆盖电平 > -105dBm。
+- **功耗**：站点工作功耗 `p_work_base + p_work_coef*min(load_s/site_cap, 1.0)` kW（load\_s 为其覆盖栅格需求之和，负载相关；当前实例 `p_work_base=3.0, p_work_coef=3.0`）；
+  站点静默功耗 **`p_silent`** kW（当前实例 `p_silent=0.05`；电源关闭时其下站点静默：不提供覆盖，但仍耗静默电）。功耗参数随实例提供（见实例 JSON 与 README）。
+- **负载迁移**：电源关闭 → 站点静默 → 其覆盖栅格**即时**接入最强存活站点 → 接收站点负载与功耗上升。
+- **电量**：电源 k 每时隙扣电 `Δt × Σ(站点功耗)`；电量 ≤ 0 → 该电源停服（不耗电、不覆盖）。
+- **备电时长**：从时隙 0 起模拟推进，首个"良好栅格比例 < 80%"的时隙的前一时刻即为备电终点；
+  撑满 horizon 则备电时长 = horizon×5 分钟。
+
+## 决策输出（你的程序必须输出的格式）
+
+运行方式：`python baseline/solver.py <instance.json>`，在 **stdout** 打印一个 JSON：
+
+```json
+{"on": [
+  [[0, 96]],                  // 电源 0：全程开启
+  [[0, 40], [60, 96]],        // 电源 1：0..39 与 60..95 开启
+  []                          // 电源 2：从不开启
+]}
+```
+
+- `on[k]` 是电源 k 的**开启时隙区间**列表，每个区间为半开区间 `[a, b)`（0-index，0 ≤ a ≤ b ≤ horizon）。
+- 电源 k 在时隙 s 开启 ⟺ 存在区间使 `a ≤ s < b`。相邻/重叠区间会被合并，写错顺序没关系。
+- `[[0, horizon]]` = 全程开启（最简单合法的解）；`[]` = 全程关闭。
+- **格式错误、区间越界、崩溃、超时 → 该实例得 0 分**（模拟器会验证每个调度）。
+
+## 评测与分数
+
+- 实例池 = 固定公开实例（默认 8 个，跨规模 N=20\~40、K=6\~12）+ **运行时生成实例**（防硬编码）。
+- 分数 = 各实例备电时长的**平均值（分钟）**。
+- 每个实例给固定求解时间预算（默认 **60s**），超时判 0 分。
+- 时间预算可配置：`--time-budget 300`（10min）/ `60`（1min）/ `10`（10s），预算越紧分数通常越低。
+
+本地运行：
+
+```powershell
+python verification/evaluate.py baseline/solver.py            # 固定 8 实例
+python verification/evaluate.py baseline/solver.py --time-budget 10
+python verification/evaluate.py baseline/solver.py --generate-seed 42   # 固定 8 + 运行时生成 8（防硬编码）
+```
+
+框架（unified）评测时，**必须在宿主环境设置** **`TELECOM_EVAL_GENERATE_SEED`** 开启运行时生成
+（否则只有固定公开实例）：
+
+```powershell
+$env:TELECOM_EVAL_GENERATE_SEED = "<SEED>"
+python -m frontier_eval task=unified task.benchmark=PowerSystems/TelecomBackup algorithm.iterations=0 "task.runtime.shell=<path to Git Bash>"
+```
+
+### 完整性 / 防作弊（威胁模型）
+
+- **运行时生成实例**：设了 `TELECOM_EVAL_GENERATE_SEED` 后，评测现场按种子生成新实例
+  （只存在于临时目录，不落仓库/沙箱），候选无法预置针对它们的解。
+- **候选环境剥离**：候选子进程的环境变量剥离 `FRONTIER_*` 与 `TELECOM_EVAL_*`，
+  封死通过宿主环境定位评测基线的侧信道。
+- **静态检查**：评分前检查 EVOLVE-BLOCK 标记与标记外代码（与初始 baseline 逐字节比对）、
+  禁 import 评测/生成模块、禁绝对路径、禁按实例名硬编码；同一实例跑两次输出必须一致
+  （确定性探针）。任何违规判 0 分。
+- **诚实说明**：process 模式下候选进程有宿主文件系统访问权（框架的限制）；本基准
+  依赖"运行时生成 + env 剥离 + 静态检查"多层防御，评测器（`verification/simulator.py`）
+  是有意暴露给候选做内部搜索的白盒。
+
+## 参考分数（实测，固定 8 实例）
+
+| 策略                                | 平均备电时长                      |
+| --------------------------------- | --------------------------- |
+| baseline（朴素全程开启，不调度）           | **176.2** 分钟               |
+| agent（AB-MCTS，15 迭代，最优保存程序实测）   | **266.9** 分钟（+52%）      |
+| 参考启发式（`verification/ref_solver.py`，多路休息轮换） | **271.2** 分钟（+54%） |
+| agent（ShinkaEvolve，15 代，最佳代程序实测）  | **312.5** 分钟（+77%）      |
+| agent（openevolve，25 迭代，最优保存程序实测） | **414.4** 分钟（+135%）    |
+| horizon（覆盖永不跌破 80% 的上限）         | 480 分钟                   |
+
+baseline 只做"全程开启"（覆盖最高但电池并行耗尽）；`ref_solver.py` 的"多路轮流休息"错峰调度
+能显著延长备电时长；三个 agent 框架在足够迭代下都能发现超过全程开启的调度，其中 openevolve
+在 25 迭代下找到接近 horizon 上限的**最小覆盖子集轮换**调度（甚至超过简单参考启发式 53%）。
+实例已校准（静默功耗便宜、工作功耗贵），生成器保证错峰相对全程开启每实例 ≥25% 的可复现提升
+（实测 +28%..+119%，平均 +54%）。
+
+> 注意：agent 分数以**保存程序直跑**为准，且必须**从任务目录内**重新评估——候选求解器按自身
+> 位置解析 `verification/simulator.py`，从任意路径直跑会静默退化为全程开启（分数虚低）。
+> ShinkaEvolve 的"保存 best"会低估其最佳代（框架追踪问题），取其最高分代为准。
+> ShinkaEvolve 25 代时曾产生更高分（~409）但为**时间型模拟退火、非确定性**，不过确定性探针，
+> 故不计（有效最高为 15 代的 312.5）。
+> 5~15 迭代时 agent 大多停在基线附近；迭代越多越能充分搜索——本任务奖励"彻底的模拟器内搜索"。
+
+## 优化方向提示
+
+1. 备电终点几乎总是"某电源耗尽导致覆盖跌破"——策略本质是**让电池错峰放电**。
+2. 覆盖是**冗余**的（80% 约束远小于全开覆盖），可以让部分电源轮换休息（静默省电），
+   只要保证任意时刻覆盖 ≥ 80%。
+3. 关闭电源的负载会迁移到剩余站点，其功耗上升——轮换分组要**地理邻近**（同簇一组），
+   避免单组覆盖不足。
+4. 你可以在求解器里 `import verification/simulator.py`（只读）来评估候选调度的备电时长，
+   做"生成-模拟-改进"的搜索（时间预算内迭代）。
+5. 先保证输出**永远合法**（宁可全开也别格式错），再追求调度质量。
+