feat(WaterDistribution): add PumpScheduling benchmark

benchmarks/WaterDistribution/PumpScheduling/README.md

@@ -0,0 +1,56 @@
+# WaterDistribution/PumpScheduling
+
+A CPU-only closed-loop pump-scheduling benchmark on EPANET Net3 through WNTR.
+The unified benchmark ID is `WaterDistribution/PumpScheduling`.
+
+## Environment
+
+From the repository root, create the repository-owned task runtime. The task is
+tested with Python 3.11 on Linux.
+
+```bash
+python3.11 -m venv .venvs/frontier-wntr
+.venvs/frontier-wntr/bin/python -m pip install \
+  -r benchmarks/WaterDistribution/PumpScheduling/verification/requirements.txt
+```
+
+## Direct evaluation
+
+From the benchmark directory:
+
+```bash
+../../../.venvs/frontier-wntr/bin/python verification/evaluator.py \
+  scripts/init.py --json-out metrics.json --artifacts-out artifacts.json
+```
+
+## Unified baseline evaluation
+
+From the repository root:
+
+```bash
+python -m frontier_eval \
+  task=unified \
+  task.benchmark=WaterDistribution/PumpScheduling \
+  task.runtime.python_path=uv-env:frontier-wntr \
+  algorithm=openevolve \
+  algorithm.iterations=0
+```
+
+Baseline-only validation does not require a model API key.
+
+## Tests
+
+From the benchmark directory:
+
+```bash
+../../../.venvs/frontier-wntr/bin/python -m pytest -q verification/tests
+```
+
+## Resources and execution assumptions
+
+The evaluator uses three documented public scenarios and three frozen hidden
+variants. It requires CPU only, WNTR 1.4.0 and its EPANET runtime; no GPU,
+Docker, external dataset, or network access is required. The evaluator executes
+on the host. Candidate controllers run in an isolated subprocess with only the
+JSON causal-observation interface, but the evaluator itself should still be run
+only from a trusted benchmark checkout.

benchmarks/WaterDistribution/PumpScheduling/README_zh-CN.md

@@ -0,0 +1,24 @@
+# 配水系统泵站调度
+
+基于 WNTR/EPANET Net3 的纯 CPU、闭环泵站调度基准。统一任务 ID 为
+`WaterDistribution/PumpScheduling`。安装依赖后直接评测：
+
+```bash
+python -m pip install -r verification/requirements.txt
+python verification/evaluator.py scripts/init.py --json-out metrics.json --artifacts-out artifacts.json
+```
+
+从仓库根目录运行统一基线评测：
+
+```bash
+python -m frontier_eval \
+  task=unified \
+  task.benchmark=WaterDistribution/PumpScheduling \
+  task.runtime.python_path=/path/to/wntr-python \
+  algorithm=openevolve \
+  algorithm.iterations=0
+```
+
+任务仅需 CPU、WNTR 1.4.0 和 EPANET 运行时，不需要 GPU、Docker、外部数据集
+或网络。候选控制器在隔离子进程中仅通过 JSON 因果观测接口执行；评测器本身
+仍应只从可信的 benchmark checkout 运行。

benchmarks/WaterDistribution/PumpScheduling/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,8 @@
+# Third-party notices
+
+- WNTR is distributed under the BSD 3-Clause license.
+- EPANET is public-domain software developed by the US EPA.
+- Net3 is an EPANET example network distributed with WNTR. This task references
+  WNTR's installed copy and does not redistribute the network file.
+
+See `references/provenance.md` for exact versions and source links.

benchmarks/WaterDistribution/PumpScheduling/Task.md

@@ -0,0 +1,75 @@
+# Net3 Pump Scheduling
+
+## Engineering problem and value
+
+Drinking-water utilities must operate pumps while maintaining service pressure
+and storage reserves. Pump operation consumes electricity, creates demand peaks,
+and incurs mechanical wear when speeds change. This benchmark asks for a causal
+closed-loop controller that balances those costs against hydraulic feasibility
+under changing demand, tariffs, starting storage, and small leaks.
+
+## Physical model
+
+The evaluator uses the EPANET Net3 example network distributed with WNTR 1.4.0.
+Each deterministic scenario uses one continuous 24-hour EPANET hydraulic
+session with hourly control intervals and five-minute hydraulic steps; the
+network and its tank state are not recreated between intervals.
+The controller sets the speeds of pumps `10` and `335`; link `330` follows the
+existing Net3 relationship with pump `335`. The evaluator removes the original
+Net3 controls so that the submitted controller supplies the hourly actions.
+
+## Editable program and input
+
+Edit only the code between `EVOLVE-BLOCK-START` and `EVOLVE-BLOCK-END` in
+`scripts/init.py`, preserving `control(observation)`. The controller receives one
+JSON-compatible dictionary per hour containing only causal information:
+
+| Field | Type | Meaning |
+|---|---|---|
+| `hour` | integer | Current interval, from 0 through 23 |
+| `tank_levels_m` | mapping | Current pressure-head level in metres for tanks `1`, `2`, and `3` |
+| `tariff` | number | Current hourly electricity-price coefficient |
+| `demand_multiplier` | number | Current dimensionless network demand multiplier |
+| `previous_action` | mapping | Previous speeds for pumps `10` and `335`; both are 0 initially |
+
+The candidate runs in an isolated subprocess and may use only this observation
+and state derived from earlier observations in the same scenario. It cannot read
+the verifier, benchmark files, or hidden scenarios.
+
+## Output
+
+Return a dictionary with exactly the string keys `10` and `335`. Each value is a
+finite pump-speed multiplier in the closed interval `[0, 1]`.
+
+## Hard constraints
+
+Every public and hidden scenario must:
+
+- complete all 24 hydraulic intervals without controller, protocol, or EPANET
+  failure;
+- keep pressure at or above 20 m at every declared service node: `153`, `15`,
+  `253`, `103`, `127`, `101`, `129`, `251`, `255`, and `105`;
+- keep tanks `1`, `2`, and `3` within their Net3 minimum and maximum levels;
+- finish each tank no more than 0.25 m below its scenario initial level.
+
+Any failed scenario makes the complete candidate invalid and gives it zero
+aggregate score.
+
+## Objective and scoring
+
+For each valid scenario the evaluator measures hourly energy cost, peak pump
+power in kW, total speed switching, and terminal storage deficit in metres. Each
+metric is compared with the frozen shipped baseline using
+`baseline_metric / candidate_metric`, clipped to `[0, 1.5]`. The scenario score
+is:
+
+```text
+100 * (0.55 * energy_ratio
+     + 0.20 * peak_power_ratio
+     + 0.10 * switching_ratio
+     + 0.15 * terminal_recovery_ratio)
+```
+
+The six scenario scores aggregate as `0.70 * mean + 0.30 * worst_case`.
+`score` therefore has a theoretical range of 0 to 150 and higher is better.
+The unified `combined_score` is `score / 100`; the shipped baseline scores 1.0.

benchmarks/WaterDistribution/PumpScheduling/Task_zh-CN.md

@@ -0,0 +1,56 @@
+# Net3 泵站调度
+
+## 工程问题与价值
+
+饮用水配水系统需要在保证用户压力和水箱储备的同时调度水泵。水泵运行会产生
+电费与功率峰值，频繁改变泵速还会增加机械磨损。本任务要求设计因果闭环控制器，
+在需求、电价、初始储水量和小型泄漏变化下兼顾运行成本与水力可行性。
+
+## 物理模型
+
+评测使用 WNTR 1.4.0 自带的 EPANET Net3 示例网络。每个确定性场景在单个
+连续 24 小时 EPANET 水力会话中仿真，每小时控制一次并以五分钟水力步长
+推进；区间之间不重建管网或水箱状态。控制器设置水泵 `10` 和
+`335` 的速度；连接 `330` 按 Net3 中与泵 `335` 的关系切换。评测器移除 Net3
+原始控制规则，由候选控制器提供逐小时动作。
+
+## 可编辑程序与输入
+
+仅修改 `scripts/init.py` 中 `EVOLVE-BLOCK-START` 与 `EVOLVE-BLOCK-END` 之间的
+代码，并保留 `control(observation)`。每小时收到一个仅含因果信息的字典：
+
+| 字段 | 类型 | 含义 |
+|---|---|---|
+| `hour` | 整数 | 当前区间，取值 0 至 23 |
+| `tank_levels_m` | 映射 | 水箱 `1`、`2`、`3` 当前以米计的压力水头水位 |
+| `tariff` | 数值 | 当前小时电价系数 |
+| `demand_multiplier` | 数值 | 当前无量纲网络需求倍率 |
+| `previous_action` | 映射 | 水泵 `10`、`335` 上一小时速度；初始均为 0 |
+
+候选程序在隔离子进程中运行，只能使用当前观测以及同一场景中过去观测形成的状态，
+不能读取验证器、benchmark 文件或隐藏场景。
+
+## 输出
+
+返回恰好包含字符串键 `10`、`335` 的字典；每个值必须是闭区间 `[0,1]` 内的
+有限泵速倍率。
+
+## 硬约束
+
+每个公开及隐藏场景均必须完成全部 24 个区间。指定服务节点 `153`、`15`、
+`253`、`103`、`127`、`101`、`129`、`251`、`255` 和 `105` 的压力均不得低于
+20 m；三个水箱不得超出 Net3 上下界，且每个水箱终态不得比该场景初态低超过
+0.25 m。任一场景失败都会使整个候选无效并得到零聚合分数。
+
+## 目标与评分
+
+每个有效场景测量逐小时能源成本、kW 峰值功率、泵速切换总量和以米计的终态储水
+缺口。各项以 `冻结基线指标 / 候选指标` 归一化并截断到 `[0,1.5]`，场景分数为：
+
+```text
+100 * (0.55 * 能源比率 + 0.20 * 峰值功率比率
+     + 0.10 * 切换比率 + 0.15 * 终态恢复比率)
+```
+
+六个场景按 `0.70 * 平均分 + 0.30 * 最差分` 聚合。`score` 理论范围为 0 至
+150，越高越好；统一评测的 `combined_score = score / 100`，初始基线为 1.0。

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/agent_files.txt

@@ -0,0 +1,9 @@
+README.md
+README_zh-CN.md
+Task.md
+Task_zh-CN.md
+scripts/init.py
+references/scenarios_public.json
+references/service_nodes.json
+references/provenance.md
+frontier_eval/constraints.txt

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/artifact_files.txt

@@ -0,0 +1 @@
+# metrics.json and artifacts.json are handled by UnifiedTask.

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/candidate_destination.txt

@@ -0,0 +1 @@
+scripts/init.py

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/constraints.txt

@@ -0,0 +1,7 @@
+PumpScheduling unified constraints:
+1) Modify only code between EVOLVE-BLOCK-START and EVOLVE-BLOCK-END in scripts/init.py.
+2) Preserve control(observation) and return exactly pump keys 10 and 335.
+3) Pump speeds must be finite numbers in [0, 1].
+4) Use only the causal observation; do not access evaluator or hidden-case files.
+5) All six scenarios must converge and satisfy pressure, tank-bound, and terminal-storage constraints.
+6) Do not modify documentation, references, verification, tests, or frontier_eval metadata.

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/copy_files.txt

@@ -0,0 +1 @@
+.

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/eval_command.txt

@@ -0,0 +1 @@
+{python} verification/evaluator.py {candidate} --json-out metrics.json --artifacts-out artifacts.json

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/eval_cwd.txt

@@ -0,0 +1 @@
+.

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/evaluate_submission.py

@@ -0,0 +1,24 @@
+"""Python compatibility entry point for task-local evaluation."""
+
+import subprocess
+import sys
+from pathlib import Path
+
+
+def main():
+    task_root = Path(__file__).parents[1]
+    candidate = sys.argv[1] if len(sys.argv) > 1 else "scripts/init.py"
+    command = [
+        sys.executable,
+        "verification/evaluator.py",
+        candidate,
+        "--json-out",
+        "metrics.json",
+        "--artifacts-out",
+        "artifacts.json",
+    ]
+    return subprocess.call(command, cwd=task_root)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/initial_program.txt

@@ -0,0 +1 @@
+scripts/init.py

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/readonly_files.txt

@@ -0,0 +1,12 @@
+README.md
+README_zh-CN.md
+Task.md
+Task_zh-CN.md
+THIRD_PARTY_NOTICES.md
+references
+verification
+frontier_eval
+frontier_eval/run_eval.sh
+verification/evaluate.py
+verification/evaluator.py
+frontier_eval/evaluate_submission.py

benchmarks/WaterDistribution/PumpScheduling/frontier_eval/run_eval.sh

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+set -euo pipefail
+candidate="${1:-scripts/init.py}"
+"${PYTHON:-python}" verification/evaluator.py "$candidate" --json-out metrics.json --artifacts-out artifacts.json

benchmarks/WaterDistribution/PumpScheduling/references/provenance.md

@@ -0,0 +1,8 @@
+# Provenance
+
+The hydraulic model is the Net3 example bundled with WNTR 1.4.0 and loaded by
+`WaterNetworkModel("Net3")`. WNTR source and licensing:
+https://github.com/USEPA/WNTR . EPANET source:
+https://github.com/USEPA/EPANET2.2 . Scenario demand and tariff arrays are
+original synthetic benchmark data committed in this directory; no external
+operational data is included.

benchmarks/WaterDistribution/PumpScheduling/references/scenario_schema.json

@@ -0,0 +1,12 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "type": "object",
+  "required": ["id", "demand_multipliers", "tariff", "initial_tank_offsets_m", "leak"],
+  "properties": {
+    "id": {"type": "string"},
+    "demand_multipliers": {"type": "array", "minItems": 24, "maxItems": 24, "items": {"type": "number", "minimum": 0.5, "maximum": 1.6}},
+    "tariff": {"type": "array", "minItems": 24, "maxItems": 24, "items": {"type": "number", "minimum": 0.01}},
+    "initial_tank_offsets_m": {"type": "object"},
+    "leak": {"type": ["object", "null"]}
+  }
+}

benchmarks/WaterDistribution/PumpScheduling/references/scenarios_public.json

@@ -0,0 +1,5 @@
+[
+  {"id":"public_normal","demand_multipliers":[0.72,0.68,0.65,0.64,0.68,0.78,0.92,1.08,1.15,1.12,1.05,1.00,1.04,1.08,1.10,1.12,1.18,1.28,1.32,1.24,1.10,0.96,0.84,0.76],"tariff":[0.10,0.10,0.10,0.10,0.10,0.10,0.14,0.18,0.22,0.22,0.18,0.16,0.16,0.16,0.18,0.22,0.30,0.30,0.30,0.24,0.20,0.16,0.12,0.10],"initial_tank_offsets_m":{"1":0,"2":0,"3":0},"leak":null},
+  {"id":"public_peak","demand_multipliers":[0.75,0.70,0.68,0.67,0.72,0.84,1.02,1.22,1.34,1.30,1.17,1.08,1.10,1.14,1.18,1.23,1.32,1.44,1.48,1.38,1.20,1.02,0.90,0.80],"tariff":[0.08,0.08,0.08,0.08,0.08,0.10,0.14,0.20,0.28,0.28,0.22,0.18,0.18,0.18,0.22,0.28,0.36,0.36,0.36,0.30,0.22,0.16,0.10,0.08],"initial_tank_offsets_m":{"1":0.2,"2":0.1,"3":0.1},"leak":null},
+  {"id":"public_low_storage","demand_multipliers":[0.70,0.68,0.66,0.65,0.70,0.82,0.98,1.14,1.20,1.16,1.08,1.02,1.05,1.10,1.13,1.18,1.24,1.34,1.38,1.30,1.16,1.00,0.88,0.78],"tariff":[0.10,0.10,0.10,0.10,0.10,0.12,0.16,0.22,0.28,0.28,0.20,0.17,0.17,0.17,0.20,0.25,0.32,0.32,0.32,0.26,0.20,0.16,0.12,0.10],"initial_tank_offsets_m":{"1":-0.4,"2":-0.5,"3":-0.5},"leak":null}
+]

benchmarks/WaterDistribution/PumpScheduling/references/service_nodes.json

@@ -0,0 +1,4 @@
+{
+  "service_nodes": ["153", "15", "253", "103", "127", "101", "129", "251", "255", "105"],
+  "minimum_pressure_m": 20.0
+}

benchmarks/WaterDistribution/PumpScheduling/scripts/init.py

@@ -0,0 +1,24 @@
+"""Baseline controller for the Net3 pump-scheduling task."""
+
+
+# EVOLVE-BLOCK-START
+def control(observation: dict) -> dict[str, float]:
+    """Return pump speeds using only causal information in ``observation``."""
+    hour = int(observation["hour"])
+    tank_1 = float(observation["tank_levels_m"]["1"])
+    price = float(observation["tariff"])
+
+    # Pump 10 replenishes the network from the lake, mostly off peak.
+    pump_10 = 1.0 if hour in range(1, 21) else 0.0
+    if min(observation["tank_levels_m"].values()) < 2.5:
+        pump_10 = 1.0
+
+    # Pump 335 and pipe 330 regulate tank 1 in the original Net3 controls.
+    if tank_1 < 5.25:
+        pump_335 = 1.0
+    elif tank_1 > 5.80:
+        pump_335 = 0.0
+    else:
+        pump_335 = 0.0 if price > 0.20 else float(observation["previous_action"]["335"])
+    return {"10": pump_10, "335": pump_335}
+# EVOLVE-BLOCK-END

benchmarks/WaterDistribution/PumpScheduling/verification/baseline.py

@@ -0,0 +1,17 @@
+"""Frozen reference controller for evaluator normalization."""
+
+
+def control(observation: dict) -> dict[str, float]:
+    hour = int(observation["hour"])
+    tank_1 = float(observation["tank_levels_m"]["1"])
+    price = float(observation["tariff"])
+    pump_10 = 1.0 if hour in range(1, 21) else 0.0
+    if min(observation["tank_levels_m"].values()) < 2.5:
+        pump_10 = 1.0
+    if tank_1 < 5.25:
+        pump_335 = 1.0
+    elif tank_1 > 5.80:
+        pump_335 = 0.0
+    else:
+        pump_335 = 0.0 if price > 0.20 else float(observation["previous_action"]["335"])
+    return {"10": pump_10, "335": pump_335}