Add PrimerDesignOptimization benchmark for PCR primer design

.gitignore

@@ -16,6 +16,7 @@ benchmarks/Astrodynamics/MannedLunarLanding/results.txt
 benchmarks/Astrodynamics/MannedLunarLanding/outputlog.txt
 .vscode
 *.csv
+!leaderboard/*.csv
 *.h5ad
 *.h5ad.tmp
 *.h5ad.tmp.*
@@ -46,6 +47,7 @@ outputlog.txt
 frontier_eval/conf/batch/*
 !frontier_eval/conf/batch/example_matrix.yaml
 !frontier_eval/conf/batch/v1.yaml
+!frontier_eval/conf/batch/v1_lite.yaml
 metrics.json
 artifacts.json
 debug-*.log

README.md

@@ -14,6 +14,10 @@ Frontier-Eng is a benchmark for **generative optimization**: agents iteratively
 
 The benchmark currently covers **47 tasks** across computing, quantum information, operations research, robotics and control, optics and communications, and physical sciences. The project homepage and paper frame it as a missing evaluation axis between pass/fail coding benchmarks and real engineering work: most engineering problems start from a feasible baseline and reward iterative improvement, not one-shot correctness.
 
+## News
+
+- **2026-06-30** — **New scoring metric: the Medal Score (gold/silver/bronze).** Alongside average rank, we now release a peer-relative *Medal Score* (normalized to `[0,1]`). On each task the top-3 best-feasible scores in the v1 snapshot are frozen as gold/silver/bronze baselines; a model earns 1.00 / 0.67 / 0.33 for reaching each, averaged over the task set, and is reported on both v1 (47 tasks) and v1-lite (10 tasks). It rewards only reaching each task's frontier and ignores negligible long-tail margins, making cross-task aggregation fairer. Per-task podium values and the leaderboard live in [`leaderboard/`](leaderboard/README.md).
+- **2026-06-30** — **`v1-lite` released.** A 10-task representative subset of `v1` covering all five categories with distinct benchmark families, selected for tasks whose scores climb gradually under budget (not one-shot-saturated or all-or-nothing). Run it with `frontier_eval/conf/batch/v1_lite.yaml`.
 
 ## 0. Host Requirements
 
@@ -97,6 +101,14 @@ bash scripts/batch/validate_v1_task_envs.sh
 
 That command runs the batch config for the `v1` problem set with `algorithm.iterations=0`, which evaluates each task's shipped baseline instead of asking an LLM to improve it.
 
+### `v1-lite` quick subset
+
+For fast iteration and ablations, use the 10-task `v1-lite` matrix
+([`frontier_eval/conf/batch/v1_lite.yaml`](frontier_eval/conf/batch/v1_lite.yaml))
+instead of the full `v1` config. It spans all five categories with distinct
+benchmark families and favors tasks whose scores improve gradually under budget,
+so a short run still exercises the full optimization loop.
+
 If you want the full `v1` problem set with normal optimization runs later, see [`run.md`](run.md).
 
 ## Where To Go Next
@@ -108,18 +120,20 @@ If you want the full `v1` problem set with normal optimization runs later, see [
 
 ## Leaderboard
 
-Detailed leaderboard: [lab.einsia.ai/frontier-eng/leaderboard.html](https://lab.einsia.ai/frontier-eng/leaderboard.html)
-
-| Rank | Model | Average Rank |
-| :--: | :--- | --: |
-| 1 | GPT-5.4 | 3.54 |
-| 2 | Claude Opus 4.6 | 3.63 |
-| 3 | GLM-5 | 4.34 |
-| 4 | DeepSeek V3.2 | 4.76 |
-| 5 | Gemini 3.1 Pro Preview | 5.53 |
-| 6 | Grok 4.20 | 5.82 |
-| 7 | SEED 2.0 Pro | 5.86 |
-| 8 | Qwen3 Coder Next | 6.71 |
+Detailed leaderboard (incl. average rank): [lab.einsia.ai/frontier-eng/leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard). Released score tables and the per-task medal podium: [`leaderboard/`](leaderboard/README.md).
+
+**Medal Score** (gold/silver/bronze podium, higher is better, normalized to `[0,1]` = mean per-task podium credit). On each task the top-3 best scores in the **v1 snapshot (2026-04-14)** are frozen as gold/silver/bronze baselines; a model earns 1.00 / 0.67 / 0.33 for reaching each. Reported on both the full **v1** set (47 tasks) and the **v1-lite** subset (10 tasks); gold/silver/bronze counts are for v1 (see [`leaderboard/`](leaderboard/README.md)):
+
+| Rank | Model | Medal (v1) | Medal (v1-lite) | 🥇 | 🥈 | 🥉 |
+| :--: | :--- | --: | --: | --: | --: | --: |
+| 1 | GPT-5.4 | 0.596 | 0.667 | 24 | 5 | 2 |
+| 2 | Claude Opus 4.6 | 0.490 | 0.501 | 9 | 18 | 6 |
+| 3 | GLM-5 | 0.312 | 0.233 | 4 | 10 | 12 |
+| 4 | DeepSeek V3.2 | 0.248 | 0.166 | 3 | 9 | 8 |
+| 5 | Gemini 3.1 Pro Preview | 0.213 | 0.200 | 3 | 6 | 9 |
+| 6 | Seed 2.0 Pro | 0.185 | 0.100 | 3 | 7 | 3 |
+| 7 | Grok 4.20 | 0.184 | 0.133 | 3 | 6 | 5 |
+| 8 | Qwen3 Coder Next | 0.121 | 0.000 | 3 | 3 | 2 |
 
 ## Contributing
 

README_zh-CN.md

@@ -8,6 +8,11 @@ Frontier-Eng 是一个面向 **generative optimization** 的 benchmark：Agent 
 
 当前版本包含 **47 个任务**，覆盖计算系统、量子信息、运筹优化、机器人控制、光学通信、物理与工程设计。主页和论文的核心观点是：真实工程问题通常从一个可行 baseline 出发，价值来自持续优化，而不是 pass/fail。
 
+## News
+
+- **2026-06-30** — **新增评测指标：金银铜 Medal Score。** 在 average rank 之外，我们发布同侪相对的 *Medal Score*（归一化到 `[0,1]`）：每道题取 v1 snapshot 中最好的前三名分数冻结为金/银/铜 baseline，模型达到金/银/铜分别得 1.00 / 0.67 / 0.33，对题集求均值；同时汇报 v1（47 题）与 v1-lite（10 题）。它只奖励"达到该题最前沿（领奖台）"，忽略长尾里可忽略的微小差距，使跨题汇总更公平。每题 podium 分数与榜单见 [`leaderboard/`](leaderboard/README.md)。
+- **2026-06-30** — **发布 `v1-lite`。** `v1` 的 10 题代表性子集，覆盖全部五大类、family 各不相同，专选"分数随预算逐步提升（而非一步做满或非高即低）"的题，配置见 `frontier_eval/conf/batch/v1_lite.yaml`。
+
 ## 这个 benchmark 在测什么
 
 和传统 agent benchmark 相比，Frontier-Eng 更关注三件事：
@@ -110,18 +115,20 @@ bash scripts/batch/validate_v1_task_envs.sh
 
 ## Leaderboard
 
-详细榜单见 [lab.einsia.ai/frontier-eng/leaderboard.html](https://lab.einsia.ai/frontier-eng/leaderboard.html)。
-
-| 排名 | Model | Average Rank |
-| :--: | :--- | --: |
-| 1 | Claude Opus 4.6 | 3.18 |
-| 2 | GLM-5 | 4.02 |
-| 3 | DeepSeek V3.2 | 4.41 |
-| 4 | Gemini 3.1 Pro Preview | 5.34 |
-| 5 | Grok 4.20 | 5.60 |
-| 6 | SEED 2.0 Pro | 5.63 |
-| 7 | GPT-5.4 | 5.68 |
-| 8 | Qwen3 Coder Next | 6.68 |
+详细榜单（含 average rank）见 [lab.einsia.ai/frontier-eng/leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard)。发布的分数表与每题金银铜 podium 见 [`leaderboard/`](leaderboard/README.md)。
+
+**Medal Score**（金银铜 podium，越高越好，归一化到 `[0,1]`，即每题领奖台得分的均值）。每题取 **v1 snapshot (2026-04-14)** 的前三名分数冻结为金/银/铜 baseline，模型达到金/银/铜分别得 1.00 / 0.67 / 0.33。同时汇报 **v1**（47 题）与 **v1-lite**（10 题）两个集合；金银铜次数为 v1（`gpt-5.4` 采用其 47 题全量重测结果）：
+
+| 排名 | Model | Medal (v1) | Medal (v1-lite) | 🥇 | 🥈 | 🥉 |
+| :--: | :--- | --: | --: | --: | --: | --: |
+| 1 | GPT-5.4 | 0.596 | 0.667 | 24 | 5 | 2 |
+| 2 | Claude Opus 4.6 | 0.490 | 0.501 | 9 | 18 | 6 |
+| 3 | GLM-5 | 0.312 | 0.233 | 4 | 10 | 12 |
+| 4 | DeepSeek V3.2 | 0.248 | 0.166 | 3 | 9 | 8 |
+| 5 | Gemini 3.1 Pro Preview | 0.213 | 0.200 | 3 | 6 | 9 |
+| 6 | Seed 2.0 Pro | 0.185 | 0.100 | 3 | 7 | 3 |
+| 7 | Grok 4.20 | 0.184 | 0.133 | 3 | 6 | 5 |
+| 8 | Qwen3 Coder Next | 0.121 | 0.000 | 3 | 3 | 2 |
 
 ## 贡献
 

benchmarks/Bioinformatics/PrimerDesignOptimization/README.md

@@ -0,0 +1,123 @@
+# PCR Primer Design Optimization
+
+Design forward and reverse PCR primers for a synthetic DNA template.
+This benchmark evaluates candidate primer pairs on melting temperature, GC content, complementarity,
+and structural stability under a nearest-neighbor thermodynamic model.
+
+## File Structure
+
+```text
+PrimerDesignOptimization/
+├── README.md
+├── README_zh-CN.md
+├── Task.md
+├── Task_zh-CN.md
+├── references/
+│   ├── primer_config.json
+│   └── hidden_templates/
+│       ├── hidden_at_rich_001.json
+│       ├── hidden_gc_rich_001.json
+│       └── hidden_medium_001.json
+├── scripts/
+│   └── init.py
+├── baseline/
+│   ├── solution.py
+│   └── result_log.json
+├── verification/
+│   ├── evaluator.py
+│   ├── test_evaluator.py
+│   └── requirements.txt
+└── frontier_eval/
+    ├── eval_command.txt
+    ├── initial_program.txt
+    ├── agent_files.txt
+    ├── artifact_files.txt
+    ├── constraints.txt
+    ├── evaluator.py
+    └── run_eval.py
+```
+
+## Quick Start
+
+### 1. Install Dependencies
+
+```bash
+pip install -r verification/requirements.txt
+```
+
+### 2. Run the Baseline Optimizer
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python scripts/init.py
+# Outputs: submission.json
+```
+
+### 3. Evaluate a Submission
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python verification/evaluator.py --submission submission.json
+```
+
+### 4. Evaluate a Candidate Program Directly
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python verification/evaluator.py scripts/init.py
+```
+
+## Submission Format
+
+Write `submission.json` containing the candidate primer pair:
+
+```json
+{
+  "forward_primer": "GCTAGCTAGCTAGCTAGCT",
+  "reverse_primer": "GCTAGCTAGCTAGCTAGCT"
+}
+```
+
+Both keys are required. Each primer must be an uppercase DNA string consisting only of the characters A, T, C, and G.
+
+## Task Summary
+
+- **Template**: 120 bp synthetic DNA sequence
+- **Amplicon region**: bases 21–100 (80 bp target)
+- **Primer length**: 18–25 bp
+- **GC content**: 40–60%
+- **Melting temperature**: 50–58°C (optimal 55°C)
+- **Max Tm difference**: 3°C between primers
+- **Thermodynamics**: Nearest-neighbor model (SantaLucia 1998)
+- **Salt conditions**: 50 mM monovalent, 2 mM divalent, 0.8 mM dNTP
+
+## Scoring
+
+Hard validation gates determine feasibility. Among feasible candidates,
+ten weighted quality metrics rank primer pairs.
+
+The ten quality metrics are:
+
+- **tm_score**: proximity to optimal melting temperature
+- **gc_content_score**: proximity to 50% GC content
+- **length_score**: proximity to 20–22 bp primer length
+- **gc_clamp_score**: G/C fraction in the last 3 bases (3′ end)
+- **self_complementarity_score**: degree of self-complementarity
+- **pair_complementarity_score**: degree of cross-dimer complementarity
+- **repeat_score**: degree of internal sequence repeats
+- **hairpin_score**: degree of hairpin structure
+- **mononucleotide_run_score**: degree of homopolymer runs
+- **product_length_score**: proximity to preferred product length
+
+An infeasible submission (one that violates any hard validation gate) receives a final score of zero.
+Higher final score is better.
+
+## Run with frontier_eval (unified)
+
+Unified benchmark: `task=unified task.benchmark=Bioinformatics/PrimerDesignOptimization`
+
+```bash
+python -m frontier_eval \
+task=unified task.benchmark=Bioinformatics/PrimerDesignOptimization \
+algorithm.iterations=10
+```

benchmarks/Bioinformatics/PrimerDesignOptimization/README_zh-CN.md

@@ -0,0 +1,122 @@
+# PCR 引物设计优化（PCR Primer Design Optimization）
+
+基于人工合成的 DNA 模板，设计正向与反向 PCR 引物。
+该 Benchmark 通过熔解温度（Tm）、GC 含量、互补性及结构稳定性，
+在最邻近热力学模型下对候选引物对进行评估。
+
+## 文件结构（File Structure）
+
+```text
+PrimerDesignOptimization/
+├── README.md
+├── README_zh-CN.md
+├── Task.md
+├── Task_zh-CN.md
+├── references/
+│   ├── primer_config.json
+│   └── hidden_templates/
+│       ├── hidden_at_rich_001.json
+│       ├── hidden_gc_rich_001.json
+│       └── hidden_medium_001.json
+├── scripts/
+│   └── init.py
+├── baseline/
+│   ├── solution.py
+│   └── result_log.json
+├── verification/
+│   ├── evaluator.py
+│   ├── test_evaluator.py
+│   └── requirements.txt
+└── frontier_eval/
+    ├── eval_command.txt
+    ├── initial_program.txt
+    ├── agent_files.txt
+    ├── artifact_files.txt
+    ├── constraints.txt
+    ├── evaluator.py
+    └── run_eval.py
+```
+
+## 快速开始（Quick Start）
+
+### 1. 安装依赖
+
+```bash
+pip install -r verification/requirements.txt
+```
+
+### 2. 运行基线优化程序
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python scripts/init.py
+# 输出：submission.json
+```
+
+### 3. 评估提交结果
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python verification/evaluator.py --submission submission.json
+```
+
+### 4. 直接评估候选程序
+
+```bash
+cd benchmarks/Bioinformatics/PrimerDesignOptimization
+python verification/evaluator.py scripts/init.py
+```
+
+## 提交格式（Submission Format）
+
+编写 `submission.json` 文件，包含候选引物对：
+
+```json
+{
+  "forward_primer": "GCTAGCTAGCTAGCTAGCT",
+  "reverse_primer": "GCTAGCTAGCTAGCTAGCT"
+}
+```
+
+两个键均为必填。每条引物必须为大写 DNA 字符串，仅包含 A、T、C、G 四种字符。
+
+## 任务概要（Task Summary）
+
+- **模板（Template）**：120 bp 人工合成 DNA 序列
+- **扩增区域（Amplicon region）**：碱基 21–100（目标 80 bp）
+- **引物长度（Primer length）**：18–25 bp
+- **GC 含量（GC content）**：40–60%
+- **熔解温度（Melting temperature）**：50–58°C（最优 55°C）
+- **引物间最大 Tm 差（Max Tm difference）**：3°C
+- **热力学模型（Thermodynamics）**：最邻近模型（SantaLucia 1998）
+- **盐浓度条件（Salt conditions）**：50 mM 一价阳离子、2 mM 二价阳离子、0.8 mM dNTP
+
+## 评分（Scoring）
+
+硬性验证门（Hard Validation Gates）决定可行性（feasibility）。在可行候选方案中，
+十项加权质量评价指标（Weighted Quality Metrics）对引物对进行排序。
+
+十项质量评价指标如下：
+
+- **tm_score**：接近最优熔解温度的程度
+- **gc_content_score**：接近 50% GC 含量的程度
+- **length_score**：接近 20–22 bp 引物长度的程度
+- **gc_clamp_score**：最后 3 个碱基（3′ 端）中 G/C 的比例
+- **self_complementarity_score**：自互补性程度
+- **pair_complementarity_score**：引物间二聚体互补性程度
+- **repeat_score**：内部序列重复程度
+- **hairpin_score**：发夹结构程度
+- **mononucleotide_run_score**：同聚核苷酸连续重复（Homopolymer Run）程度
+- **product_length_score**：接近偏好扩增产物长度的程度
+
+违反任何硬性验证门的提交（不可行提交）将获得最终得分（final_score）零分。得分越高越好。
+
+## 使用 frontier_eval（统一）运行
+
+统一 Benchmark：`task=unified task.benchmark=Bioinformatics/PrimerDesignOptimization`
+
+```bash
+python -m frontier_eval \
+task=unified task.benchmark=Bioinformatics/PrimerDesignOptimization \
+algorithm.iterations=10
+```