Add anti-self-bias evaluation suite

.gitignore

@@ -6,3 +6,4 @@ __pycache__/
 *.pyd
 *.egg-info/
 data/growth_logs.jsonl
+evals/results/

CHANGELOG.md

@@ -0,0 +1,50 @@
+# Changelog
+
+## 1.0.0 - 2026-05-13
+
+- mark the repository as a stable offline prototype release
+- consolidate README and architecture documentation around the implemented
+  pipeline, safety boundaries, and evaluation surface
+- add release checklist artifacts
+- keep the project explicitly bounded: no LLM, no network reasoning, no real
+  external actions, and human judgment preserved for high-risk scenarios
+- add CLI smoke coverage for release commands
+
+## 0.9.0 - 2026-05-13
+
+- consolidate architecture, attention, collision lane, growth memory, safety,
+  and evaluation documentation
+- align README, roadmap, and docs with the implemented prototype
+
+## 0.8.0 - 2026-05-13
+
+- add self-improvement proposals without autonomous self-modification
+
+## 0.7.0 - 2026-05-13
+
+- add evaluation harness, benchmark scenarios, deterministic baselines,
+  structured reports, and failure analysis
+
+## 0.6.0 - 2026-05-13
+
+- add versioned rule configuration
+
+## 0.5.0 - 2026-05-13
+
+- add attention sovereignty and input immunity scoring
+
+## 0.4.0 - 2026-05-13
+
+- add growth-log replay and memory feedback
+
+## 0.3.0 - 2026-05-13
+
+- add factor families and collision lanes
+
+## 0.2.0 - 2026-05-13
+
+- add dynamic factor generation and richer big-situation output
+
+## 0.1.0 - 2026-05-13
+
+- initialize the offline rule-based InnerBrain-Factor prototype

README.md

@@ -1,32 +1,39 @@
 # InnerBrain-Factor
 
-InnerBrain-Factor v0.1 is a small, fully offline Python prototype for the
-InnerBrain-AGI Seed research direction. This repository does not claim to
-implement AGI. It focuses on one narrow validation loop:
+InnerBrain-Factor is an offline AGI Seed research prototype. This `v1.0.0`
+release does not claim to implement AGI. It is a deterministic, rule-based
+system for studying:
+
+- input disturbance
+- attention sovereignty
+- dynamic small factors
+- value-field guidance
+- factor families and collision lanes
+- big-situation integration
+- growth-log replay and memory feedback
+- self-improvement proposals under human approval
+- evaluation against simpler baselines
+
+## Current Status
+
+The project currently provides a stable offline prototype with:
+
+- a fully local rule pipeline
+- structured CLI commands
+- JSONL growth logs
+- evaluation scenarios and baseline comparison
+- self-improvement proposal generation without autonomous self-modification
+- pytest and GitHub Actions coverage
+- a release baseline for deterministic benchmark comparison
+
+This repository is intentionally bounded:
 
-input disturbance -> small-factor generation -> value-field weighting ->
-factor collision and fusion -> big-situation output -> growth log recording
-
-## Scope
-
-This version is intentionally constrained:
-
-- pure rule-based reasoning
 - no LLM integration
-- no network access inside the pipeline
+- no network reasoning
 - no real external actions
-- explicit human review for risky scenarios
-
-## What Is Included
+- no autonomous override of human judgment
 
-- `MASTER_SPEC.md`: long-term theory and research framing
-- `BUILD_V0.1.md`: the executable scope for the current prototype
-- `innerbrain/`: the rule engine, models, pipeline, CLI, and growth log store
-- `tests/`: pytest coverage for scoring, factor generation, value review,
-  collisions, situation integration, and growth logs
-- `docs/`: supporting architecture notes
-
-## Quick Start
+## Install
 
 Use Python 3.11+.
 
@@ -37,52 +44,185 @@ pip install -e .[dev]
 pytest
 ```
 
-Run the CLI demo:
+## Core CLI
+
+Run one reasoning pass:
 
 ```bash
 innerbrain run \
-  --question "是否应该让内脑 AGI 自动联网搜索最新论文？" \
+  --question "是否应该让系统自动联网搜索最新论文？" \
   --goal "评估是否允许受控联网研究"
 ```
 
-The CLI prints:
+Replay recent growth logs:
+
+```bash
+innerbrain replay --limit 5
+```
+
+Summarize memory signals:
+
+```bash
+innerbrain summarize-memory --limit 20
+```
+
+Run the benchmark harness:
 
-- 核心判断
-- 扰动评分
-- 激活小因子
-- 价值冲突
-- 风险点
-- 是否需要人类裁决
-- 成长日志摘要
+```bash
+innerbrain eval \
+  --scenario-dir evals/scenarios \
+  --output-path evals/results/latest.json \
+  --failure-report-path evals/results/latest_failures.md
+```
+
+Render an existing evaluation report:
+
+```bash
+innerbrain eval-report --report-path evals/results/latest.json
+```
+
+Run the external-style benchmark validation pack:
 
-Growth logs are appended to `data/growth_logs.jsonl`.
+```bash
+innerbrain external-benchmark \
+  --benchmark-dir evals/external_benchmarks \
+  --output-path evals/results/external_benchmark.json \
+  --failure-report-path evals/results/external_benchmark_failures.md
+```
 
-## Repository Layout
+Render an existing external benchmark report:
+
+```bash
+innerbrain external-benchmark-report \
+  --report-path evals/results/external_benchmark.json
+```
+
+Run the anti-self-bias suite:
+
+```bash
+innerbrain anti-bias run \
+  --dataset-path evals/anti_self_bias_dataset.json \
+  --output-path evals/results/anti_self_bias_report.json \
+  --report-path evals/results/anti_self_bias_report.md
+```
+
+Render an existing anti-self-bias report:
+
+```bash
+innerbrain anti-bias report \
+  --report-path evals/results/anti_self_bias_report.json
+```
+
+Render the anti-self-bias comparison view:
+
+```bash
+innerbrain anti-bias compare \
+  --left-report-path evals/results/anti_self_bias_report.json
+```
+
+Generate improvement proposals:
+
+```bash
+innerbrain propose-improvements \
+  --from-report evals/results/latest.json
+```
+
+## Safety Boundaries
+
+- Networking, self-modification, privileged tools, autonomous execution, and
+  real-world impact require `human_judgment_required = true`.
+- The system may recommend offline prototyping, simulation, logging, or human
+  escalation, but it never executes the external step.
+- Core safety boundaries, value constraints, and human authorization are not
+  self-modified by the system.
+- Self-improvement is proposal-only. All proposals remain under human review.
+
+## Evaluation Focus
+
+The benchmark harness compares `innerbrain_factor` against:
+
+- `cot`
+- `reflection`
+- `multi_agent_debate`
+- `risk_rule`
+
+Measured dimensions include:
+
+- human judgment correctness
+- risk detection
+- value conflict detection
+- evidence gap detection
+- lane activation coverage
+- robustness
+- consistency
+- long-term goal preservation
+- long-horizon stability
+- attention stability
+- distraction resistance
+- safety correctness
+- creativity-safety balance
+- over-gating rate
+
+The external-style benchmark pack also supports ablation re-runs that disable:
+
+- value field
+- collision lanes
+- attention sovereignty
+- growth memory
+
+The anti-self-bias suite is intentionally stricter than the in-repo benchmark
+packs. It treats `innerbrain` as a black-box answer generator and scores only
+the final text output with an external evaluator. Its goal is to detect whether
+the system is improving on independent tasks or only improving on its own
+internal benchmark structures. The suite also logs external failure classes
+such as false positives, false negatives, attention hijack, value collapse,
+and unstable decisions so regression review does not collapse into a single
+headline score.
+
+On the bundled deterministic scenario set, `innerbrain_factor` is expected to
+outperform the included heuristic baselines on the composite benchmark while
+preserving the project's offline safety boundaries. To refresh the current
+numbers, run `innerbrain eval` and inspect the CLI dashboard plus structured
+JSON output.
+
+## Project Layout
 
 ```text
 innerbrain-factor/
+  configs/
+  docs/
+  evals/
+  innerbrain/
+  tests/
   README.md
   MASTER_SPEC.md
   BUILD_V0.1.md
   pyproject.toml
-  innerbrain/
-  tests/
-  docs/
-  data/
 ```
 
-## Safety Boundaries
-
-- Network access, autonomous action, self-modification, high-privilege tools,
-  and real-world impact always trigger `human_judgment_required = true`.
-- The pipeline may recommend offline prototyping or human escalation, but it
-  never executes the external step.
-- The system preserves uncertainty and records its reasoning trace in JSONL.
-
-## Development Notes
-
-- Models are implemented with Pydantic.
-- The CLI uses Typer.
-- Tests are written with pytest.
-- The design is intentionally verbose in structure so that later v0.2/v0.3
-  iterations can deepen the rule engine without rewriting the package layout.
+## Completed Versions
+
+- `v0.1`: core offline rule pipeline
+- `v0.2`: dynamic factor generation and richer situation output
+- `v0.3`: factor families and collision lanes
+- `v0.4`: growth-log replay and memory feedback
+- `v0.5`: attention sovereignty and input immunity scoring
+- `v0.6`: versioned rule configuration
+- `v0.7`: evaluation harness, scenarios, baselines, metrics, failure reports
+- `v0.8`: self-improvement proposals without autonomous self-modification
+- `v0.9`: architecture and safety documentation consolidation
+- `v1.0`: stable offline prototype release packaging and CLI smoke validation
+- `v1.1`: external-style benchmark validation, ablations, and typed failure logging
+- `v1.2`: anti-self-bias evaluation suite with black-box scoring, external baselines, and ASBI
+
+## Documentation
+
+- [Architecture](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/architecture.md)
+- [Factor Model](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/factor_model.md)
+- [Value Field](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/value_field.md)
+- [Collision Lanes](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/collision_lanes.md)
+- [Growth and Memory](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/growth_memory.md)
+- [Attention Sovereignty](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/attention_sovereignty.md)
+- [Safety Boundaries](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/safety_boundaries.md)
+- [Evaluation](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/evaluation.md)
+- [Roadmap](/Users/qianjinus/Documents/类脑/innerbrain-factor-v07/docs/roadmap.md)

RELEASE_CHECKLIST.md

@@ -0,0 +1,36 @@
+# Release Checklist
+
+## Release Boundary
+
+- [x] Repository states clearly that it is not AGI
+- [x] Repository states clearly that it is an offline AGI Seed research prototype
+- [x] No LLM integration in the reasoning pipeline
+- [x] No network reasoning in the reasoning pipeline
+- [x] No real external action execution from the reasoning pipeline
+- [x] Human judgment remains required for networking, self-modification,
+  privileged tools, autonomous external execution, and real-world impact
+
+## Code And Tests
+
+- [x] `pytest` passes before release
+- [x] GitHub Actions `tests` workflow exists for PR and `main`
+- [x] CLI smoke coverage validates release commands
+- [x] Evaluation harness, scenario datasets, metrics, and failure reports are
+  present in the repository
+
+## Documentation
+
+- [x] `README.md` matches the current CLI and safety boundaries
+- [x] `docs/architecture.md` matches the implemented pipeline
+- [x] `docs/evaluation.md` explains metrics, baselines, and failure analysis
+- [x] `CHANGELOG.md` summarizes major completed versions
+- [x] `MASTER_SPEC.md` and `BUILD_V0.1.md` remain as historical scope anchors
+
+## Release Commands
+
+- [x] `innerbrain run`
+- [x] `innerbrain replay`
+- [x] `innerbrain summarize-memory`
+- [x] `innerbrain eval`
+- [x] `innerbrain eval-report`
+- [x] `innerbrain propose-improvements`