Deploy: Claude leaderboard entries + PR #71 fixes + your look-ahead fix, reconciled

.github/workflows/ci.yml

@@ -29,6 +29,11 @@ jobs:
           pip install pytest pytest-timeout matplotlib
 
       - name: Run backend tests
+        # DATABASE_PATH is a belt-and-suspenders backstop: tests/conftest.py
+        # already self-isolates into a tempdir, but if that fixture ever
+        # regresses, this keeps CI from writing into the committed seed DB.
+        env:
+          DATABASE_PATH: ${{ runner.temp }}/ci-backtest.db
         run: pytest dashboard/backend/tests/ --timeout=180 -p no:cacheprovider
 
   packaging-tests:

.gitignore

@@ -242,4 +242,12 @@ FinAgents/agent_pools/backtest_agent/qlib_data/
 # Local Alpaca credentials (use credentials/alpaca.json.example as template)
 credentials/*
 !credentials/alpaca.json.example
-!credentials/README.md
+!credentials/README.md
+
+# claude-mem auto-generates per-folder recall stubs named CLAUDE.md in every
+# directory it visits (empty <claude-mem-context> placeholders that fill in over
+# time). The feature stays ON — but these are machine-managed noise in git, so
+# ignore every *nested* CLAUDE.md while keeping the real, team-shared root file.
+# To commit a genuine nested CLAUDE.md on purpose: git add -f path/to/CLAUDE.md
+**/CLAUDE.md
+!/CLAUDE.md

CLAUDE.md

@@ -2,11 +2,10 @@
 
 This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
 
-> **Packaging contract (this branch).** This branch restructured the backend from
-> flat top-level modules into the **`dashboard.backend` Python package**. This
-> `CLAUDE.md` documents that packaged contract. It intentionally **supersedes** the
-> older flat-imports `CLAUDE.md` on `main` — when this merges, take **this** version
-> (that older guidance describes code that no longer exists here). See
+> **Packaging contract.** The backend is the **`dashboard.backend` Python package**
+> (restructured from flat top-level modules in PR #67, hardened by PR #71 — both long
+> since merged; this packaged layout is the only one that exists now). Import by full
+> package path; run via the app's import string, never by file path. See
 > `docs/architecture/dashboard-target-structure.md` for the full layout.
 
 ## What this repo is
@@ -49,7 +48,7 @@ pytest packaging/agentictrading/tests/ -v
 python dashboard/scripts/backtest_hourly_agent.py     # main hourly agent backtest
 ```
 
-`tests/conftest.py` points `DATABASE_PATH` at a temp file before any backend import, so tests never touch the committed `dashboard/storage/data/backtest.db`.
+`dashboard/backend/tests/conftest.py` points `DATABASE_PATH` at a temp file before any backend import, so tests never touch the committed `dashboard/storage/data/backtest.db`. The suite is green end-to-end (the old "5 pre-existing failures" were retired in PR #71) — a red test on a fresh run is a real regression.
 
 ## Environment & credentials
 
@@ -61,18 +60,21 @@ python dashboard/scripts/backtest_hourly_agent.py     # main hourly agent backte
 
 Pipeline is **backtest → SQLite → API → dashboard**. The backend is layered (see `docs/architecture/dashboard-target-structure.md`):
 
-- **`api/`** — FastAPI surface. Business routers live in `api/routers/*` and are mounted by `api/router.py` under `/api`. **Paper-trading routes stay outside `/api`** (registered directly on the app), so `/paper/*` is the external contract. `app.py` is the composition root (creates the app, middleware, startup hooks, serves the frontend).
-- **`domain/`** — business logic by area: `runs/` (Agent-Environment Protocol: Run/Step/Decision), `agents/`, `leaderboard/` (contest + baseline strategies registry), `backtesting/` (engine, `external_run_service`, portfolio manager), `strategies/` (free-form strategy store), `chat/`, `trading/`. Domain must not import `api/`/`app.py`.
-- **`infrastructure/`** — `llm/` (the `validator` security boundary, `token_cost`, `backtest_harness`/gateway client) and `market_data/` (Alpaca bars).
-- **`services/` / re-export shims** — several moved modules keep thin compatibility re-exports.
-- **Persistence** (`database.py` + per-store repositories like `domain/runs/repository.py`, `domain/strategies/repository.py`): thin SQLite wrappers over `DATABASE_PATH`; schema is created lazily and self-migrates.
-- **Frontend** (`dashboard/frontend/`): vanilla JS + Chart.js, no build step. Served by the backend locally/Render; also deployed static to Vercel.
+- **`api/`** — FastAPI surface. Business routers live in `api/routers/*` and are mounted by `api/router.py` under `/api`; the canonical agent contract is `api/v2/*` (see "Agent API v2" below). **Paper-trading routes stay outside `/api`** (registered directly on the app), so `/paper/*` is the external contract. `app.py` is the composition root (creates the app, middleware, startup hooks, serves both frontends).
+- **`domain/`** — business logic by area: `runs/` (Agent-Environment Protocol: Run/Step/Decision), `agents/`, `leaderboard/` (contest + baseline strategies registry + the H6 integrity guard), `backtesting/` (engine, `external_run_service`, portfolio manager, `baselines/` subpackage), `strategies/` (free-form strategy store), `chat/`, `trading/` (live paper trading: `paper_session`, `execution`, `portfolio`). Domain must not import `api/`/`app.py`.
+- **`execution/`** — v2 execution backends binding domain engines to the `/api/v2` contract: `base.py` (interface), `backtest_backend.py` (implemented), `paper_backend.py` (**stub** — raises `NotImplementedError`; Phase B not built). Deliberately at the backend root (not `domain/`) so it can bridge domain→API without tripping the `domain/`→`api/` import ban.
+- **`infrastructure/`** — `llm/` (the `validator` security boundary, `token_cost`, `backtest_harness`/gateway client), `market_data/` (Alpaca bars), and `brokers/` (`alpaca_paper.py`, the isolated Alpaca paper-trading HTTP adapter).
+- **Backend-root modules** — `middleware.py` (session enforcement + CSP), `users.py` (auth/bcrypt/session store), `cache.py` (TTL cache for paper-trading responses), `baseline_generator.py`/`baseline_resolver.py`/`baselines_endpoint.py` (shared baseline equity-curve generation + DJIA/buy-hold baselines for backtests and paper trading), `llm_integration_example.py` (reference safe-LLM pattern). (`engines/` and `services/` are **not** packages — they were pre-refactor compatibility shims, deleted once their code moved under `domain/`; `test_architecture_boundaries.py`'s `_DELETED_SHIMS` list asserts they stay non-importable.)
+- **Persistence** (`database.py` + per-store repositories like `domain/runs/repository.py`, `domain/strategies/repository.py`): thin SQLite wrappers over `DATABASE_PATH` in **WAL journal mode** (readers aren't blocked by finalize's heavy writes); schema is created lazily and self-migrates. `agent_runs` carries a JSON `metadata` column recording the effective `LLM_MAX_OUTPUT_TOKENS` per run.
+- **Frontend** — `dashboard/frontend/` is the served static root and holds **both** UIs: the **landing page** (`index.html` + `assets/`) served at **`/`**, and the vanilla-JS + Chart.js **dashboard** (`app.html`, `app.js`, `styles.css`, no build step) served at **`/app`**. The landing page is a Vite/React marketing site whose **source** lives in `dashboard/landing/` (Replit-exported, de-monorepo'd; `npm run build`); its build output ships as `frontend/index.html` + `frontend/assets/`. `app.py` adds a `/app/`→`/app` 308 redirect so the dashboard's relative asset paths resolve. Vercel deploys the static `dashboard/frontend`.
 - **Paths** (`dashboard/backend/paths.py`): single source of truth for on-disk locations.
 
 ### Baseline strategies (registry pattern)
 
 `dashboard/backend/domain/leaderboard/strategies/` holds benchmark strategies (`buy_hold`, `equal_weight_index`, `market_index`, `mean_variance`, `llm_agent`, …). To add one: subclass `BaselineStrategy` (`base.py`), give it a `key`, add the class to `_STRATEGY_CLASSES` in `registry.py`. `get_strategy(config)` resolves by `strategy`/`type` key.
 
+**H6 leaderboard integrity guard.** An LLM-backed entry can only publish if the model actually drove ≥95% of its steps (`MIN_LLM_DECISION_COVERAGE = 0.95`). The guard (`domain/leaderboard/service.py`) keys on `PortfolioManager.llm_decisions` — steps the model genuinely drove, incremented only at the *success exit* of the decision path — **not** `llm_calls` (a pure billing counter that also ticks on truncated/unparseable responses that then silently fall back to rule-based). This stops a rule-based fallback curve from being published under an LLM's name. See the memory note `leaderboard-h6-integrity-model` for the full rationale. All 6 LLM entries currently on the board (Claude Haiku 4.5, Sonnet 4.6, GPT-5.5, Gemini 3.1 Pro, Qwen3.7 Plus, DeepSeek V4 Pro) cleared it; only DeepSeek beat the passive baselines.
+
 ### LLM safety boundary
 
 `infrastructure/llm/validator.py` is a hard security boundary: LLM trading responses must be JSON-only matching the trading schema — `tool_calls`/`function_calls` are rejected, portfolio constraints enforced, decisions logged. Do not loosen this to allow tool/web access from agent responses.
@@ -87,8 +89,9 @@ Pipeline is **backtest → SQLite → API → dashboard**. The backend is layere
 
 Two step-driven agent surfaces coexist; they are **not peers**:
 
-- **`/api/v2` is canonical** (`api/v2/*` routers + `execution/` backends over the same domain engines): typed Pydantic contract, per-agent scopes + token-bucket rate limits, canonical `run_id`, DB-backed idempotency (`(run_id, idem_key)`), `context_ref` provenance, self-describing `GET /api/v2/schema`. Spec/plan: `docs/superpowers/{specs,plans}/2026-06-23-agent-api-foundation-*`. New agent-facing features land here (Phase B: paper/live via `ExecutionBackend`; Phase C: MCP façade).
+- **`/api/v2` is canonical** (`api/v2/*` routers + `execution/` backends over the same domain engines): typed Pydantic contract, per-agent scopes + token-bucket rate limits, canonical `run_id`, DB-backed idempotency (`(run_id, idem_key)`), `context_ref` provenance, self-describing `GET /api/v2/schema`. Spec/plan: `docs/superpowers/{specs,plans}/2026-06-23-agent-api-foundation-*`. New agent-facing features land here. **Phase B** (paper/live via `ExecutionBackend`) and **Phase C** (MCP façade) are **not built yet** — `execution/paper_backend.py` is a stub.
 - **`/api/v1` is the compatibility surface** for the shipping SDK (`packaging/agentictrading`), Discord bot, and built-in agents. Keep it working; do not grow it. Migrating the SDK to v2 is the gate for publishing `agentictrading` 0.2.0.
+- **Unified run lifecycle (v1 + v2).** The two surfaces share one active-run cap ledger (under a single lock), one reaper sweep (`register_reaper_sweep()` reaches v2 runs), and multi-worker heartbeat recovery (`owner_instance`/`heartbeat_at` columns, `RUN_HEARTBEAT_STALE_SECONDS`). Terminal v2 runs are swapped for a DB-backed `ArchivedBacktestBackend` tombstone; step/idempotency state persists across process restarts; v2 `cancel`/`status` report the true terminal status (not always "closed").
 - `execution/` sits at the backend root (not `domain/`) deliberately: the backends bind domain engines to the v2 API contract, and `test_architecture_boundaries` forbids `domain/` → `api/` imports.
 
 ## Deployment
@@ -104,3 +107,5 @@ Two step-driven agent surfaces coexist; they are **not peers**:
 - The committed `dashboard/storage/data/backtest.db` holds seed runs referenced by `dashboard/config/defaults.json`. Importing a store module runs `CREATE TABLE IF NOT EXISTS` against `DATABASE_PATH`, so running the app locally can add empty tables to that file — don't commit those mutations. If you regenerate the DB, update `defaults.json`.
 - Pytest is not in `requirements.txt`; install it separately.
 - `discord.py` (for `integrations/discord_bot.py`) is an **optional** dep declared in `requirements-discord.txt` (like `requirements-sphinx.txt` for the docs build), not core `requirements.txt` — run `pip install -r requirements-discord.txt` to run the bot. It's kept out of core so web/API/backtest installs stay lean; its tests `importorskip('discord')`.
+- **Prod deploy reality vs `render.yaml`.** The live Render service runs on the **free tier** and tracks the **Allan-Feng/AgenticTrading** fork's `main`, not this repo's `render.yaml` — there is **no persistent `/data` disk**, so the running DB is the ephemeral committed seed `backtest.db` (writes evaporate on redeploy). Merging to Open-Finance-Lab `main` therefore doesn't update prod until a maintainer syncs OFL→allan-fork. The disk/plan in `render.yaml` is aspirational.
+- **Phantom `test_deleted_shim_is_not_importable` failures = stale bytecode, not a regression.** If those cases fail locally with `DID NOT RAISE ModuleNotFoundError`, it's leftover `dashboard/backend/{engines,services}/__pycache__/*.pyc` from the pre-refactor layout, which Python resolves as a PEP-420 namespace package. The dirs are untracked so CI is green; `rm -rf dashboard/backend/engines dashboard/backend/services` clears it.

dashboard/backend/api/dependencies.py

@@ -35,6 +35,12 @@ def _owner_context(request: Request, authorization: Optional[str]) -> Dict[str,
     trading_session = request.headers.get("x-session-id") or request.headers.get("X-Session-Id")
     browser_owner = request.headers.get("x-browser-id") or request.headers.get("X-Browser-Id")
     if not browser_owner:
+        # Fallback: with no X-Browser-Id, the X-Session-Id doubles as the owner
+        # identity. For agents created via import_session (which stores
+        # owner_browser_session = session_id) the session id therefore IS an
+        # ownership credential — "session_id is never a credential" only holds
+        # for built-in agents. Clients that can send X-Browser-Id should; this
+        # branch exists for API-only importers with no browser identity.
         browser_owner = trading_session
     user = _optional_user(authorization)
     return {