Skip to content

migration_roadmap.md

Latest commit

 

History

History
103 lines (73 loc) · 7.11 KB

migration_roadmap.md

File metadata and controls

103 lines (73 loc) · 7.11 KB

Migrating ccxt_extract to rmap

Adopting the data-first roadmap workflow in this repo specifically. Companion to DESIGN.md and dashboard_roadmap.md.

Current state

  • ROADMAP.md — ~130+ tasks across phases, prose tables. Source of truth today.
  • ROADMAP.livemd — staged (AM in git status). Experimental Livebook workbench.
  • Skills assuming current shape: task-driver, audit-review, task-prioritization include, linear-workflow.md § "ROADMAP-Fallback Flow".
  • Cross-repo coordination: ccxt_client ROADMAP referenced in CLAUDE.md § "Documentation invariants" rule 6.

Target state

  • roadmap/tasks.toml — source of truth (schema in DESIGN.md).
  • ROADMAP.md — regenerated by rmap render. Marker-bounded table blocks; phase prose, "Current Focus," and notes hand-edited above/below markers.
  • roadmap/data.json — gitignored, regenerated, consumed by portfolio dashboard.
  • ROADMAP.livemd — keep as workbench (read data.json) OR deprecate. Decide separately.
  • Skills: continue reading ROADMAP.md (rendered) initially. Migrate to read tasks.toml directly only after the rendering shape stabilizes.

Migration order

  1. Ship rmap Phase 1-2 in ~/_DATA/code/rmap/. validate + render to ROADMAP.md must work end-to-end.
  2. Pilot phase: Phase 12 — recent, mid-size, well-understood ticker bundle. Hand-author roadmap/tasks.toml covering only Phase 12 tasks. Run rmap render --dry.
  3. Iterate template until rendered Phase 12 block matches the existing ROADMAP.md Phase 12 table closely enough that the diff is intentional improvements, not noise.
  4. Add markers <!-- TASKS:BEGIN phase=12 --> / <!-- TASKS:END --> around the current Phase 12 table. Run rmap render. Verify only the marked block changed.
  5. Pre-commit guard: scripts/check-roadmap-sync.sh runs rmap validate --check-render and exits non-zero if tasks.toml was edited without re-rendering. Wire via existing hook setup (no new framework).
  6. Round-trip test: flip Task 74 status in tasks.toml, rerun rmap render, confirm only the relevant row changed in ROADMAP.md.
  7. Migrate remaining phases one at a time. Order: current focus phase first (active editing), then in reverse chronological order (recent phases more likely to need flips).
  8. Update .gitignore with roadmap/data.json and roadmap/dist/ (the HTML render output dir from rmap Phase 6).
  9. (Deferred) Update skills to read tasks.toml directly. Prefer rmap commands over hand-parsing TOML — keeps skills schema-version-aware automatically:
    • task-driver: replace ROADMAP.md grep with rmap next --json (rmap Phase 8). Add rmap show <id> --json for body/AC lookup.
    • audit-review: replace markdown regex-rewrite with rmap status N done (already shipped). Use rmap diff (Phase 8) to summarize roadmap changes per PR.
    • linear-workflow / cloud delegation: use rmap delegate <id> --to <agent> (Phase 9) to generate the prompt payload instead of templating by hand.
    • task-prioritization include: update doc references if the canonical scoring location moves.
  10. Decide ROADMAP.livemd fate. If keeping: have it read roadmap/data.json (consume the same source the dashboard does, no parallel parsing logic). If deprecating: archive note in CHANGELOG + remove from CLAUDE.md references.

Pre-commit guard

scripts/check-roadmap-sync.sh:

#!/usr/bin/env bash
set -euo pipefail
rmap validate || exit 1
diff <(rmap render --stdout) ROADMAP.md > /dev/null || {
  echo "ROADMAP.md out of sync with roadmap/tasks.toml — run 'rmap render'"
  exit 1
}

Wire into existing hook configuration. If harness.yml (CI) runs the same check, drift can't even land in a PR.

Risk: ID stability

Existing ROADMAP.md task IDs (74, 75, 90, 121, etc.) MUST be preserved during migration. They're referenced by:

  • task-driver skill ("pick task N")
  • audit-review commit messages
  • Linear issues (when applicable)
  • PR titles (Task 74: parseTicker field map (#21))
  • TODO(Task N): markers in source code per CLAUDE.md convention

The TOML migration is a reformat, not a renumber. Verification: a one-time script that asserts every Task N heading in old ROADMAP.md has a matching id = N row in tasks.toml after migration.

Risk: [CX] / [CSR] / [P] marker preservation

Delegation and parallelism markers in current ROADMAP.md rows (Task 80 [CX]) must round-trip through markers = ["cx"] in tasks.toml. Render template emits the [CX] suffix. Verify on pilot phase before broad migration.

Risk: cross-repo references

CLAUDE.md § "Documentation invariants" rule 6 says: "a ccxt_extract task is not complete until its downstream ccxt_client impact is reflected." In the new schema, this is cross_repo = { repo = "ccxt_client", task_id = N }. The dashboard's cross-repo DAG view depends on this field being populated correctly during migration.

Audit existing ROADMAP.md for prose-form cross-repo notes ("blocks ccxt_client Task X", "depends on ccxt_client Y") and convert them to the structured field.

Optional schema fields (defer)

rmap Phases 9–10 add assignee, acceptance_criteria, created_at, started_at, done_at, scored_at, and blocked_reason. Do not backfill these during migration. They're all optional; populate going forward as tasks transition states. Backfilling 130+ tasks burns a day for low marginal value — the recently-active tasks accrue timestamps naturally as you flip statuses post-migration.

Exception: blocked_reason becomes required once rmap Phase 10 ships. Any task currently status = "blocked" needs a one-liner reason added at that point — that's a focused sweep, not a backfill.

Validation gates

  • After Phase 12 pilot migration: full skill roundtrip — task-driver picks a Phase 12 task correctly, audit-review flips status, commit-review parses correctly. If any skill breaks, the rendering template needs adjustment, not the skill.
  • After full migration: synthetic drift test — edit tasks.toml, skip rmap render, attempt commit, confirm pre-commit hook blocks.
  • After full migration: rmap doctor (rmap Phase 11) returns clean — no orphan deps, no cycles, no stale in_progress older than the migration date.
  • After portfolio dashboard ships: dashboard successfully consumes ccxt_extract's data.json and renders all phases correctly.

Estimated effort

  • rmap Phases 1-2 (in ~/_DATA/code/rmap/): 1-2 days
  • Phase 12 pilot migration in this repo: half a day
  • Template iteration to match existing format: half a day
  • Remaining phases migration: 1 day (mechanical, paste-and-shape)
  • Skill updates (deferred step 9): 1 day when triggered

Total to "fully migrated, skills unchanged": ~3-4 days. Skills migration: another day when you decide to do it.

Out of scope (deliberately)

  • Renumbering tasks. IDs stay stable forever, even when tasks are superseded.
  • Migrating CHANGELOG.md to structured format. Markdown changelog is fine — humans read it sequentially, not query-style.
  • Migrating SCHEMA.md, CONSUMER_CONTRACT.md, REFACTOR.md. These are prose docs, not task databases.
  • Linear sync from tasks.toml. Skills handle that externally.