Skip to content

Backend: Add scheduled reconciliation drift detection with alert hooks #861

Description

@Junirezz

Context

reconciliationReport.ts exposes an on-demand drift report comparing Horizon ledger records to persisted DB state (Issue #724).

Problem / Gap

Reconciliation is manual-only. Drift (MISSING_IN_DB, AMOUNT_MISMATCH, etc.) is not detected proactively, so indexer gaps can go unnoticed until user-facing data is wrong.

Proposed approach

  • Register a scheduled job (via jobGovernance.ts) that runs reconciliation over a rolling window.
  • Emit structured logs and Prometheus counters when status === 'DRIFT_DETECTED'.
  • Optionally open a webhook or alert hook when drift exceeds configurable thresholds.
  • Store the last successful reconciliation summary for the diagnostics bundle.

Acceptance criteria

  • Scheduled reconciliation runs on a configurable interval with retry/backoff.
  • Drift counters and last-run timestamp appear in /metrics or job metrics endpoint.
  • Admin can fetch the latest automated reconciliation summary without re-running Horizon queries.
  • Tests cover clean and drifted scenarios with mocked ledger fetcher.

Files/areas affected

  • backend/src/reconciliationReport.ts
  • backend/src/positionReconciliationJob.ts
  • backend/src/jobGovernance.ts
  • backend/src/metrics.ts

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions