Skip to content

Feature/issue 52 database backed event ledger#266

Open
Danitello123 wants to merge 4 commits into
DelegoLabs:mainfrom
Danitello123:feature/issue-52-database-backed-event-ledger
Open

Feature/issue 52 database backed event ledger#266
Danitello123 wants to merge 4 commits into
DelegoLabs:mainfrom
Danitello123:feature/issue-52-database-backed-event-ledger

Conversation

@Danitello123

@Danitello123 Danitello123 commented Jun 29, 2026

Copy link
Copy Markdown

closes #52

Summary

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation

Test plan

  • pnpm typecheck
  • pnpm test
  • Manual testing (describe):

Checklist

  • Follows project code conventions
  • TODOs reference issues where applicable
  • No secrets or credentials committed

Summary by CodeRabbit

  • New Features

    • Added escrow event monitoring so notifications can be sent automatically when escrow activity changes.
    • Added persistent workflow recovery, helping in-progress purchases resume after restarts.
    • Added transaction tracking for payments to record submission and confirmation status.
  • Bug Fixes

    • Improved notification delivery to reduce duplicate sends.
    • Added safer workflow updates with concurrency checks to prevent overwriting newer progress.
  • Documentation

    • Expanded setup docs with required environment variables for notifications and payments.

@coderabbitai

coderabbitai Bot commented Jun 29, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Three backend services gain new persistence and event-processing capabilities: the payments service adds a Sequelize-backed soroban_transaction_ledger for tracking transaction submission/confirmation states; the orchestrator gains PostgreSQL-backed workflow persistence with optimistic concurrency and crash recovery; and the notifications service gains a Stellar Soroban escrow event polling listener that dispatches idempotent email/push notifications. Two SQL migrations and an updated migration runner support all new tables.

Changes

Payments: Soroban Transaction Ledger

Layer / File(s) Summary
DB migration and Sequelize model
database/migrations/004_soroban_transaction_ledger.sql, apps/backend/payments/package.json, apps/backend/payments/src/db.ts, apps/backend/payments/src/models/SorobanTransactionLedger.ts
SQL migration creates soroban_transaction_ledger with a CHECK-constrained status column and indexes; SorobanTransactionLedger Sequelize model maps all columns with snake_case field names and timestamp defaults; db.ts initializes the pool and exports connectDb.
logSubmission, updateLedgerStatus helpers, and tests
apps/backend/payments/events/index.ts, tests/unit/src/payments-ledger.test.js, apps/backend/payments/README.md
logSubmission upserts a PENDING entry; updateLedgerStatus validates and transitions to CONFIRMED or FAILED with appropriate timestamp/error field updates; unit tests verify all three status paths using in-memory stubs.

Orchestrator: DB-backed Workflow Persistence

Layer / File(s) Summary
purchase_workflows migration and WorkflowSnapshot interface
database/migrations/005_purchase_workflows.sql, apps/backend/orchestrator/src/index.ts (lines 1–31)
SQL migration creates purchase_workflows with JSONB context, integer version, and updated_at; orchestrator module declares the WorkflowSnapshot interface and initializes the Postgres pool via createRequire.
persistWorkflowState, recoverWorkflowState, and onTransition hook
apps/backend/orchestrator/src/index.ts (lines 33–91)
persistWorkflowState inserts or performs optimistic-concurrency updates using context._dbVersion; recoverWorkflowState loads and injects _dbVersion; onTransition hook calls persistWorkflowState on every state machine transition.
Startup recovery, guarded execution, and tests
apps/backend/orchestrator/src/index.ts (lines 92–142), apps/backend/orchestrator/src/index.test.ts
startup() queries unfinished workflows, restores each as a machine snapshot, then starts the HTTP server; execution is guarded against test environments; tests cover insert, update, optimistic concurrency failure, and recovery success/failure paths.

Notifications: Escrow Event Listener

Layer / File(s) Summary
EscrowContractEvent interface and polling loop
apps/backend/notifications/src/escrowListener.ts (lines 1–145)
Defines EscrowContractEvent; startEscrowEventListener polls Soroban RPC every 5 seconds, tracks last processed ledger in Redis, maps raw SCValues to typed event types (using release_to_seller to differentiate released vs. refunded), and dispatches notifications; stopEscrowEventListener clears the timeout.
Idempotent notification dispatch
apps/backend/notifications/src/escrowListener.ts (lines 146–252)
dispatchEscrowNotification selects target wallet by event type, resolves user via wallet lookup adapter, and sends email and push notifications gated by Redis-backed checkAndMarkDispatched idempotency checks.
Startup wiring, tests, and docs
apps/backend/notifications/src/index.ts, apps/backend/notifications/src/escrowListener.test.ts, apps/backend/notifications/README.md
index.ts conditionally starts the listener when ESCROW_CONTRACT_ID and SOROBAN_RPC_URL are set; tests mock Redis, Stellar RPC, wallet lookup, and notification senders to assert idempotency and email dispatch; README documents required environment variables.

Migration Runner

Layer / File(s) Summary
Versioned SQL migration runner
scripts/setup/migrate.js
After the initial schema migration, the script reads database/migrations, sorts .sql files, runs them sequentially, and skips on "already exists" errors while rethrowing others.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

Possibly related PRs

  • DelegoLabs/Delego#156: Modifies apps/backend/orchestrator/src/index.ts and purchase workflow state/machine infrastructure that this PR now extends with DB persistence.
  • DelegoLabs/Delego#229: Introduces checkAndMarkDispatched Redis idempotency used directly in escrowListener.ts dispatch logic.
  • DelegoLabs/Delego#244: Adds getWalletLookupAdapter().lookupByWalletAddress(...) used by escrowListener.ts to resolve notification targets.

Poem

🐇 Hoppity-hop through the ledger I go,
Stamping each hash with a PENDING—hello!
Workflows wake up from their PostgreSQL sleep,
Escrow events polled from the blockchain so deep.
Migrations run tidy, idempotent and neat,
This bunny declares: the persistence is complete! 🌟

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR also adds unrelated notifications and orchestrator persistence changes outside the payments ledger scope. Split the escrow listener and workflow-persistence work into separate PRs or remove them from this change so the diff stays focused on the payments ledger.
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: adding a database-backed event ledger for issue #52.
Linked Issues check ✅ Passed The PR implements the #52 ledger table, helpers, tests, and docs for pending/confirmed/failed transaction tracking.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch feature/issue-52-database-backed-event-ledger

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

🧹 Nitpick comments (1)
apps/backend/notifications/src/escrowListener.test.ts (1)

64-108: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Exercise at least two polls in this test.

Because checkAndMarkDispatched is mocked to always return true and the test never asserts call counts, this only proves the happy path. It will still pass if the same event is sent again on the next poll. Drive a second poll and assert duplicate suppression so regressions in the idempotency path are caught.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/notifications/src/escrowListener.test.ts` around lines 64 - 108,
The Escrow listener test only covers a single happy-path poll, so it does not
verify idempotency across repeated event fetches. Update the test around
startEscrowEventListener and SorobanRpc.Server.prototype.getEvents to exercise
at least two polling cycles, then assert checkAndMarkDispatched is not
reprocessed for the same escrow event and that email.sendEmail is only triggered
once. Use the existing mocks and event fixture in escrowListener.test.ts to
drive the duplicate suppression path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/backend/notifications/README.md`:
- Around line 13-21: The environment variable checklist for the notifications
docs is missing the VAPID keys required by the push notification path. Update
the list in the README section that documents deployment/configuration for the
notifications service to include VAPID_PUBLIC_KEY and VAPID_PRIVATE_KEY, using
the existing env-var naming style alongside sendPushNotification and
dispatchEscrowNotification-related settings.

In `@apps/backend/notifications/src/escrowListener.ts`:
- Around line 57-67: The polling logic in escrowListener is advancing the
checkpoint too early, which can skip unprocessed events when getEvents returns a
partial batch or when parse/dispatch errors are swallowed. Update the event loop
in escrowListener to paginate through all results from server.getEvents, and
only persist latestLedger after successfully processing up to the highest
ledger/event handled without failure. Use the existing escrowListener polling
flow and the latestLedger checkpoint write to ensure the checkpoint reflects
only completed work.
- Around line 193-214: The idempotency flow in `checkAndMarkDispatched`
currently marks the notification as dispatched before `sendEmail` or
`sendPushNotification` succeeds, so failures in `escrowListener` can block all
retries. Update the `escrowListener` send paths to use a temporary in-flight
lock or to clear the Redis marker when `sendEmail`/`sendPushNotification` fails,
and only persist the final dispatched marker after a successful provider call.
Apply this to both the email and push branches so retries remain possible after
transient config/provider errors.
- Around line 47-54: The initial backfill in escrowListener’s checkpoint logic
can produce a non-positive start ledger when there is no stored checkpoint and
latestLedger is below 100. Update the startLedger initialization in
escrowListener so it is clamped to a valid minimum before the first RPC query,
while keeping the existing latestLedger upper bound check intact. Use the
existing lastProcessedKey, lastProcessedStr, and startLedger flow to make the
fix in the same block.

In `@apps/backend/orchestrator/src/index.ts`:
- Around line 95-125: The startup recovery flow in index.ts should not continue
to startHttpServer() after a recovery failure. Update the try/catch around the
pool.query and restorePurchaseWorkflow loop so that a failure either aborts
startup by rethrowing/failing fast, or you isolate row-level errors inside the
loop and only proceed when recovery completes successfully. Make the behavior
clear in the startup path around startHttpServer, getPool, and
restorePurchaseWorkflow.
- Around line 33-64: persistWorkflowState currently writes to purchase_workflows
but never updates context._dbVersion after a successful save, so callers keep
using stale version state. Update the persistWorkflowState flow to return the
new version from the INSERT/UPDATE SQL in both branches, then assign that
version back onto context._dbVersion before returning; use the existing
persistWorkflowState and expectedVersion logic to keep the optimistic
concurrency path consistent.

In `@apps/backend/payments/events/index.ts`:
- Around line 201-211: The submission write path in
SorobanTransactionLedger.findOrCreate is still race-prone because it relies on a
read-then-insert pattern. Replace this with an atomic upsert/insert-or-ignore
approach around the hash key so concurrent submissions reuse the same row
instead of colliding. Update the ledger write logic in the submission flow to
ensure idempotent behavior without primary key failures.
- Around line 252-258: The status transition logic in the ledger update path
leaves confirmedAt populated when a record moves from CONFIRMED to FAILED.
Update the conditional handling in the ledgerEntry status update block so that
when the new status is not CONFIRMED, confirmedAt is explicitly cleared along
with setting errorDetails, keeping the ledgerEntry state consistent.
- Around line 213-219: The retry path in the transaction submission update block
leaves terminal data behind when a row is moved back to PENDING. In the existing
ledgerEntry update logic inside the submission/retry flow, clear any
terminal-only fields such as confirmedAt and errorDetails whenever status is
reset to PENDING, alongside updating method/orderId/contractId. Keep the fix
localized to the same created/ledgerEntry branch so a previously CONFIRMED or
FAILED row no longer appears terminal during reconciliation.

In `@apps/backend/payments/src/db.ts`:
- Around line 11-13: The Sequelize pool config in db.ts should validate
DATABASE_POOL_MIN and DATABASE_POOL_MAX before constructing the pool settings
instead of blindly using Number(...). Add explicit parsing/validation so
non-numeric values are rejected and ensure the resolved min is not greater than
max, then fail fast with a clear config error before creating the Sequelize
instance.

In `@database/migrations/005_purchase_workflows.sql`:
- Around line 1-8: The purchase_workflows table currently allows any string in
state, but the orchestrator code in apps/backend/orchestrator/src/index.ts
treats it as PurchaseState and relies on a fixed set of valid terminal states.
Update the migration for purchase_workflows to constrain state with a CHECK
constraint or enum that matches the workflow-state contract used by
PurchaseState and the terminal-state logic, so only valid states can be
persisted or restored.

In `@scripts/setup/migrate.js`:
- Around line 34-42: The migration retry handling in migrate.js is too broad
because the catch around client.query(migrationSql) treats any "already exists"
error as a successful file-level skip, which can hide partially applied SQL
batches. Update the migration flow so duplicate-object errors are handled per
statement or otherwise detected before executing the batch, and do not mark the
whole migration file as applied when only part of it ran. Use the existing
migrate script logic and the client.query/migrationSql execution path to locate
the fix, and ensure files like 002_gateway_auth_limits.sql can be safely rerun
without suppressing later DDL.

In `@tests/unit/src/payments-ledger.test.js`:
- Around line 45-74: The payments ledger tests are order-dependent because
`updateLedgerStatus()` is exercised against state created by the previous
`logSubmission()` test. Add per-test setup/teardown in `payments-ledger.test.js`
so each `it(...)` creates its own submission record before calling
`updateLedgerStatus`, and keep the `logSubmission`/`updateLedgerStatus`
assertions isolated. Use the existing helpers directly in each test so
`CONFIRMED`, `FAILED`, and any retry-to-`PENDING` cases validate independently
of test execution order.

---

Nitpick comments:
In `@apps/backend/notifications/src/escrowListener.test.ts`:
- Around line 64-108: The Escrow listener test only covers a single happy-path
poll, so it does not verify idempotency across repeated event fetches. Update
the test around startEscrowEventListener and
SorobanRpc.Server.prototype.getEvents to exercise at least two polling cycles,
then assert checkAndMarkDispatched is not reprocessed for the same escrow event
and that email.sendEmail is only triggered once. Use the existing mocks and
event fixture in escrowListener.test.ts to drive the duplicate suppression path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 54df3ec0-e213-4954-8c80-20cd71d7b259

📥 Commits

Reviewing files that changed from the base of the PR and between 65de858 and 830d4f3.

📒 Files selected for processing (15)
  • apps/backend/notifications/README.md
  • apps/backend/notifications/src/escrowListener.test.ts
  • apps/backend/notifications/src/escrowListener.ts
  • apps/backend/notifications/src/index.ts
  • apps/backend/orchestrator/src/index.test.ts
  • apps/backend/orchestrator/src/index.ts
  • apps/backend/payments/README.md
  • apps/backend/payments/events/index.ts
  • apps/backend/payments/package.json
  • apps/backend/payments/src/db.ts
  • apps/backend/payments/src/models/SorobanTransactionLedger.ts
  • database/migrations/004_soroban_transaction_ledger.sql
  • database/migrations/005_purchase_workflows.sql
  • scripts/setup/migrate.js
  • tests/unit/src/payments-ledger.test.js

Comment on lines +13 to +21
## Environment Variables

- `ESCROW_CONTRACT_ID`: The Soroban contract ID for the escrow contract, to listen for events.
- `SOROBAN_RPC_URL`: The Stellar RPC URL to poll for on-chain events.
- `REDIS_URL`: Redis connection URL, used for worker idempotency and deduplication.
- `DATABASE_URL`: PostgreSQL connection URL, used for wallet lookup adapter.
- `SENDGRID_API_KEY`: SendGrid API key for emails.
- `FROM_EMAIL`: Sender email address.
- `LOG_LEVEL`: Log level (e.g. info, debug).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Document the VAPID keys used by the push path.

dispatchEscrowNotification can send push notifications, and sendPushNotification fails when VAPID_PUBLIC_KEY / VAPID_PRIVATE_KEY are unset. This new env-var checklist should include them.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/notifications/README.md` around lines 13 - 21, The environment
variable checklist for the notifications docs is missing the VAPID keys required
by the push notification path. Update the list in the README section that
documents deployment/configuration for the notifications service to include
VAPID_PUBLIC_KEY and VAPID_PRIVATE_KEY, using the existing env-var naming style
alongside sendPushNotification and dispatchEscrowNotification-related settings.

Comment on lines +47 to +54
const lastProcessedKey = `escrow_listener:last_ledger:${contractId}`;
const lastProcessedStr = await redis.get(lastProcessedKey);

let startLedger = lastProcessedStr ? parseInt(lastProcessedStr, 10) + 1 : latestLedger - 100;

if (startLedger > latestLedger) {
startLedger = latestLedger;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Clamp the initial backfill ledger to a valid minimum.

When there is no checkpoint and latestLedger < 100, Line 50 computes startLedger <= 0. That produces an impossible ledger range for the first RPC query on fresh/local networks.

💡 Small fix
-      let startLedger = lastProcessedStr ? parseInt(lastProcessedStr, 10) + 1 : latestLedger - 100;
+      let startLedger = lastProcessedStr
+        ? parseInt(lastProcessedStr, 10) + 1
+        : Math.max(1, latestLedger - 100);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const lastProcessedKey = `escrow_listener:last_ledger:${contractId}`;
const lastProcessedStr = await redis.get(lastProcessedKey);
let startLedger = lastProcessedStr ? parseInt(lastProcessedStr, 10) + 1 : latestLedger - 100;
if (startLedger > latestLedger) {
startLedger = latestLedger;
}
const lastProcessedKey = `escrow_listener:last_ledger:${contractId}`;
const lastProcessedStr = await redis.get(lastProcessedKey);
let startLedger = lastProcessedStr
? parseInt(lastProcessedStr, 10) + 1
: Math.max(1, latestLedger - 100);
if (startLedger > latestLedger) {
startLedger = latestLedger;
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/notifications/src/escrowListener.ts` around lines 47 - 54, The
initial backfill in escrowListener’s checkpoint logic can produce a non-positive
start ledger when there is no stored checkpoint and latestLedger is below 100.
Update the startLedger initialization in escrowListener so it is clamped to a
valid minimum before the first RPC query, while keeping the existing
latestLedger upper bound check intact. Use the existing lastProcessedKey,
lastProcessedStr, and startLedger flow to make the fix in the same block.

Comment on lines +57 to +67
const eventsResponse = await server.getEvents({
startLedger,
filters: [
{
type: "contract",
contractIds: [contractId],
topics: [["*"]], // Match any topics for this contract
},
],
limit: 1000,
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Don't checkpoint latestLedger after a partial batch.

Lines 57-67 fetch at most 1000 events, but Line 124 always stores latestLedger. Any backlog larger than that, or any parse/dispatch failure swallowed on Lines 119-121, will be skipped permanently on the next poll. Advance the checkpoint only to the highest ledger/event that finished successfully, and keep paging until the batch is exhausted.

Also applies to: 74-125

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/notifications/src/escrowListener.ts` around lines 57 - 67, The
polling logic in escrowListener is advancing the checkpoint too early, which can
skip unprocessed events when getEvents returns a partial batch or when
parse/dispatch errors are swallowed. Update the event loop in escrowListener to
paginate through all results from server.getEvents, and only persist
latestLedger after successfully processing up to the highest ledger/event
handled without failure. Use the existing escrowListener polling flow and the
latestLedger checkpoint write to ensure the checkpoint reflects only completed
work.

Comment on lines +193 to +214
const shouldSend = await checkAndMarkDispatched(redis, {
userId: target.userId,
channel: "email",
eventType: event.eventType,
eventId,
});

if (shouldSend) {
tasks.push(
sendEmail({
to: target.email,
subject,
templateName,
templateData: {
orderId: event.orderId,
amount: event.amountStroops,
},
}).catch((err) =>
log.error("Failed to send escrow email", { error: err, userId: target.userId })
)
);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Mark notifications as dispatched only after the provider call succeeds.

checkAndMarkDispatched in apps/backend/notifications/src/idempotency.ts:22-29 writes the Redis key before sendEmail / sendPushNotification run, and those helpers can fail on config/provider errors (apps/backend/notifications/email/index.ts:38-51, apps/backend/notifications/push/index.ts:25-33). A transient failure therefore suppresses every future retry for that event. Use a short-lived in-flight lock plus a separate success marker, or delete the key on send failure.

Also applies to: 219-247

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/notifications/src/escrowListener.ts` around lines 193 - 214, The
idempotency flow in `checkAndMarkDispatched` currently marks the notification as
dispatched before `sendEmail` or `sendPushNotification` succeeds, so failures in
`escrowListener` can block all retries. Update the `escrowListener` send paths
to use a temporary in-flight lock or to clear the Redis marker when
`sendEmail`/`sendPushNotification` fails, and only persist the final dispatched
marker after a successful provider call. Apply this to both the email and push
branches so retries remain possible after transient config/provider errors.

Comment on lines +33 to +64
/** Saves current machine state and context. */
export async function persistWorkflowState(
orderId: string,
state: string,
context: Record<string, any>
): Promise<void> {
const pool = getPool();
const userId = context.userId;
const expectedVersion = context._dbVersion as number | undefined;

if (expectedVersion !== undefined) {
const res = await pool.query(
`UPDATE purchase_workflows
SET state = $1, context = $2, version = version + 1, updated_at = CURRENT_TIMESTAMP
WHERE order_id = $3 AND version = $4`,
[state, JSON.stringify(context), orderId, expectedVersion]
);
if (res.rowCount === 0) {
throw new Error(`Optimistic concurrency failure: workflow ${orderId}`);
}
} else {
await pool.query(
`INSERT INTO purchase_workflows (order_id, user_id, state, context, version)
VALUES ($1, $2, $3, $4, 1)
ON CONFLICT (order_id) DO UPDATE SET
state = EXCLUDED.state,
context = EXCLUDED.context,
version = purchase_workflows.version + 1,
updated_at = CURRENT_TIMESTAMP`,
[orderId, userId, state, JSON.stringify(context)]
);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Write the new DB version back into context.

Line 89 reuses the live record.context, but this function never updates _dbVersion after a successful write. Fresh workflows therefore stay on the unguarded UPSERT path forever, and recovered workflows will false-conflict on their second persisted transition because they still send the old version. Return the new version from SQL and assign it back to context._dbVersion.

Suggested fix
 export async function persistWorkflowState(
   orderId: string,
   state: string,
   context: Record<string, any>
 ): Promise<void> {
   const pool = getPool();
   const userId = context.userId;
   const expectedVersion = context._dbVersion as number | undefined;
+  let res;

   if (expectedVersion !== undefined) {
-    const res = await pool.query(
+    res = await pool.query(
       `UPDATE purchase_workflows
        SET state = $1, context = $2, version = version + 1, updated_at = CURRENT_TIMESTAMP
-       WHERE order_id = $3 AND version = $4`,
+       WHERE order_id = $3 AND version = $4
+       RETURNING version`,
       [state, JSON.stringify(context), orderId, expectedVersion]
     );
     if (res.rowCount === 0) {
       throw new Error(`Optimistic concurrency failure: workflow ${orderId}`);
     }
   } else {
-    await pool.query(
+    res = await pool.query(
       `INSERT INTO purchase_workflows (order_id, user_id, state, context, version)
        VALUES ($1, $2, $3, $4, 1)
        ON CONFLICT (order_id) DO UPDATE SET
          state = EXCLUDED.state,
          context = EXCLUDED.context,
          version = purchase_workflows.version + 1,
-         updated_at = CURRENT_TIMESTAMP`,
+         updated_at = CURRENT_TIMESTAMP
+       RETURNING version`,
       [orderId, userId, state, JSON.stringify(context)]
     );
   }
+
+  context._dbVersion = res.rows[0].version;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/** Saves current machine state and context. */
export async function persistWorkflowState(
orderId: string,
state: string,
context: Record<string, any>
): Promise<void> {
const pool = getPool();
const userId = context.userId;
const expectedVersion = context._dbVersion as number | undefined;
if (expectedVersion !== undefined) {
const res = await pool.query(
`UPDATE purchase_workflows
SET state = $1, context = $2, version = version + 1, updated_at = CURRENT_TIMESTAMP
WHERE order_id = $3 AND version = $4`,
[state, JSON.stringify(context), orderId, expectedVersion]
);
if (res.rowCount === 0) {
throw new Error(`Optimistic concurrency failure: workflow ${orderId}`);
}
} else {
await pool.query(
`INSERT INTO purchase_workflows (order_id, user_id, state, context, version)
VALUES ($1, $2, $3, $4, 1)
ON CONFLICT (order_id) DO UPDATE SET
state = EXCLUDED.state,
context = EXCLUDED.context,
version = purchase_workflows.version + 1,
updated_at = CURRENT_TIMESTAMP`,
[orderId, userId, state, JSON.stringify(context)]
);
}
/** Saves current machine state and context. */
export async function persistWorkflowState(
orderId: string,
state: string,
context: Record<string, any>
): Promise<void> {
const pool = getPool();
const userId = context.userId;
const expectedVersion = context._dbVersion as number | undefined;
let res;
if (expectedVersion !== undefined) {
res = await pool.query(
`UPDATE purchase_workflows
SET state = $1, context = $2, version = version + 1, updated_at = CURRENT_TIMESTAMP
WHERE order_id = $3 AND version = $4
RETURNING version`,
[state, JSON.stringify(context), orderId, expectedVersion]
);
if (res.rowCount === 0) {
throw new Error(`Optimistic concurrency failure: workflow ${orderId}`);
}
} else {
res = await pool.query(
`INSERT INTO purchase_workflows (order_id, user_id, state, context, version)
VALUES ($1, $2, $3, $4, 1)
ON CONFLICT (order_id) DO UPDATE SET
state = EXCLUDED.state,
context = EXCLUDED.context,
version = purchase_workflows.version + 1,
updated_at = CURRENT_TIMESTAMP
RETURNING version`,
[orderId, userId, state, JSON.stringify(context)]
);
}
context._dbVersion = res.rows[0].version;
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/orchestrator/src/index.ts` around lines 33 - 64,
persistWorkflowState currently writes to purchase_workflows but never updates
context._dbVersion after a successful save, so callers keep using stale version
state. Update the persistWorkflowState flow to return the new version from the
INSERT/UPDATE SQL in both branches, then assign that version back onto
context._dbVersion before returning; use the existing persistWorkflowState and
expectedVersion logic to keep the optimistic concurrency path consistent.

Comment on lines +252 to +258
ledgerEntry.status = status;
if (status === "CONFIRMED") {
ledgerEntry.confirmedAt = new Date();
ledgerEntry.errorDetails = null;
} else {
ledgerEntry.errorDetails = error || "Transaction failed";
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

FAILED transitions should null out confirmedAt.

If a record is ever corrected from CONFIRMED to FAILED, this leaves both states set at once. The row should not retain a confirmation timestamp after entering the failure state.

Suggested fix
     if (status === "CONFIRMED") {
       ledgerEntry.confirmedAt = new Date();
       ledgerEntry.errorDetails = null;
     } else {
+      ledgerEntry.confirmedAt = null;
       ledgerEntry.errorDetails = error || "Transaction failed";
     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ledgerEntry.status = status;
if (status === "CONFIRMED") {
ledgerEntry.confirmedAt = new Date();
ledgerEntry.errorDetails = null;
} else {
ledgerEntry.errorDetails = error || "Transaction failed";
}
ledgerEntry.status = status;
if (status === "CONFIRMED") {
ledgerEntry.confirmedAt = new Date();
ledgerEntry.errorDetails = null;
} else {
ledgerEntry.confirmedAt = null;
ledgerEntry.errorDetails = error || "Transaction failed";
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/payments/events/index.ts` around lines 252 - 258, The status
transition logic in the ledger update path leaves confirmedAt populated when a
record moves from CONFIRMED to FAILED. Update the conditional handling in the
ledgerEntry status update block so that when the new status is not CONFIRMED,
confirmedAt is explicitly cleared along with setting errorDetails, keeping the
ledgerEntry state consistent.

Comment on lines +11 to +13
pool: {
min: Number(process.env.DATABASE_POOL_MIN ?? 2),
max: Number(process.env.DATABASE_POOL_MAX ?? 10),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Validate the pool env vars before constructing Sequelize.

Number(...) will happily turn a typo like "ten" into NaN, and this also accepts DATABASE_POOL_MIN > DATABASE_POOL_MAX. That pushes a simple config error into harder-to-diagnose pool failures at startup/runtime.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/backend/payments/src/db.ts` around lines 11 - 13, The Sequelize pool
config in db.ts should validate DATABASE_POOL_MIN and DATABASE_POOL_MAX before
constructing the pool settings instead of blindly using Number(...). Add
explicit parsing/validation so non-numeric values are rejected and ensure the
resolved min is not greater than max, then fail fast with a clear config error
before creating the Sequelize instance.

Comment on lines +1 to +8
CREATE TABLE purchase_workflows (
order_id VARCHAR(255) PRIMARY KEY,
user_id UUID NOT NULL,
state VARCHAR(100) NOT NULL,
context JSONB NOT NULL,
version INTEGER NOT NULL DEFAULT 1,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Constrain state to the workflow-state contract.

Line 4 currently accepts any string, but Line 107 in apps/backend/orchestrator/src/index.ts later casts this column straight to PurchaseState, and Line 100 hard-codes terminal-state names. One typo or out-of-band update can persist a value the recovery path cannot safely restore. Add a CHECK/enum that matches the orchestrator's valid states.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@database/migrations/005_purchase_workflows.sql` around lines 1 - 8, The
purchase_workflows table currently allows any string in state, but the
orchestrator code in apps/backend/orchestrator/src/index.ts treats it as
PurchaseState and relies on a fixed set of valid terminal states. Update the
migration for purchase_workflows to constrain state with a CHECK constraint or
enum that matches the workflow-state contract used by PurchaseState and the
terminal-state logic, so only valid states can be persisted or restored.

Comment thread scripts/setup/migrate.js
Comment on lines +34 to +42
try {
await client.query(migrationSql);
console.log(`[delego] db:migrate — migration ${file} applied successfully.`);
} catch (err) {
if (err.message && err.message.includes("already exists")) {
console.log(`[delego] db:migrate — warning: relation or type in ${file} already exists, skipping...`);
} else {
throw err;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Don't treat one duplicate-object error as success for the whole migration file.

client.query(migrationSql) executes the entire .sql file as a batch. If a later statement fails with "already exists", earlier statements in that same file may already have been committed, and this catch then suppresses the rest forever. database/migrations/002_gateway_auth_limits.sql already has that shape: users_email_key can fail after earlier ALTER TABLE statements, which means later DDL in that file may never run on rerun.

💡 Safer direction
+    await client.query(`
+      CREATE TABLE IF NOT EXISTS schema_migrations (
+        name TEXT PRIMARY KEY,
+        applied_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
+      )
+    `);
+
      for (const file of files) {
+        const applied = await client.query(
+          "SELECT 1 FROM schema_migrations WHERE name = $1",
+          [file]
+        );
+        if (applied.rowCount > 0) continue;
+
         console.log(`[delego] db:migrate — running migration: ${file}...`);
         const migrationPath = path.join(migrationsDir, file);
         const migrationSql = fs.readFileSync(migrationPath, "utf8");
         
+        await client.query("BEGIN");
         try {
           await client.query(migrationSql);
+          await client.query(
+            "INSERT INTO schema_migrations(name) VALUES ($1)",
+            [file]
+          );
+          await client.query("COMMIT");
           console.log(`[delego] db:migrate — migration ${file} applied successfully.`);
         } catch (err) {
-          if (err.message && err.message.includes("already exists")) {
-            console.log(`[delego] db:migrate — warning: relation or type in ${file} already exists, skipping...`);
-          } else {
-            throw err;
-          }
+          await client.query("ROLLBACK");
+          throw err;
         }
       }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
try {
await client.query(migrationSql);
console.log(`[delego] db:migrate — migration ${file} applied successfully.`);
} catch (err) {
if (err.message && err.message.includes("already exists")) {
console.log(`[delego] db:migrate — warning: relation or type in ${file} already exists, skipping...`);
} else {
throw err;
}
await client.query("BEGIN");
try {
await client.query(migrationSql);
await client.query(
"INSERT INTO schema_migrations(name) VALUES ($1)",
[file]
);
await client.query("COMMIT");
console.log(`[delego] db:migrate — migration ${file} applied successfully.`);
} catch (err) {
await client.query("ROLLBACK");
throw err;
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/setup/migrate.js` around lines 34 - 42, The migration retry handling
in migrate.js is too broad because the catch around client.query(migrationSql)
treats any "already exists" error as a successful file-level skip, which can
hide partially applied SQL batches. Update the migration flow so
duplicate-object errors are handled per statement or otherwise detected before
executing the batch, and do not mark the whole migration file as applied when
only part of it ran. Use the existing migrate script logic and the
client.query/migrationSql execution path to locate the fix, and ensure files
like 002_gateway_auth_limits.sql can be safely rerun without suppressing later
DDL.

Comment on lines +45 to +74
it("should log transaction submission with PENDING status", async () => {
const hash = "0000000000000000000000000000000000000000000000000000000000000001";
const method = "create_escrow";
const orderId = "order_123";
const contractId = "contract_abc";

const entry = await logSubmission(hash, method, orderId, contractId);
assert.equal(entry.hash, hash);
assert.equal(entry.method, method);
assert.equal(entry.status, "PENDING");
assert.equal(entry.orderId, orderId);
assert.equal(entry.contractId, contractId);
});

it("should update ledger status to CONFIRMED", async () => {
const hash = "0000000000000000000000000000000000000000000000000000000000000001";

const entry = await updateLedgerStatus(hash, "CONFIRMED");
assert.equal(entry.status, "CONFIRMED");
assert.ok(entry.confirmedAt);
assert.equal(entry.errorDetails, null);
});

it("should update ledger status to FAILED with error details", async () => {
const hash = "0000000000000000000000000000000000000000000000000000000000000001";

const entry = await updateLedgerStatus(hash, "FAILED", "Simulation failed");
assert.equal(entry.status, "FAILED");
assert.equal(entry.errorDetails, "Simulation failed");
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make these tests independent.

The CONFIRMED and FAILED cases depend on the first test having already created the row, so the suite becomes order-dependent and they stop validating updateLedgerStatus() in isolation. Per-test setup would also make it easy to cover the retry-to-PENDING path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/src/payments-ledger.test.js` around lines 45 - 74, The payments
ledger tests are order-dependent because `updateLedgerStatus()` is exercised
against state created by the previous `logSubmission()` test. Add per-test
setup/teardown in `payments-ledger.test.js` so each `it(...)` creates its own
submission record before calling `updateLedgerStatus`, and keep the
`logSubmission`/`updateLedgerStatus` assertions isolated. Use the existing
helpers directly in each test so `CONFIRMED`, `FAILED`, and any
retry-to-`PENDING` cases validate independently of test execution order.

@ScriptedBro

Copy link
Copy Markdown
Contributor

Fix the conflicts

@ScriptedBro

Copy link
Copy Markdown
Contributor

Please fix the conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Payments] Implement Database-Backed Event Ledger for Off-Chain Transaction Sync

2 participants