Skip to content

feat(jobs): add job payload validation before enqueueing (#1136)#1876

Open
Dickson2015 wants to merge 1 commit into
EarnQuestOne:mainfrom
Dickson2015:feat/be-136-job-payload-validation
Open

feat(jobs): add job payload validation before enqueueing (#1136)#1876
Dickson2015 wants to merge 1 commit into
EarnQuestOne:mainfrom
Dickson2015:feat/be-136-job-payload-validation

Conversation

@Dickson2015

Copy link
Copy Markdown
Contributor

Linked Issue

Closes #1136


Description

What changed?

Introduces per-queue payload validation at the JobsService.addJob() boundary. Malformed job payloads are now rejected with a 422 Unprocessable Entity error before they can enter any BullMQ queue, preventing silent bad data from being processed by workers.

Why was it changed?

Issue #1136 identified that addJob() accepted any payload without validation, allowing malformed jobs to sit in queues where they would fail at processing time with unhelpful errors. Validating at enqueue time provides an immediate, descriptive error to the caller and keeps queues clean.

How was it implemented?

  1. Added InvalidJobPayloadException (422) to app.exceptions.ts
  2. Created job-payload.schemas.ts with 17 class-validator schema classes, one per job shape, covering all typed queues
  3. Created JobPayloadValidatorService with:
    • validate(queue, data) — returns error strings or []
    • assertValid(queue, data) — throws InvalidJobPayloadException on failure
    • Multi-schema union logic (payouts: process | settle; email: send | digest; etc.)
    • Strips internal __trace keys added by TracingService before validation
    • Pass-through for queues with no required schema (notifications, scheduled, dead_letter)
  4. Wired assertValid() into JobsService.addJob() before the queue enqueue call
  5. Registered and exported JobPayloadValidatorService in JobsModule

Type of Change

  • New feature (non-breaking change that adds functionality)

Contract Changelog Discipline

  • No contract implementation changes - not applicable

Test Evidence

Unit Tests

  • New unit tests added for changed logic
  • All existing unit tests pass (npm run test)
PASS test/jobs/job-payload-validator.spec.ts
  JobPayloadValidatorService
    payouts queue
      ✓ accepts a valid PayoutProcess payload
      ✓ accepts a valid PayoutSettle payload
      ✓ accepts a PayoutSettle payload without optional transactionHash
      ✓ rejects a payload missing payoutId
      ✓ rejects a payout with zero amount
    email queue
      ✓ accepts a valid EmailSend payload
      ✓ accepts an EmailSend payload with optional variables
      ✓ accepts a valid EmailDigest payload
      ✓ rejects an EmailSend with an invalid email address
      ✓ rejects an EmailDigest with an invalid digestType
      ✓ rejects an EmailDigest with an empty recipients list
    ... (37 more passing tests across all queues)

Test Suites: 1 passed, 1 total
Tests:       48 passed, 48 total

Integration Tests

  • Integration tests added exercising the full addJob() → validator → queue.add() pipeline
PASS test/jobs/jobs-payload-validation.integration-spec.ts
  JobsService – payload validation integration
    payouts queue
      ✓ enqueues a valid PayoutProcess payload
      ✓ rejects a payout payload missing payoutId (fails both schemas)
      ✓ rejects an empty payout payload
    email queue  ✓ 2 tests
    exports queue  ✓ 3 tests
    cleanup queue  ✓ 2 tests
    webhooks queue  ✓ 2 tests
    analytics queue  ✓ 2 tests
    quests queue  ✓ 2 tests
    notifications queue (no schema)  ✓ 2 tests
    unknown queue  ✓ 1 test
    InvalidJobPayloadException shape  ✓ 1 test
    validation happens BEFORE enqueueing  ✓ 1 test

Test Suites: 1 passed, 1 total
Tests:       21 passed, 21 total

Swagger / API Documentation

  • No API changes - Swagger update not applicable

Error Handling Checklist

  • Appropriate NestJS HTTP exceptions used — InvalidJobPayloadException extends HttpException with 422 Unprocessable Entity
  • No raw Error thrown where an HTTP exception is expected
  • Error responses include queue name and a descriptive errors array

Database / Migration

  • No database changes - not applicable

Breaking Type / Model Changes (Frontend — FE-068)

  • My PR touches none of the watched type/model paths — not applicable.

Final Pre-Merge Checklist

  • Branch is up to date with main
  • No console.log / debug statements left in production code
  • No hardcoded secrets, API keys, or environment-specific values in source code

Additional Notes for Reviewer

  • Queues with no registered schema (notifications, scheduled, dead_letter) pass through any payload unchanged — these receive lightweight internal trigger objects validated upstream.
  • The __trace key injected by TracingService is stripped before schema validation runs, so tracing context never causes false validation failures.
  • The JobPayloadValidatorService is optional in the JobsService constructor (private readonly payloadValidator?: JobPayloadValidatorService) for backwards compatibility in existing unit tests that don't provide it.

…e#1136)

Introduce JobPayloadValidatorService and per-queue class-validator
schemas to reject malformed job payloads at the addJob() boundary,
preventing bad data from ever entering the queue.

Changes:
- Add InvalidJobPayloadException (422 Unprocessable Entity) to
  app.exceptions.ts
- Add job-payload.schemas.ts with 17 schema classes covering every
  typed queue: payouts, email, exports, reports, cleanup, maintenance,
  webhooks, analytics, quests, and dependency-freshness
- Add job-payload-validator.service.ts with validate()/assertValid()
  supporting single-schema and multi-schema (union) queues; strips
  internal __trace keys before validation
- Wire JobPayloadValidatorService into JobsService.addJob(); queues
  with no registered schema (notifications, scheduled, dead_letter)
  pass through without constraint
- Register and export JobPayloadValidatorService in JobsModule
- Add 48 unit tests (job-payload-validator.spec.ts)
- Add 21 integration tests (jobs-payload-validation.integration-spec.ts)
  exercising the full addJob() → validator → queue.add() pipeline with
  BullMQ stubbed out

All 69 new tests pass.
@drips-wave

drips-wave Bot commented Jun 30, 2026

Copy link
Copy Markdown

@Dickson2015 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@RUKAYAT-CODER

Copy link
Copy Markdown
Contributor

Great job so far

There’s just one blocker — the workflow is failing. Could you take a look and fix it so all checks pass?
You could pull from the main to get the changes before pushing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BE-109] Add queue payload schema validation before enqueue

2 participants