Skip to content

Webhook receiver, job queue, SQL views, and API scaffold#2

Merged
anderdc merged 6 commits into
mainfrom
test
Apr 15, 2026
Merged

Webhook receiver, job queue, SQL views, and API scaffold#2
anderdc merged 6 commits into
mainfrom
test

Conversation

@anderdc
Copy link
Copy Markdown
Collaborator

@anderdc anderdc commented Apr 15, 2026

Summary

  • Webhook receiverPOST /webhooks/github with HMAC signature verification, dedup, and 7 event handlers (PRs, issues, reviews, comments, review comments, labels, installations)
  • BullMQ job queue — Redis-backed queue for GitHub API fetch jobs (closing issues via GraphQL, PR files via REST). Concurrency-limited, retries with exponential backoff, dedup by job ID.
  • SQL viewscontributor_repo_roles, pr_review_summary, pr_linked_issues, pr_scoring_inputs (no materialized views — counts are live with validator-supplied lookback)
  • First API endpointGET /api/v1/contributors/:githubId/scoring-inputs and /counts?days=N with Swagger at /swagger
  • Schema updates — issues: title, state_reason, labels, solved_by_pr. PRs: labels. label_events: target_number + target_type. reviews: reviewer_association.
  • Infrastructure — Redis in docker-compose, Postgres on port 5434 to avoid conflicts, DESIGN.md updated with architecture docs

Test plan

  • tsc --noEmit passes
  • eslint passes
  • prettier --check passes
  • jest passes
  • CI pipeline green
  • Deploy to droplet, verify webhook endpoint accepts test payload
  • Create GitHub App, install on a test repo, verify data flows into DB

anderdc added 6 commits April 14, 2026 19:14
- Create 10 entity files matching DB schemas (data mapper pattern, following das-gittensor conventions)
- Add missing scoring-critical columns: title, closed_at, last_edited_at (pull_requests), changes (pr_files), reviewer_association (reviews)
- Update DESIGN.md: resolve closing_issue_numbers extraction (GraphQL), add scope constraints (public-only, no precomputed AST), update pr_review_summary with maintainer-only filter, expand pr_scoring_inputs view with all scoring fields and edited_after_merge flag
- Remove stale __tests__/ directory (spec already colocated)
…I endpoint

Webhook receiver:
- POST /webhooks/github with HMAC-SHA256 signature verification
- Dedup via webhook_deliveries table (X-GitHub-Delivery header)
- Event handlers for: pull_request, issues, reviews, comments,
  review_comments, labels, installation/uninstallation
- Labels captured on issues and PRs (current snapshot + append-only history)

Job queue (BullMQ + Redis):
- Webhook handlers enqueue fetch jobs instead of calling GitHub API inline
- Worker processes fetch-closing-issues (GraphQL) and fetch-pr-files (REST)
- Concurrency-limited, retry with exponential backoff, dedup by job ID
- Sets solved_by_pr on issues when merged PR closes them

SQL views:
- view_contributor_repo_roles: latest association per contributor per repo
- view_pr_review_summary: review counts with maintainer-only filter
- view_pr_linked_issues: PR × closing issues join with full issue data
- view_pr_scoring_inputs: main scoring view (PR + reviews, no baked-in time window)

API (first endpoint + Swagger):
- GET /api/v1/contributors/:githubId/scoring-inputs
- GET /api/v1/contributors/:githubId/counts?days=N
- Swagger UI at /swagger

Schema updates:
- issues: add title, state_reason, labels, solved_by_pr
- pull_requests: add labels
- label_events: rename issue_number→target_number, add target_type
- Entity nullable types fixed (null vs undefined for TypeORM)

Infrastructure:
- Redis added to docker-compose
- Postgres mapped to port 5434 (avoids conflict with other services)
- DESIGN.md updated: architecture docs, dropped materialized views in favor
  of live aggregation with validator-supplied lookback window
- Postgres and Redis healthchecks in docker-compose; das waits for both
  to be healthy before starting
- Number-prefix SQL files (01-10 tables, 20-23 views) for deterministic
  init order in docker-entrypoint-initdb.d
- Add .dockerignore to exclude node_modules/dist from build context
- Align API_PORT to 6969 across .env.example and Dockerfile
Drop spec files, jest config, test deps (@nestjs/testing, jest, ts-jest),
coverage thresholds, and test CI workflow. Lint glob simplified to src only.
@anderdc anderdc merged commit 5dee8ac into main Apr 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant