Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,27 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht

## [Unreleased]

### Added

### Changed

### Fixed

### Removed

### Security

## [0.4.0] - 2026-07-29

Reliability and security release. A full audit closed the delivery-correctness and API defects behind double delivery under stale-lock recovery, silent message loss on a failed fan-out enqueue, a racy manual retry, a circuit-open endpoint burning a message's retry budget, unbounded list pagination, and a dashboard whose live counters silently froze after 20 events — plus three security fixes: an unauthenticated SignalR hub leaking cross-tenant delivery data, connect-time IP-allowlist enforcement against DNS rebinding, and login rate limiting. Adds the operator backlog metrics an alert can actually fire on, and repairs three query paths that degraded as `messages` and `message_attempts` grew. No breaking API changes; the `v1` prefix, the Standard Webhooks signature surface, and the `WebhookEngine.Sdk` public API are unchanged.

### Added
- **Operator backlog metrics an alert can actually fire on.** Three new Prometheus gauges expose the real delivery backlog: `webhookengine_queue_depth` (messages `Pending` and due for delivery), `webhookengine_queue_oldest_pending_age` (seconds since the oldest due message was scheduled — head-of-line latency, the real SLO signal), and `webhookengine_circuit_open` (endpoints whose circuit breaker is currently Open). A background collector refreshes them from the database every 15s so the scrape path stays cheap.

### Changed
- **Backend dependency refresh.** The .NET 10 runtime stack moves `10.0.9 → 10.0.10` (EF Core, EF Core Relational/Design, `Microsoft.Extensions.Http`/`Hosting`/`DependencyInjection`, `Microsoft.AspNetCore.OpenApi`, `Microsoft.AspNetCore.Mvc.Testing`), OpenTelemetry `1.16.0 → 1.17.0` (OTLP exporter, hosting extensions, ASP.NET Core and runtime instrumentation), `Scalar.AspNetCore` `2.16.11 → 2.16.16`, and `System.IdentityModel.Tokens.Jwt` `8.19.1 → 8.21.0`. Test-only: `Microsoft.NET.Test.Sdk` `18.7.0 → 18.8.1` and `NSubstitute` `5.3.0 → 6.0.0` (major; the substitute APIs the suite uses are unchanged and the full suite passes). No user-facing behaviour change.
- **Frontend dependency refresh.** Shipping in the dashboard bundle: `react-router` `8.2.0 → 8.3.0`, `lucide-react` `1.24.0 → 1.27.0`, `recharts` `3.9.2 → 3.10.1`, and `tailwindcss` / `@tailwindcss/vite` `4.3.2 → 4.3.3`. Build and lint tooling: `vite` `8.1.4 → 8.1.5`, `@vitejs/plugin-react` `6.0.3 → 6.0.4`, `eslint` `10.7.0 → 10.8.0`, and `typescript-eslint` `8.63.0 → 8.65.0` across the dashboard and `@webhookengine/endpoint-manager`, plus `@testing-library/jest-dom` `6.10.0 → 7.0.0` (dev-only) in the package. TypeScript stays on 6.0.3 — the 7.0 line crashes `@typescript-eslint/typescript-estree`, so the bump waits on upstream support. No user-facing behaviour change.
- **CI and container base images refreshed.** `actions/setup-node` `6 → 7` and `actions/setup-dotnet` `5 → 6` in the workflows; the `mcr.microsoft.com/dotnet/sdk:10.0` digest is repinned `ea8bde3 → ed034a8` (build stage only) and the runtime `mcr.microsoft.com/dotnet/aspnet:10.0-alpine` digest `57bd717 → 27b6b84`.

### Fixed
- **Query performance on the growth paths: two indexes and a streaming p95.** As `messages` and `message_attempts` grow into the millions, three hot paths degraded. (1) The retry sweep (`RetryScheduler`, every 10s) filters `status='Failed' AND scheduled_at <= now` across all applications, which the app-scoped `idx_messages_app_status` cannot serve (its leading column is `app_id`, and PostgreSQL 17 has no B-tree skip scan) — it was a sequential scan of the whole table every tick. A new partial index `idx_messages_retry` on `scheduled_at WHERE status='Failed'` serves it directly. (2) The dashboard timeline query scans `message_attempts` by `created_at` alone, which no existing index covered; a new `idx_message_attempts_created_at` serves it. (3) The per-endpoint stats route computed p95 latency by loading every attempt's latency for the period into memory and indexing the sorted list — an out-of-memory vector on a busy endpoint over a 30-day window, reachable with an ordinary API key. It now computes count, success, average, and p95 in a single aggregate query using `percentile_cont(0.95)`, so nothing is materialized. The stats response shape is unchanged; p95 is now a continuous-interpolated percentile (the standard definition) rather than the previous nearest-rank sample, so the value can differ slightly for the same data.
Expand All @@ -21,8 +38,6 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
- **Delivery lifecycle hardened against double-delivery, lost attempt rows, and state regression.** Three intertwined worker defects that broke the "at-least-once, no-double-delivery, crash-safe" contract are now closed. (1) A batch of messages received a single `locked_at` at dequeue, so the tail of a slow sequential batch could cross the stale-lock threshold while a live worker was still delivering it — `StaleLockRecoveryWorker` would reclaim it and a second worker would redeliver. `DeliveryWorker` now re-asserts ownership per message via a CAS `RefreshLockAsync` (only the current owner of a still-`Sending` row keeps the lease); a lost lease skips delivery instead of duplicating it. (2) The graceful-shutdown token was threaded into both the HTTP POST and the follow-on DB writes, so on SIGTERM an in-flight delivery was aborted and mislabeled a timeout while its attempt row and terminal status were never persisted — leaving the row stuck `Sending` for a later redelivery. Delivery now runs on a per-request timeout independent of shutdown, and every attempt/terminal write commits with `CancellationToken.None`, so an in-flight message finalizes cleanly inside the shutdown grace window. (3) Error recovery reset a message to `Pending` from stale in-memory status, which after a stale-lock steal could regress a `Delivered`/`DeadLetter` row back to `Pending`; the reset is now a CAS `ResetToPendingIfOwnedAsync` that only fires when the worker still owns a `Sending` row. A fail-fast `DeliveryOptions` validator (`ValidateOnStart`) now rejects a configuration where `TimeoutSeconds * 1.5 >= StaleLockMinutes * 60` (or any non-positive core knob), so an operator cannot silently re-open the stale-steal window by mistuning timeouts. Covered by real-PostgreSQL (Testcontainers) lease-ownership and stale-steal regressions, a full `MessageStateMachine` transition-matrix suite, and validator unit tests.
- **A failed enqueue no longer silently drops a sibling message (poisoned-`DbContext` fix).** When a webhook was fanned out to several subscribed endpoints, `PostgresMessageQueue.EnqueueAsync` added the `Message` and saved it per endpoint on one request-scoped `DbContext`. If a save threw — typically the idempotency partial-unique index raising `23505` — the failed entity stayed tracked in `Added` state, so the next endpoint's `SaveChanges` re-flushed it, threw again, rolled back the whole batch, and the controller's idempotency-conflict catch swallowed it: the API returned success while that later endpoint never received the webhook. `EnqueueAsync` now detaches the entity when the save fails (rethrowing the original exception, so idempotency replay still works) — leaving the context clean for the next iteration. A real-PostgreSQL (Testcontainers) regression asserts a distinct message still persists after a `23505` on the same context.

### Removed

### Security
- **Per-endpoint IP allowlist is now enforced at connect time (DNS-rebinding hardening).** The allowlist was checked in the worker against its own DNS resolution, but the actual delivery connection re-resolved the host in the `webhook-delivery` client's connect callback and pinned to *that* result — checking only the private-IP SSRF guard, not the allowlist. A hostile or compromised authoritative DNS could return an allowlisted IP to the worker's check and a different, non-allowlisted IP to the connection, defeating the static-egress guarantee that enterprise/fintech deployments rely on. The connect callback now enforces the endpoint's allowlist on the same resolution it pins to (the allowlist is threaded to it via the request), so a resolved address outside the allowlist refuses the connection. The worker's pre-resolve check is kept as an early, clean rejection for a genuinely-misconfigured endpoint; the connect-time check is the authoritative guard.
- **The dashboard login endpoint is now rate-limited.** `POST /api/v1/auth/login` had no throttle or lockout, so the seeded admin credentials could be brute-forced online (PBKDF2's 100k iterations cap hashing cost but not attempt count) while every other route was rate-limited. Login now carries a fixed-window limiter (5 attempts per minute per client IP by default, configurable under `WebhookEngine:LoginRateLimit`), reusing the existing `RATE_LIMIT_EXCEEDED` 429 envelope + `Retry-After`. `Logout` and `Me` are unaffected.
Expand Down
4 changes: 2 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# WebhookEngine — Strategic Roadmap

**Last Updated:** 2026-05-29
**Last Updated:** 2026-07-29
**Status:** Active — Phase 2 (Traction & Feedback)
**Latest release:** v0.2.2 (2026-05-29) — maintenance patch (dependency refresh + CI Bun/lockfile fix). The embeddable customer portal landed in v0.2.0 (B1 Steps 1-5: `/api/v1/portal/*` surface, HS256 JWT middleware, per-app dynamic CORS, operator dashboard controls, `<PortalAccessModal />`); v0.2.1 hardened it (portal CORS deny-cache, PUT→PATCH, validator consolidation); and the `@webhookengine/endpoint-manager` npm package shipped in the `portal-v0.1.0` tag (2026-05-11).
**Latest release:** v0.4.0 (2026-07-29) — reliability and security release (delivery-correctness audit fixes, authenticated SignalR hub, connect-time IP allowlist, login rate limiting, operator backlog metrics). v0.3.0 (2026-06-08) closed the SDK coverage and portal-client drift gaps; v0.3.1 (2026-06-29) was a container CVE refresh. The embeddable customer portal landed in v0.2.0 (B1 Steps 1-5: `/api/v1/portal/*` surface, HS256 JWT middleware, per-app dynamic CORS, operator dashboard controls, `<PortalAccessModal />`); v0.2.1 hardened it (portal CORS deny-cache, PUT→PATCH, validator consolidation); and the `@webhookengine/endpoint-manager` npm package shipped in the `portal-v0.1.0` tag (2026-05-11).

> **Note:** Phase 1 is complete (launch posts and the engineering blog post remain deferred). Phase 2 core tasks (2.2, 2.3, 2.4, **2.5 payload transformation across all three rollout phases**, 2.7 application layer cleanup) are done. v0.1.6 added eight follow-on features (endpoint test webhook, validate-time URL guard, per-resource overrides, IP allowlist, audit log, SignalR endpoint health, TanStack Query data layer). v0.2.0 adds the embeddable customer portal (engine + dashboard half). The `@webhookengine/endpoint-manager` npm package (B1 Step 7 / Step 11) shipped in the `portal-v0.1.0` tag (2026-05-11). Remaining Phase 2 items (TypeScript SDK gated on demand signal, comparison / best-practice blog posts) are planned.

Expand Down
2 changes: 1 addition & 1 deletion src/WebhookEngine.Sdk/WebhookEngine.Sdk.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- NuGet package metadata -->
<PackageId>WebhookEngine.Sdk</PackageId>
<Version>0.3.1</Version>
<Version>0.4.0</Version>
<Authors>WebhookEngine</Authors>
<Description>.NET SDK for WebhookEngine — self-hosted webhook delivery platform. Send webhooks, manage endpoints and event types, retry failed deliveries.</Description>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
Expand Down
Loading