Event-Driven Order Processing System

Event-driven order processing pipeline in Go with Kafka, PostgreSQL, Docker, and Kubernetes.

Stack

• Go — services and load generator
• Kafka — message bus (via segmentio/kafka-go)
• PostgreSQL — orders, outbox, idempotency ledger
• Docker / Docker Compose — local dev
• Kubernetes — kind locally, EKS-ready manifests

Services

Service	Purpose
order-api	HTTP intake. Writes the order and its event to Postgres in one transaction (outbox).
outbox-relay	Polls the outbox, publishes events to Kafka, marks rows published.
order-processor	Consumes orders.created, marks orders PROCESSED. Idempotent.
payment-processor	Consumes orders.created, writes payment records. Idempotent.

Key patterns

• Transactional outbox — exactly-once publish across the DB ↔ Kafka boundary
• Idempotent consumers — consumed_events ledger turns at-least-once delivery into effectively-once side effects
• Per-partition ordering — events keyed by order_id

Quickstart

make up           # bring up postgres + kafka + services
make topics       # create Kafka topics
make smoke        # send 100 orders, verify all PROCESSED
make load RPS=5000 DUR=60s   # load test

Kubernetes (kind)

make kind-up
# build & load images
docker build -t order-api:dev         -f services/order-api/Dockerfile .
docker build -t outbox-relay:dev      -f services/outbox-relay/Dockerfile .
docker build -t order-processor:dev   -f services/order-processor/Dockerfile .
docker build -t payment-processor:dev -f services/payment-processor/Dockerfile .
kind load docker-image --name order-pipeline \
  order-api:dev outbox-relay:dev order-processor:dev payment-processor:dev
make k8s-deploy

Results

Sustained 5K+ events/sec on a 3-node cluster with zero data loss. See loadtest/results.md.

Layout

services/   order-api, outbox-relay, order-processor, payment-processor
pkg/        db, kafkax, outbox, idempotency, observability, shutdown
migrations/ versioned SQL (golang-migrate compatible)
deploy/k8s/ kind config + manifests
loadtest/   loadgen + scenarios + results