Skip to content

feat: bot-blocker feature set (PoW difficulty, rate-limits/bans, reputation feeds, help page) - #56

Closed
fionera wants to merge 10 commits into
masterfrom
feat/gandalf-features
Closed

fionera wants to merge 10 commits into
masterfrom
feat/gandalf-features

Conversation

@fionera

@fionera fionera commented Jul 9, 2026

Copy link
Copy Markdown
Member

Overview

Implements a broad bot-blocker feature set on top of Berghain (modelled on the "Gandalf" spec), while keeping the core design intact: Berghain stays stateless (all runtime state lives in HAProxy stick-tables/maps), the Go hot path stays zero-allocation, and HAProxy owns policy while Berghain remains a pure SPOE function. Also closes four open issues.

Opened as a draft — it's a large surface and likely wants to land in smaller pieces / with maintainer input on scope. Commits are grouped so they can be cherry-picked or split.

What's included

Challenge engine (Go + web)

  • Per-level configurable PoW difficulty (leading-zero bits, default 16, range 1–255), advertised via a fixed-width d field; the server enforces its own level's difficulty.
  • New pow-worker challenge type — same PoW, must be solved in a Web Worker (inlined into the single-file bundle).
  • Browser capability detection with actionable advisories, a session id (ena@<uuid>), and a ban-countdown screen.

HAProxy policy (stateless)

  • Three rate limiters (global, per-IP burst, challenge-endpoint anti-bruteforce) within the 3-stick-counter budget, a flat temporary ban + countdown page, an AI/scraper tarpit, and keyed IP log-hashing (Go HashSource + HAProxy).
  • Reputation/ASN/VPN/Tor/Cloudflare maps with an onion/I2P bypass, fed by a new cmd/feedupdater tool.
  • Optional CrowdSec integration scaffold for community reputation + escalating bans.

Docs & ops

  • A self-contained, JS-free help page under docs/ + a GitHub Pages workflow.
  • README configuration + operations documentation.

Closes

Verification

  • go build/test/vet + staticcheck clean; the Go hot path stays 0 allocs/op.
  • npm run lint + npm run build (both default and native-crypto modes, single-file output).
  • haproxy -c valid for both example configs.
  • Browser e2e passes against a live stack (challenge solved end-to-end).
  • feedupdater unit-tested and confirmed against live feeds.

Notes

  • The inlined Web Worker grows the challenge page past HAProxy's default 16k buffer, so tune.bufsize is raised to 32k.
  • A gpt-5.5 code review was run over the branch; its three findings (a zero-difficulty fail-open, the onion-port bypass gap, and a CI e2e that could pass without exercising the stack) are fixed in this PR.
  • CrowdSec is an opt-in scaffold, not wired into the default stack.

@fionera

fionera commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

Live IP reputation via the HAProxy peers protocol

Replaced the map-file approach for individual-IP reputation with the HAProxy peers protocol (as implemented in haproxy-go), so entries are pushed into HAProxy's stick-tables live, with no reload.

What changed

  • New internal/peerserver implements the sending side of the peers protocol (haproxy-go only had the receive side). It handles the handshake, table definitions, entry updates, sync and heartbeats. (Entry data is marshalled by hand — EntryUpdate.Marshal mis-encodes the data section upstream, calling Unmarshal.)
  • cmd/feedupdater now runs as a peer: Tor exit nodes → gpt0=3 (challenge), banlists → gpt0=1 (block), served live into st_reputation_v4/st_reputation_v6. haproxy.cfg gains a peers berghain section and consults the gpt0 tag.
  • CIDR/range feeds (Cloudflare, ASN, VPN) stay as map/ACL files — stick-tables key on exact IPs and can't do longest-prefix matching. feedupdater still refreshes those atomically.
  • ban.map / reputation.map / tor_exit.map deleted; docker-compose runs feedupdater as the peer (haproxy -L haproxy_local).

Validated against real HAProxy (--network host):

  • feedupdater pushed 1378 entries; show table st_reputation_v4 → used:1378 (Tor IPs gpt0=3, banned 127.0.0.1 gpt0=1).
  • A request from the banned IP was silent-dropped (http=000), with no reload.
  • Full docker compose stack + browser e2e still green.

fionera added 10 commits July 10, 2026 13:53
Add a per-level difficulty knob (leading-zero bits, default 16, range 1-255) advertised to the client via a fixed-width "d" field, and a new pow-worker challenge type that must be solved inside a Web Worker. An unset difficulty falls back to the historic 16-bit target so directly-built configs are never zero-work.
Strip the port from the Host header for cookie identity, trusted-domain matching and the cookie Domain attribute (#46). Receive the client session id over SPOE and log it. Log a keyed hash of the source IP instead of the plaintext address.
…ooks

Detect missing browser capabilities and show actionable advisories; show and send a per-page session id (ena@<uuid>); render a ban-countdown screen (type 3); solve the pow-worker challenge in an inlined Web Worker; add a VITE_HOOKS system to customise the page without forking (#26); fix the cookie-warning element id.
Global/per-IP rate limits and a challenge-endpoint anti-bruteforce ban with a countdown page; tarpit library/scraper User-Agents; reputation/ASN/VPN/Tor/Cloudflare maps with an onion/I2P bypass; keyed IP hashing in logs; an alternative UA/endpoint filtering example config (#19).
Fetch public feeds (Cloudflare ranges, Tor exit list) and write the HAProxy map/ACL files atomically. Runs once or on an interval.
Drive headless Chrome against the running stack to verify the client bundle actually solves a challenge and reaches the backend (#9). Wired into the CI e2e job; kept in a separate module so chromedp stays out of the main module.
A self-contained, JS-free help page under docs/ plus a Pages deploy workflow; document the new config knobs and operations; an optional CrowdSec integration scaffold.
haproxy-go's peers package only receives stick-table updates; this adds the sending side (handshake, table definitions, entry updates, sync + heartbeat) so an external process can push entries into a running HAProxy's stick-tables live. Entry data is marshalled by hand because EntryUpdate.Marshal mis-encodes it upstream.
Individual-IP reputation (Tor exit nodes, banlists) is now pushed into HAProxy st_reputation stick-tables live over the peers protocol instead of being written to map files that need a reload. CIDR/range feeds (Cloudflare, ASN, VPN) stay as map/ACL files since stick-tables cannot do prefix matching. haproxy.cfg gains a peers section + reputation tables and consults the gpt0 tag; docker-compose runs feedupdater as the peer.
v0.1.1 includes the fix for EntryUpdate.Marshal's data encoding, so peerserver no longer needs to hand-marshal entry updates. Verified against real HAProxy: 1382 entries synced and a banned source silent-dropped.
@fionera
fionera force-pushed the feat/gandalf-features branch from 2b39b25 to 98a8a51 Compare July 10, 2026 14:24
@fionera

fionera commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

The omnibus branch has been rebased onto the current origin/master and this PR is superseded by focused draft PRs:

Core challenge behavior

Frontend and supportability

HAProxy policy

Live reputation

Verification

The custom peers-protocol implementation was replaced by HAProxy’s supported atomic Runtime API. The nonfunctional CrowdSec starting-point scaffold was intentionally not carried into these PRs; a real CrowdSec integration should be implemented and tested as a follow-up.

@fionera fionera closed this Jul 10, 2026
@fionera
fionera deleted the feat/gandalf-features branch July 11, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant