Add the streaming monitor engine for high-scale checks#104

Draft

chrisbliss18 wants to merge 71 commits into

v2from

feature/streaming-monitor-engine

Commits on May 10, 2026

Prototype streaming monitor scheduler
Chris Jean
committed
Allow auto defaults for capacity guardrails
Chris Jean
committed
Default omitted checker sizing config
Chris Jean
committed
Normalize zero-value capacity defaults
Chris Jean
committed
Reload streaming targets when active count changes
Chris Jean
committed
Make checker results lossless under backpressure
Chris Jean
committed
Prewarm streaming workers after target reloads
Chris Jean
committed
Hold streaming workers at the rate-derived target
Chris Jean
committed
Align streaming projection with rollback freshness gates
Chris Jean
committed
Project streaming freshness after every interval check
Chris Jean
committed
Project streaming freshness at commit time
Chris Jean
committed
Add streaming scheduler freshness headroom
Chris Jean
committed
Stabilize streaming worker scaling under failures
Chris Jean
committed
Decouple streaming result drain from side effects
Chris Jean
committed
Short-circuit streaming no-op successes
Chris Jean
committed
Honor WPCOM notification disable config
Chris Jean
committed
Discard stale streaming work after deactivation
Chris Jean
committed
Reduce disabled WPCOM notification log volume
Chris Jean
committed
Batch streaming failure history writes
Chris Jean
committed
Scale streaming side-effect shards with target volume
Chris Jean
committed
Autosize DB pool for streaming side effects
Chris Jean
committed
Let streaming workers scale to observed latency
Chris Jean
committed
Temper streaming worker latency headroom
Chris Jean
committed
Stabilize streaming scheduler under failure pressure
Chris Jean
committed
Pace streaming catch-up dispatch
Chris Jean
committed
Scale streaming dispatch budget by elapsed time
Chris Jean
committed
Let streaming workers grow for backlog catch-up
Chris Jean
committed
Cap elapsed streaming dispatch bursts
Chris Jean
committed
Bypass local DNS stub for high-volume checks
Chris Jean
committed
Control streaming failure pressure directly
Chris Jean
committed
Make streaming pressure control sticky
Chris Jean
committed
Cache checker DNS resolutions between passes
Chris Jean
committed
Suppress new local failures during pressure
Chris Jean
committed
Raise streaming pressure worker target
Chris Jean
committed
Cache streaming checker requests
Chris Jean
committed
Prefer IPv4 DNS lookups for checks
Chris Jean
committed
Drain streaming results in batches
Chris Jean
committed
Keep real failure retries under pressure
Chris Jean
committed
Stabilize streaming checks under resolver pressure
Chris Jean
committed
Add explicit resolver control for capacity isolation
Chris Jean
committed
Fix resolver override transport rebuild
Chris Jean
committed
Keep streaming target reloads off the hot path
Chris Jean
committed
Defer streaming reloads while the hot path is behind
Chris Jean
committed
Treat local overload failures as monitor pressure
Chris Jean
committed
Disable checker keep-alives for fleet-scale scans
Chris Jean
committed

Commits on May 11, 2026

Keep streaming dispatch active under result backlog
Chris Jean
committed
Drain streaming results before scheduling more work
Chris Jean
committed
Ramp streaming workers faster under capacity pressure
Chris Jean
committed
Keep streaming scheduler ticks moving under result load
Chris Jean
committed
Move streaming freshness projection off the hot path
Chris Jean
committed
Ramp streaming dispatch more quickly at fleet scale
Chris Jean
committed
Pool cleartext checks by resolved IP
Chris Jean
committed
Smooth streaming dispatch under result pressure
Chris Jean
committed
Keep streaming ticks responsive during result flow
Chris Jean
committed
Cap streaming pressure recovery worker growth
Chris Jean
committed
Try alternate HTTP addresses before failing checks
Chris Jean
committed
Use an ordered heap for streaming due buckets
Chris Jean
committed
Let streaming backlog pressure grow workers
Chris Jean
committed
Avoid streaming worker surges from transient latency
Chris Jean
committed
Dispatch faster when streaming freshness falls behind
Chris Jean
committed
Keep streaming backlog catch-up conservative
Chris Jean
committed
Reduce streaming workers during failure pressure
Chris Jean
committed
Scale streaming target reload cadence by fleet size
Chris Jean
committed
Reduce streaming reload and projection pressure
Chris Jean
committed
Smooth streaming scheduler dispatch ticks
Chris Jean
committed
Dispatch streaming backlog outside ticker ticks
Chris Jean
committed
Interleave streaming dispatch during result drains
Chris Jean
committed
Relax streaming result pause at fleet scale
Chris Jean
committed
Reduce streaming hot-path allocation pressure
Chris Jean
committed
Keep streaming dispatch alive while results drain
Chris Jean
committed
Use more memory to reduce streaming scheduler pressure
Chris Jean
committed