feat(eventbus): add retry backoff and dead-letter routing with replay - #204
Merged
Conversation
Add a consumer error topology: a DefaultErrorHandler with bounded exponential backoff that, once retries are exhausted, routes the record to its dead-letter topic via a DeadLetterPublishingRecoverer, so a poison message can never block a partition. RetryTopicNaming derives the retry and dead-letter topic names from configurable suffixes; the dead-letter route uses partition -1 so the producer places the record by key, preserving per-key affinity. DeadLetterReplayer re-drives a dead-letter topic back to a target topic preserving keys and awaiting each send, so the returned count reflects confirmed re-publishes. All beans are conditional-on-missing and load only when the bus is configured. Covered by unit tests plus a Testcontainers-Redpanda integration test. Closes #193
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Consumer-side error topology for E-03: bounded retry with backoff, terminal dead-letter routing for poison messages, and a replay path. Generic building blocks in
libs/fincore-eventbus; a concrete@KafkaListenerwiring lands with the first consumer (E-02).RetryDlqProperties(fincore.eventbus.retry): max attempts, exponential backoff (initial/multiplier/max), retry/dead-letter suffixes; validated.RetryTopicNaming- puredeadLetterTopic(base)/retryTopic(base, tier)conventions.RetryDlqConfiguration- aDefaultErrorHandlerwith a boundedExponentialBackOff(maxAttempts - 1retries, since the initial delivery is attempt 1) + aDeadLetterPublishingRecovererthat routes an exhausted record to<topic><dlt-suffix>. The route uses partition-1so the producer places the record by key, preserving affinity without assuming equal partition counts. A consumer attaches the handler to its container factory.DeadLetterReplayer- re-drives a dead-letter topic back to a target topic, preserving keys and awaiting each send so the returned count is confirmed (incident-recovery safe). Non-destructive (re-driving duplicates is safe with the idempotent consumer from F-03.5 Idempotent consumer base class / annotation #192).All beans are
@ConditionalOnMissingBeanand load only when the bus is configured (@Importinto the@ConditionalOnProperty(bootstrap-servers)auto-config).Gates
Local (
-x integrationTest, Docker down in WSL)::libs:fincore-eventbus:{test,detekt,detektTest,spotlessCheck,assemble,compileIntegrationTestKotlin}green. The Testcontainers-Redpanda IT runs on CI: the recoverer routes a failed record to its-dlttopic, and the replayer re-drives-dltback to the base topic preserving keys. critic GO (Spring 6.2ExponentialBackOff.maxAttemptsfix applied), code-reviewer APPROVE (replayer-awaits-futures must-fix applied), security-auditor PASS, evaluator 0.89.Closes #193