Skip to content

[Fix] Prevent consensus subscription teardown from blocking unsubscribe - #18804

Merged
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:fix/consensus-subscription-async-close
Oct 10, 2026
Merged

jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:fix/consensus-subscription-async-close

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Unsubscribing a consensus based topic can time out when queue close waits for an active prefetch read lock or consensus queue deregistration. Broker teardown now fences new queue work immediately and completes blocking cleanup on the close executor.

Uncommitted lingering tablets are discarded during close so a later queue can replay them from WAL. Prefetch task deregistration checks the task instance, preventing delayed cleanup of an old queue from removing a replacement task.

A table model cluster IT covers ongoing writes, two consumers in one group, bounded unsubscribe of the last consumer, and immediate resubscription followed by delivery of new rows. The existing integration-test/pom.xml exclusion keeps subscription ITs out of the Apache build; this IT is intended to run after synchronization to the Timecho repository.

Verification

  • mvn -o -pl iotdb-core/datanode spotless:check passed.
  • English locale DataNode main and test sources compiled in the 28 module reactor.
  • mvn -o -pl iotdb-core/datanode -am -P with-zh-locale -DskipTests test-compile passed.
  • The four new ConsensusPrefetchingQueueTest methods passed via mvn -o -pl iotdb-core/datanode -Dtest=ConsensusPrefetchingQueueTest#... surefire:test.
  • mvn -o -pl integration-test -P with-integration-tests spotless:apply passed.
  • mvn -q -o -pl integration-test -am -P with-integration-tests -DskipTests test-compile passed across 39 modules, including the new IT source.
  • The new IT was reviewed and compiled locally but not run: integration-test/pom.xml excludes **/subscription/it/**, and subscription is disabled in this Apache checkout. It needs execution in the Timecho repository after synchronization.
  • A forced run of the full ConsensusPrefetchingQueueTest class had 37 passes and 6 failures in existing tests that use the default subscription memory budget. CommonConfig.SUBSCRIPTION_ENABLED=false makes that budget zero in this checkout. The repository's normal Surefire configuration excludes subscription unit tests.

Review

  • Reviewed close and rebind races.
  • Added regression tests for blocked cleanup and replacement task registration.
  • Added a cluster IT for unsubscribe under writes and immediate resubscription.

Key changed classes: ConsensusSubscriptionBroker, ConsensusPrefetchingQueue, ConsensusSubscriptionPrefetchExecutor, and ConsensusPrefetchSubtask.

@jt2594838
jt2594838 merged commit b94aa09 into apache:master Oct 10, 2026
35 of 39 checks passed
@jt2594838
jt2594838 deleted the fix/consensus-subscription-async-close branch October 10, 2026 06:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants