[fix][client] Prevent client shutdown from leaking event loop threads when DNS resolver close fails

[david-streamlio]

Motivation

PulsarClientImpl.shutdown() closes most resources defensively: each close() is wrapped in its own try/catch and the last failure is recorded in throwable so that shutdown always continues and releases the remaining resources. Two cleanup steps, however, are not guarded:

if (addressResolver != null) {
    addressResolver.close();
}

if (dnsResolverGroupLocalInstance != null) {
    dnsResolverGroupLocalInstance.close();
}

These run before shutdownEventLoopGroup(), closeCnxPool(), timer.stop() and shutdownExecutors(). If addressResolver.close() throws, the exception propagates straight to the outer catch, and the remaining cleanup never runs — so the EventLoopGroup's non-daemon threads (plus the connection pool, timer and executors) are leaked and the JVM never exits.

This is reachable in practice: when a DNS lookup is still in flight as the client shuts down, the Netty address resolver can throw java.lang.IllegalStateException: channel not registered to an event loop (a known Netty shutdown race). It reproduces consistently when upgrading Quarkus to pulsar-client 4.2.1 — the test JVM hangs and is eventually killed by the Surefire fork timeout (~20 minutes), with:

WARN  org.apache.pulsar.client.impl.PulsarClientImpl - Failed to shutdown Pulsar client: java.lang.IllegalStateException: channel not registered to an event loop
Exception in thread "pulsar-client-shutdown-thread" java.lang.IllegalStateException: channel not registered to an event loop

The DNS resolver group was introduced in #24784, and the unguarded close has been present since then (4.2.0 onward, including 4.2.1/4.2.2 and master).

Modifications

Wrap addressResolver.close() and dnsResolverGroupLocalInstance.close() in try/catch, logging the failure and recording it in throwable, exactly like every other resource closed by shutdown(). This guarantees shutdown() always proceeds to release the event loop group, connection pool, timer and executors even if the DNS resolver fails to close, while still surfacing the original failure to the caller.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage. It restores the existing best-effort cleanup contract for two steps that were missing it; the surrounding steps already follow this pattern.

Does this pull request potentially affect one of the following parts:

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

[david-streamlio]

This is currently blocking the Quarkus upgrade to pulsar-client 4.2.x — see quarkusio/quarkus#54516. The leaked event-loop (non-daemon) threads cause the smallrye-reactive-messaging-pulsar JVM tests and the reactive-messaging-pulsar integration tests to hang until the Surefire fork timeout (~20 min), failing CI on every Linux JDK.

A released 4.2.x containing this fix would unblock that PR. A backport to branch-4.2 is ready if it helps: https://github.com/david-streamlio/pulsar/tree/fix-client-shutdown-dns-resolver-leak-4.2

[david-streamlio]

Thanks for getting this merged and backported to branch-4.2 (commit 2baa1f7) so quickly! 🙏

Is there a planned timeline for cutting a 4.2.3 release? This fix is currently blocking a downstream Quarkus upgrade (quarkusio/quarkus#54516) — the Quarkus smallrye-reactive-messaging-pulsar extension can't move past pulsar-client 4.2.1 because the DNS-resolver shutdown leak this PR fixes causes JVM test hangs on client shutdown. We've validated the fix end-to-end (the regression test added here passes on the fixed branch and fails without it), so the only thing left is a published 4.2.3 artifact on Maven Central.

If a 4.2.3 release isn't on the near-term roadmap, is there anything I can help with to move it along (e.g. testing a release candidate)? Happy to assist. Thanks again!