Upgrade Pulsar client to 4.2.3

[david-streamlio]

Bumps the Apache Pulsar client from 3.3.0 to 4.2.1.

The 3.3.0 release is no longer covered by Apache Pulsar's security
support policy, and applications that override pulsar-client.version
to a 4.x release fail to build a native image with:

java.lang.ClassNotFoundException: org.apache.pulsar.client.util.WithSNISslEngineFactory

WithSNISslEngineFactory was removed from the Pulsar client in 4.x
(SNI is now applied directly in PulsarChannelInitializer); removing
the dangling addRuntimeInitializedClass(...) registration makes the
extension compatible with 4.x.

The rest of the native-image configuration in SmallRyeReactiveMessagingPulsarProcessor was reviewed against the
Pulsar 4.2.1 source and remains correct: MessageCryptoBc, SecretsSerializer, the OAuth2 protocol classes, the async-http-client hints, and the Protobuf hints are all still present in 4.2.1 with the same package layout. The @Substitute for ClientCredentialsFlow.loadPrivateKey in io.quarkus.pulsar.runtime.graal.Substitutions replaces the entire method body, so the small upstream tweak inside that method (getInputStream() → getContent()) does not affect us.

Validated by running ./mvnw verify -f integration-tests/reactive-messaging-pulsar/ -Dnative —
all six native-image builds in that module compile cleanly against Pulsar 4.2.1. Could not run the resulting binaries locally on macOS arm64 because the container build produced Linux binaries; runtime smoke tests are left to CI.

Fixes #48776

[cescoffier]

Before merging we need @ozangunalp review.

[cescoffier]

Most of the CI is failing - so it's is likely incorrect.

[ozangunalp]

I'll check why the CI is failing.

[ozangunalp]

@david-streamlio Thanks for looking into this!

I was able to fix the tests with the following change: a471f5b

[gsmet]

AFAICS, it's also failing on the JVM, not only in native. I will make this PR draft for now until the problems are resolved.

[david-streamlio]

hanks @ozangunalp! Cherry-picked your commit onto the branch.

I also kept the small additional commit I pushed just before yours (57e767a), which registers the platform Netty
SocketChannel / DatagramChannel classes (Epoll*, KQueue*, IOUring*) for reflection. They look like two distinct
reflection failures triggered by Pulsar 4.2.x:

• Your fix covers Pulsar's new SSL factory SPI — PulsarSslFactory implementations are loaded via
  Class.forName(...).newInstance().

• The CI stack trace I saw on the original failing run was:
  java.lang.IllegalArgumentException: Class EpollSocketChannel does not have a public non-arg constructor
  at io.netty.channel.ReflectiveChannelFactory.(ReflectiveChannelFactory.java:36)
  at io.netty.resolver.dns.DnsNameResolverBuilder.socketChannelType(DnsNameResolverBuilder.java:253)
  at org.apache.pulsar.client.impl.DnsResolverGroupImpl.createDnsNameResolverBuilder(DnsResolverGroupImpl.java:105)

• This is the new DnsResolverGroupImpl in 4.2.x calling
  socketChannelType(EventLoopUtil.getClientSocketChannelClass(), true), which returns EpollSocketChannel.class on Linux. Netty's ReflectiveChannelFactory then needs the no-arg constructor at runtime, which quarkus-netty
  registers only for the NIO variants.

My guess is your local repro didn't trip that path (macOS / NIO would skip it), so it didn't surface alongside
the SSL failure you saw. Happy to drop my commit if CI shows it's redundant, but on the original failing job it
looked load-bearing — letting CI decide.

[ozangunalp]

@cescoffier, what do you think about registering Netty channels as reflective classes, only for with pulsar extension ?

[ozangunalp]

@david-streamlio it doesn't look very good in my run in here : https://github.com/ozangunalp/quarkus/actions/runs/27146243773/job/80129858734?pr=165

It seems like a segfault related to LightProtoCodec. Are you aware of any changes related to that in Pulsar 4?

[david-streamlio]

@ozangunalp Yes — that segfault is a Pulsar 4 change, and it's separate from the reflection issues we already fixed here.

Pulsar 4.2.x's LightProtoCodec switched to Unsafe-based zero-copy string encoding. Its static initializer computes STRING_VALUE_OFFSET = UNSAFE.objectFieldOffset(String.value) and BYTE_ARRAY_BASE_OFFSET = UNSAFE.arrayBaseOffset(byte[]), and writeRawString (the frame in your crash, line 403) dereferences those offsets directly. The native build log shows GraalVM's automatic RecomputeFieldValue transform giving up on this code shape ("Could not determine the field where the value … is stored"), so the class gets build-time-initialized and the host-JVM offsets are baked into the image → segfault at runtime against the SubstrateVM heap layout.

Fix: initialize LightProtoCodec at run time so the offsets are recomputed correctly — same mechanism we already use for Commands/ConnectionPool (and Commands depends on LightProtoCodec, so they belong together). I've pushed that to the branch; let's see what CI says.

On your question to @cescoffier about the Netty channels: the real home for all of this is upstream. My Pulsar PR apache/pulsar#25883 (merged to master / 5.0.0-M1) already embeds the reflection + runtime-init metadata into pulsar-client-original, but it doesn't yet cover LightProtoCodec, DefaultPulsarSslFactory, or the Netty DNS-resolver channels. I'm filing a short follow-up to add those three. Once that lands and gets backported to branch-4.2, Quarkus can drop these downstream registrations entirely — including the Netty channel one, so it wouldn't need to live in the extension long-term.

[ozangunalp]

@david-streamlio thanks for looking into it. I've tested a run previously that fixes the issue with LightProtoCodec : ozangunalp#165 . I think we could also get away with a substitution.

[david-streamlio]

Thanks @ozangunalp! Looks like we landed on the exact same fix — the commit currently at the head of this PR (95d740e) adds LightProtoCodec as a runtime-initialized class, identical to your 0712414 in #165. So we're already aligned, no cherry-pick needed.

On the substitution: I think runtime-init is the better choice here. A @substitution would mean hand-maintaining a copy of writeRawString/the Unsafe-offset logic against Pulsar's internal LightProtoCodec, which we'd have to keep in sync on every Pulsar bump. Runtime-init achieves the same correctness (offsets recomputed against the SubstrateVM heap layout) with one line and no coupling to Pulsar internals. It's also consistent with how we already handle Commands and ConnectionPool. Happy to switch if you feel strongly, but I'd lean on the simpler approach.

I'll mark this ready for review so the full native matrix runs and confirms Messaging2 is green.

[cescoffier]

@ozangunalp did you look at the CI failures? It seems to be a lot more red than usual.

[cescoffier]

We need to look at the CI issue before we can merge this.
Also, @ozangunalp has done a very similar work in the Quarkus 4 branch (which uses Netty 4.2).

[david-streamlio]

Thanks @cescoffier for the review and @ozangunalp for the feedback — much appreciated. I've pushed two commits addressing both points:

• Backportability / Jandex (9395321): swapped getAllKnownImplementations → getAllKnownImplementors for Authentication and PulsarSslFactory. Both are interfaces, so the resolved set is identical, but this drops the newer-Jandex requirement and keeps the fix backportable.

• Classpath guard (d1987a1): the Epoll/KQueue/IOUring channels are now registered for reflection only when QuarkusClassLoader.isClassPresentAtRuntime(...) confirms they're present, instead of unconditionally.

Let me know if there's anything else you'd like changed — otherwise this should be ready for another look. 🙏

[david-streamlio]

Update on the JVM CI failures

The native-image issues are resolved (the Native Messaging tests now pass). The remaining failures — JVM Tests and JVM Integration Tests on every Linux JDK — share a single root cause, and it's not in the extension.

Pulsar 4.2.x's PulsarClientImpl.shutdown() calls addressResolver.close() and dnsResolverGroupLocalInstance.close() without a try/catch — the only cleanup steps in that method that aren't guarded. When addressResolver.close() throws the Netty IllegalStateException: channel not registered to an event loop race (a DNS lookup in flight at shutdown), it aborts the rest of shutdown() before shutdownEventLoopGroup(), closeCnxPool(), timer.stop() and shutdownExecutors() run. The event-loop's non-daemon threads are then leaked, so the Surefire fork never exits and is killed after ~20 min (Tests run: 0). This hits both PulsarDevModeTest (via the hot-reload restart) and the reactive-messaging-pulsar IT module.

It's a regression from the shared DnsResolverGroup added in apache/pulsar#24784 (absent in 3.3.0), present through 4.2.2 and master. There's no client config to disable the resolver group, so it can't be worked around in the extension. The failure is timing-sensitive (it didn't reproduce in 12 local runs, but triggered on all 8 Linux CI jobs).

Upstream fix submitted: apache/pulsar#26045 — it wraps both DNS-resolver close calls in try/catch like every other step, so shutdown always proceeds to release the event-loop group. A branch-4.2 backport is ready. This PR will need a 4.2.x release containing that fix before CI can go green.

[cescoffier]

@ozangunalp Do you think we can merge this one?

[ozangunalp]

@david-streamlio's fix has been released (today) on upstream Pulsar 4.2.3. Let's rebase this branch on main (Quarkus 4) and bump it to Pulsar 4.2.3.

[david-streamlio]

Rebased on main and bumped the Pulsar client to 4.2.3, which was released on 2026-07-06 and includes the PulsarClientImpl.shutdown() DNS resolver fix (apache/pulsar#26045) that was causing the JVM test jobs to hang.

Since most of the earlier commits from this PR were already merged into main, the branch is now down to three commits:

• Guard Netty native-transport reflection by classpath presence
• Use getAllKnownImplementors for Jandex backport compatibility
• Upgrade Pulsar client to 4.2.3

@ozangunalp @cescoffier this should address the remaining requested changes — ready for re-review whenever CI is approved to run.

[quarkus-bot]

Status for workflow Quarkus CI

This is the status report for running Quarkus CI on commit 3e3c8b0.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.