[improve][test] Re-enable broker-dependent Kafka Connect Adaptor tests (sink/source at 0% coverage)

[david-streamlio]

Summary

Five Kafka Connect Adaptor test files are excluded from compilation in kafka-connect-adaptor/build.gradle.kts, so the adaptor's core sink/source/offset paths have 0% executed test coverage. This is a tracking issue to re-enable them.

Background

Introduced in #31 ("Fix loading offset issues in Kafka Adaptor"), which noted the adaptor tests do not compile against the pinned Pulsar artifacts and left this exclusion in place:

// Some KCA tests depend on pulsar-broker internals that have changed since the
// last released version. Those tests will compile once matching pulsar artifacts
// are published; exclude only them for now so that self-contained unit tests
// still compile and run in CI.
val brokerDependentTestSources = listOf(
    "**/KafkaConnectSinkTest.java",
    "**/KafkaConnectSourceErrRecTest.java",
    "**/KafkaConnectSourceErrTest.java",
    "**/KafkaConnectSourceTest.java",
    "**/PulsarOffsetBackingStoreTest.java",
)
tasks.named<JavaCompile>("compileTestJava") { exclude(brokerDependentTestSources) }

The excluded suites are comprehensive (~62 @Test methods total, e.g. KafkaConnectSinkTest has 45) and are ProducerConsumerBase-based integration tests against an in-process Pulsar broker. They are written — just switched off.

Impact — measured coverage

A JaCoCo run of the module's currently-executing tests (PulsarSchemaToKafkaSchemaTest, KafkaConnectOffsetConverterTest only):

• Module: 8.7% lines, 7.2% branches, 9.3% methods.

Class	Line coverage
PulsarSchemaToKafkaSchema	73.2%
KafkaConnectData	11.4%
AbstractKafkaConnectSource	3.7%
KafkaConnectSink (+ resolver)	0% (0/282)
KafkaConnectSource (+ record)	0% (0/160)
PulsarOffsetBackingStore	0% (0/135)
PulsarKafkaSinkTaskContext	0% (0/115)
KafkaSchemaWrappedSchema, PulsarKafkaSinkContext, PulsarIOSourceTaskContext, sink config	0%

So the connector's two most important classes — the sink and the source — currently have no running test coverage in CI.

Proposed resolution

1. Determine the exact Pulsar artifact/API gap the excluded tests need against the pinned pulsar = 4.2.0 (broker tests classifier / ProducerConsumerBase API).
2. Either:
   • bump to a Pulsar release whose published pulsar-broker (+ tests jar) satisfies the tests, or
   • adapt the excluded tests to the released 4.2.0 broker test API.
3. Remove the brokerDependentTestSources exclusion and confirm the suites compile and pass in CI.
4. Add JaCoCo (module or aggregated) so this regression is visible going forward.

Acceptance

• brokerDependentTestSources exclusion removed.
• KafkaConnectSinkTest / KafkaConnectSourceTest / PulsarOffsetBackingStoreTest / source-error tests compile and run in CI.
• KafkaConnectSink and KafkaConnectSource show non-trivial executed coverage.

Related: #30, #31.

[david-streamlio]

Root cause (diagnosed) — not blocked on an unreleased Pulsar artifact

I temporarily removed the compileTestJava exclusion and compiled the five test files against the pinned pulsar = 4.2.0. All the Pulsar classes they need are already published in 4.2.0; the failures are ordinary version-drift, all fixable in this repo. The "waiting for matching pulsar artifacts" note from #31 is no longer accurate.

Concrete compile errors and fixes:

1. cannot access TestRetrySupport — ProducerConsumerBase → MockedPulsarServiceBaseTest → TestRetrySupport. TestRetrySupport is published in org.apache.pulsar:buildtools:4.2.0, which the module does not declare as a test dependency (pulsar-broker:tests and testmocks are declared, buildtools is not).
   → Fix: add testImplementation(libs.pulsar.buildtools).

2. log has private access in MockedPulsarServiceBaseTest (~15 sites across all four broker tests) — Pulsar made the base class's log field private.
   → Fix: declare a test-local private static final Logger log = LoggerFactory.getLogger(...) in each test instead of inheriting it.

3. log has private access in FileStreamSinkTask (SchemaedFileStreamSinkTask) — same pattern, but the base is Kafka Connect's FileStreamSinkTask.
   → Fix: use the subclass's own logger.

4. MockGenericObjectWrapper is not abstract and does not override getSchemaVersion() in GenericRecord — org.apache.pulsar.client.api.schema.GenericRecord added byte[] getSchemaVersion() (confirmed present in 4.2.0 client-api). This cascades into cannot find symbol method builder() on the same class.
   → Fix: implement getSchemaVersion() in MockGenericObjectWrapper.

5. cannot find symbol setField1/2/3, setStringList on test-local StructWithAnnotations / ComplexStruct — likely cascade once (4) is resolved (annotation processing aborts on the hard errors); verify after fixing the above.

Revised resolution

Purely in-repo against 4.2.0 — no upstream dependency:

1. Add testImplementation(libs.pulsar.buildtools).
2. Give each broker test its own Logger (drop inherited log).
3. Implement GenericRecord.getSchemaVersion() in MockGenericObjectWrapper.
4. Fix the file-stream sink task logger.
5. Rebuild; clean up any residual Lombok/struct setter issues.
6. Remove the brokerDependentTestSources exclusion; confirm the suites compile and pass.

These tests lean on Pulsar's internal test scaffolding (MockedPulsarServiceBaseTest, ProducerConsumerBase, TestRetrySupport), which Pulsar does not treat as stable public API — so future Pulsar bumps may need similar small adaptations, but nothing here waits on the Pulsar project.

[david-streamlio]

Update — compile blocker solved; a separate test-execution blocker found

Compilation: solved

Adding testImplementation(libs.pulsar.buildtools) (which supplies TestRetrySupport, the root of the ProducerConsumerBase → MockedPulsarServiceBaseTest → TestRetrySupport chain) makes all 5 excluded test files compile against the pinned pulsar = 4.2.0. Every earlier error (log has private access, getSchemaVersion() unimplemented, Lombok builder()/setter failures) was a cascade from the missing TestRetrySupport supertype — no logger rewrites or mock edits are needed. compileTestJava passes with just: add buildtools + remove the brokerDependentTestSources exclusion.

Execution: still blocked (different root cause)

With the exclusion removed, the 5 classes compile but do not execute — the module still runs only 35 tests (the 2 non-broker classes). Investigation:

• Gradle does hand all 7 test classes to the runner (verified by printing candidateClassFiles with scanForTestClasses=false + include("**/*Test.class")).
• TestNG silently skips all 5 ProducerConsumerBase subclasses — no error, no worker crash, no NoClassDefFoundError, no result XML, no broker startup.
• Reproduced with a 15-line minimal probe: class ProbeBrokerTest extends ProducerConsumerBase with @BeforeMethod setup()/@AfterMethod cleanup() overrides and one trivial @Test. It compiles and is handed to TestNG, but is skipped exactly like the real tests.
• Ruled out: TestNG group filtering (the probe is ungrouped and still skipped); TestNG version conflict (single org.testng:testng:7.7.1 on the test runtime classpath); missing TestRetrySupport (present in buildtools, confirmed on classpath).
• Both Gradle's static detector (scanForTestClasses=true → "No tests found") and the TestNG runtime (scanForTestClasses=false) decline to run ProducerConsumerBase subclasses.

These are the only ProducerConsumerBase-based tests in the repo, so this Gradle-9 + TestNG harness interaction has never been exercised here. The skip appears specific to how this repo's Gradle test harness runs ProducerConsumerBase subclasses; the likely next step is isolating Gradle's TestNG integration from TestNG itself (a direct org.testng.TestNG -testclass invocation on the test runtime classpath) and/or checking how apache/pulsar's own Gradle/Maven harness configures these base classes.

Suggested split

• Part A (ready): add buildtools, remove the compile exclusion — restores compilation, corrects the stale "waiting on unreleased artifacts" note.
• Part B (follow-up): fix the TestNG/Gradle harness so ProducerConsumerBase subclasses actually execute; only then is coverage restored.

[david-streamlio]

CI result from #119 — execution skip is NOT environment-specific

#119 (the buildtools compile-fix) passed all CI checks. Crucially, this settles the macOS-vs-Linux question:

• :kafka-connect-adaptor:test runs in the "Streaming & Search" CI shard on Linux, and it passed — so the compile fix is validated on Linux (the broker test files now compile in CI).
• But the broker tests still do not execute on Linux either. In that shard's log, :kafka-connect-adaptor:test completed in ~30s (15:17:07 testClasses → 15:17:37 test) with zero Pulsar-broker / BookKeeper startup markers. Only the 35 unit tests ran — the same silent skip of ProducerConsumerBase subclasses observed locally.

So the TestNG-skip of ProducerConsumerBase subclasses is a universal Gradle-9/TestNG harness issue in this repo, not a macOS quirk. Part B (making the broker suites actually execute) remains the real work before sink/source coverage is restored.