[Bug] MqttSinkIntegrationTest can hang the CI job for 45 minutes instead of failing

[david-streamlio]

Description

The Tests - Connectors job on #86 ran for 45m16s and was cancelled by the runner's timeout-minutes: 45, not failed. No diagnostics, 45 minutes of runner time consumed, and a red PR that had nothing wrong with it.

Run: https://github.com/apache/pulsar-connectors/actions/runs/29100595779

Where it hung

:mqtt:integrationTest started at 14:50:30 and never emitted another line. Everything else in the job had already completed — #86's own four modules (mongo, hdfs3, influxdb, hbase) ran and passed by 14:47, and :jdbc:pulsar-io-jdbc-sqlite:test printed its last result at 14:50:36. Then 35 minutes of silence, ending with the runner killing three orphaned java processes.

On healthy runs the same task prints its test result within milliseconds of starting (e.g. run 29102710509).

Why it can hang

MqttSinkIntegrationTest is careful inside the test — every wait is bounded (ackLatch.await(10, SECONDS), receivedPayloads.poll(10, SECONDS), connect().get(10, SECONDS)). The unbounded step is container startup, in @BeforeClass:

private final GenericContainer<?> mqttContainer = new GenericContainer<>(MOSQUITTO_IMAGE)
        .withExposedPorts(MQTT_PORT);          // no waitingFor(), no withStartupTimeout()

@BeforeClass(alwaysRun = true)
public void beforeClass() {
    mqttContainer.start();                     // image pull is not bounded
}

A hang in @BeforeClass produces exactly the observed signature: the Gradle task starts, no test ever reports, nothing times out. The test method also carries no @Test(timeOut = ...), so TestNG cannot bound it either, and Gradle has no per-test timeout. The 45-minute job limit is the only backstop.

Image pull is the most likely culprit — Testcontainers does not bound it, and this job pulls a lot of images now (Debezium MongoDB, Kafka, Solr, DynamoDB/LocalStack, InfluxDB v1+v2, MariaDB, ClickHouse, Postgres, Mosquitto). Docker Hub throttling would stall the pull silently.

Note this test only began running in CI at all when #74 added integrationTest to the matrix, so the hazard is newly live, not longstanding.

Fix

1. Bound the container: .waitingFor(...) with an explicit withStartupTimeout(Duration.ofMinutes(2)).
2. Add @Test(timeOut = ...) as a backstop, consistent with the Debezium tests hardened in [fix][test] Bound record reads in Debezium MySQL and Postgres tests #71.
3. Surface Testcontainers logs in CI — no Testcontainers output appears anywhere in the job log, which is why the cause had to be inferred from timestamps rather than read.

Same principle as #70: a test that cannot fail fast will consume the whole job budget and report a cancellation, which reads like an infrastructure problem rather than a test problem.

Related

The CI test job has roughly doubled in duration today (≈8–10 min → ≈20 min) as container-backed tests landed. At 45 minutes the ceiling is not close, but a single hang exhausts it. Worth considering splitting integrationTest into its own parallel job.