Motivation
Debezium ships a Google Cloud Spanner source connector (io.debezium:debezium-connector-spanner, present in the debezium-bom at our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.
Spanner is a managed, horizontally-scaled database with a growing Pulsar user base, and the connector itself wraps cleanly — the interesting work is in testing.
Proposal
Add a debezium/spanner module (pulsar-io-debezium-spanner) following the structure of the existing Debezium modules:
DebeziumSpannerSource extending DebeziumSource
include("debezium:pulsar-io-debezium-spanner") in settings.gradle.kts
debezium-connector-spanner = { module = "io.debezium:debezium-connector-spanner" } in the version catalog
- NAR packaging + distribution wiring
- An integration test against the Spanner emulator (
gcr.io/cloud-spanner-emulator/emulator) so CI needs no GCP credentials
Notes
Identified by a scan of the Debezium connector catalog against our connector inventory.
Motivation
Debezium ships a Google Cloud Spanner source connector (
io.debezium:debezium-connector-spanner, present in thedebezium-bomat our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.Spanner is a managed, horizontally-scaled database with a growing Pulsar user base, and the connector itself wraps cleanly — the interesting work is in testing.
Proposal
Add a
debezium/spannermodule (pulsar-io-debezium-spanner) following the structure of the existing Debezium modules:DebeziumSpannerSourceextendingDebeziumSourceinclude("debezium:pulsar-io-debezium-spanner")insettings.gradle.ktsdebezium-connector-spanner = { module = "io.debezium:debezium-connector-spanner" }in the version cataloggcr.io/cloud-spanner-emulator/emulator) so CI needs no GCP credentialsNotes
CREATE CHANGE STREAM ...) as part of test setup. Confirm the emulator supports change streams at the version we target — if it does not, the test must be credential-gated, and we should avoid repeating theazure-data-explorersituation ([improve][test] Azure Data Explorer E2E test is always skipped in CI — add a runnable integration test #49) where a credential-gated E2E test means the connector is never actually exercised in CI.task.idgap described in [Bug] Debezium SQL Server source fails to start: multi-partition connector never receives task.id #61. That issue should land first.GenericContainerpattern (see the Testcontainers version conflict noted in [improve][test] Add Debezium Oracle source integration test #62).Identified by a scan of the Debezium connector catalog against our connector inventory.