You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Debezium ships a MariaDB source connector (io.debezium:debezium-connector-mariadb, present in the debezium-bom at our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.
MariaDB is the highest-value gap:
Debezium split the MariaDB connector out of the MySQL connector, and both now build on the shared debezium-connector-binlog base. Our existing debezium/mysql module is therefore close to a drop-in template for the connector, the config plumbing, and the integration test.
Add a debezium/mariadb module (pulsar-io-debezium-mariadb) following the structure of debezium/mysql:
DebeziumMariaDbSource extending DebeziumSource, asserting Debezium's connector.class/task.class for the MariaDB connector
include("debezium:pulsar-io-debezium-mariadb") in settings.gradle.kts
debezium-connector-mariadb = { module = "io.debezium:debezium-connector-mariadb" } in the version catalog
NAR packaging + distribution wiring, mirroring the MySQL module
A Testcontainers integration test cloned from DebeziumMysqlSourceTest
Notes
Use the GenericContainer pattern rather than a specialized Testcontainers module: debezium-bom pins Testcontainers core to 2.x while the specialized modules still target 1.21.x and reference shaded classes removed in 2.x (see the discussion in [improve][test] Add Debezium Oracle source integration test #62).
Motivation
Debezium ships a MariaDB source connector (
io.debezium:debezium-connector-mariadb, present in thedebezium-bomat our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.MariaDB is the highest-value gap:
debezium-connector-binlogbase. Our existingdebezium/mysqlmodule is therefore close to a drop-in template for the connector, the config plumbing, and the integration test.jdbc/mariadb, integration test added in [improve][test] Add MariaDB JDBC sink integration test #59), so a MariaDB source completes the story for that database.Proposal
Add a
debezium/mariadbmodule (pulsar-io-debezium-mariadb) following the structure ofdebezium/mysql:DebeziumMariaDbSourceextendingDebeziumSource, asserting Debezium'sconnector.class/task.classfor the MariaDB connectorinclude("debezium:pulsar-io-debezium-mariadb")insettings.gradle.ktsdebezium-connector-mariadb = { module = "io.debezium:debezium-connector-mariadb" }in the version catalogDebeziumMysqlSourceTestNotes
GenericContainerpattern rather than a specialized Testcontainers module:debezium-bompins Testcontainers core to 2.x while the specialized modules still target 1.21.x and reference shaded classes removed in 2.x (see the discussion in [improve][test] Add Debezium Oracle source integration test #62).task.idgap described in [Bug] Debezium SQL Server source fails to start: multi-partition connector never receives task.id #61.Identified by a scan of the Debezium connector catalog against our connector inventory.