kafka-utils

Utilities for working with Kafka

Kafka Streams Utils

Utilities for working with Kafka Streams applications. This library provides:

• TopologyInformation to parse Kafka Streams TopologyDescription and extract sinks and sources
• Helpers to automatically configure Serdes for Kafka Streams applications

They are mainly used in streams-bootstrap and fluent-kafka-streams-tests, but you are free to use them in your own projects if needed.

Getting Started

You can add kafka-streams-utils via Maven Central.

Gradle

implementation group: 'com.bakdata.kafka', name: 'kafka-streams-utils', version: '1.0.0'

Maven

<dependency>
    <groupId>com.bakdata.kafka</groupId>
    <artifactId>kafka-streams-utils</artifactId>
    <version>1.0.1</version>
</dependency>

For other build tools or versions, refer to the latest version in MvnRepository.

Kafka BOM

BOM is short for Bill of Materials. It is a Maven feature that allows you to manage the versions of dependencies in a single place. This project provides a BOM for Kafka dependencies, i.e., Kafka core dependencies in group org.apache.kafka as well as Confluent dependencies in group io.confluent.

Getting Started

You can add kafka-bom via Maven Central.

Gradle

implementation(platform('com.bakdata.kafka:kafka-bom:1.0.0'))

Maven

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.bakdata.kafka</groupId>
            <artifactId>kafka-bom</artifactId>
            <version>1.0.1</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

For other build tools or versions, refer to the latest version in MvnRepository.

Usage

After adding the BOM to your project, you can use the provided dependencies without specifying a version.

Gradle

implementation group: 'org.apache.kafka' name: 'kafka-streams'
implementation group: 'io.confluent' name: 'kafka-streams-avro-serde'

Maven

<dependencies>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-streams</artifactId>
    </dependency>
    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-streams-avro-serde</artifactId>
    </dependency>
</dependencies>

Development

If you want to contribute to this project, you can simply clone the repository and build it via Gradle. All dependencies should be included in the Gradle files, there are no external prerequisites.

> git clone [email protected]:bakdata/kafka-utils.git
> cd kafka-utils && ./gradlew build

Please note, that we have code styles for Java. They are basically the Google style guide, with some small modifications.

Contributing

We are happy if you want to contribute to this project. If you find any bugs or have suggestions for improvements, please open an issue. We are also happy to accept your PRs. Just open an issue beforehand and let us know what you want to do and why.

License

This project is licensed under the MIT license. Have a look at the LICENSE for more details.