On this page
Learn how to replicate your change data capture (CDC) events with a MongoDB Kafka Connector sink connector. CDC is a software architecture that converts changes in a datastore into a stream of CDC events. A CDC event is a message containing a reproducible representation of a change performed on a datastore. Replicating data is the process of applying the changes contained in CDC events from one data store onto a different datastore so that the changes occur in both datastores.
Use a CDC handler to replicate CDC events stored on an Apache Kafka topic into MongoDB. A CDC handler is a program that translates CDC events from a specific CDC event producer into MongoDB write operations.
A CDC event producer is an application that generates CDC events. CDC event producers can be datastores, or applications that watch datastores and generate CDC events corresponding to changes in the datastores.
MongoDB change streams is an example of a CDC architecture. To learn more about change streams, see the MongoDB Kafka Connector guide on Change Streams.
If you would like to view a tutorial demonstrating how to replicate data, see the Replicate Data With a Change Data Capture Handler tutorial.
You cannot apply a post processor to CDC event data. If you attempt to specify both, the connector logs a warning.
You can specify a CDC handler on your sink connector with the following configuration option:
change.data.capture.handler=<cdc handler class>
To learn more, see change data capture configuration options in the MongoDB Kafka Connector.
The MongoDB Kafka Connector provides CDC handlers for the following CDC event producers:
Click the following tabs to learn how to configure CDC handlers for the preceding event producers:
If none of the prebuilt CDC handlers fit your use case, you can create your own.
Your custom CDC handler is a Java class that implements the
To learn more, see the source code for the CdcHandler interface.
To view examples of CDC handler implementations, see the source code for the prebuilt CDC handlers.
To configure your sink connector to use your custom CDC Handler, you must perform the following actions:
- Compile your custom CDC handler class to a JAR file.
Add the compiled JAR to the classpath/plugin path for your Kafka workers. For more information about plugin paths, see the Confluent documentation.Note
Kafka Connect loads plugins in isolation. When you deploy a custom write strategy, both the connector JAR and the CDC handler JAR should be on the same path. Your paths should resemble the following:
To learn more about Kafka Connect plugins, see this guide from Confluent.
- Specify your custom class in the
To learn how to compile a class to a JAR file, see this guide from Oracle.