Kafka Topic Properties

On this page

Overview

Settings

Overview

Use the following configuration settings to specify which Kafka topics the MongoDB Kafka source connector should publish data to.

For a list of source connector configuration settings organized by category, see the guide on Source Connector Configuration Properties.

Settings

Name

Description

topic.prefix

Type: string

Description:
The prefix to prepend to database and collection names to
generate the name of the Kafka topic on which to publish the data.

Tip

Example

The following mapping instructs the connector to perform the following actions:

Publish change stream documents originating from the myDb.myColl MongoDB collection to the topicOne Kafka topic.
Publish change stream documents originating from the myDb MongoDB database and a collection other than myColl to the Kafka topic topicTwo.<collectionName>.
Publish change stream documents originating from the myDb MongoDB database that do not bear a collection name to the topicTwo Kafka topic.

topic.namespace.map={"myDb.myColl": "topicOne", "myDb": "topicTwo"}

The following examples show which topic the connector sends different change stream documents to using the preceding mapping:

A change stream document originating from the myDb database and the myColl collection publishes to the topicOne topic.
A change stream document originating from the myDb database and the newColl collection publishes to the topicTwo.newColl topic.
A change stream document originating from dropping the myDb database, which does not bear a collection name, publishes to the topicTwo topic.

You can use the "*" wildcard character to match change stream document namespaces.

Example

The following mapping instructs the connector to publish all change stream documents to the topicThree topic:

topic.namespace.map={"*": "topicThree"}

Default: ""

Accepted Values: A valid JSON object

topic.separator

Type: string

Description:
Specifies the string the connector uses to concatenate the values used
to create the name of your topic. The connector publishes records to a
topic with a name formed by concatenating the values of the following fields
in the following order:

topic.prefix
database
collection
topic.suffix

Example

The following configuration instructs the connector to publish change stream documents from the coll collection of the db database to the prefix-db-coll topic:

   topic.prefix=prefix
   database=db
   collection=coll
   topic.separator=-

Important

Topic Separator and Topic Namespace Map

When you use the topic.separator property, keep in mind that it does not affect how you define the topic.namespace.map property. The topic.namespace.map property uses MongoDB namespaces which you must always specify with a "." character to separate the database and collection name.

Default: "."
Accepted Values: A string

topic.mapper

Type: string

Description:
The Java class that defines your custom topic mapping logic.

Default: com.mongodb.kafka.connect.source.topic.mapping.DefaultTopicMapper
Accepted Values: Valid full class name of an implementation
of the TopicMapper
class.

← MongoDB Source Connection Properties Change Stream Properties →

Kafka Topic Properties

Overview

Settings

Tip

See also:

Tip

See also:

Example

Example

Example

Important

Topic Separator and Topic Namespace Map

Kafka Topic Properties.css-134mg1q{-webkit-align-self:center;-ms-flex-item-align:center;align-self:center;padding:0 10px;visibility:hidden;}.css-6vrlzm{border-radius:0!important;display:initial!important;margin:initial!important;}.css-1l4s55v{margin-top:-175px;position:absolute;padding-bottom:2px;}

Overview

Settings

Tip

See also:

Tip

See also:

Example

Example

Example

Important

Topic Separator and Topic Namespace Map

Kafka Topic Properties