Docs Menu

Kafka Topic Properties

On this page

  • Overview
  • Settings

Use the following configuration settings to specify which Kafka topics the source connector should publish data to.

For a list of source connector configuration settings organized by category, see the guide on Source Connector Configuration Properties.

Name
Description
topic.prefix
Type: string

Description:
The prefix to prepend to database and collection names to generate the name of the Kafka topic on which to publish the data.
Tip
See also:
Default: ""
Accepted Values: A string composed of ASCII alphanumeric characters including ".", "-", and "_"
topic.suffix
Type: string

Description:
The suffix to append to database and collection names to generate the name of the Kafka topic on which to publish the data.
Default: ""
Accepted Values: A string composed of ASCII alphanumeric characters including ".", "-", and "_"
topic.namespace.map
Type: string

Description:
Specifies a JSON mapping between change stream document namespaces and topic names.
Example

The following mapping instructs the connector to perform the following actions:

  • Publish change stream documents originating from the myDb.myColl MongoDB collection to the topicOne Kafka topic.
  • Publish change stream documents originating from the myDb MongoDB database and a collection other than myColl to the Kafka topic topicTwo.<collectionName>.
  • Publish change stream documents originating from the myDb MongoDB database that do not bear a collection name to the topicTwo Kafka topic.
topic.namespace.map={"myDb.myColl": "topicOne", "myDb": "topicTwo"}

The following examples show which topic the connector sends different change stream documents to using the preceding mapping:

  • A change stream document originating from the myDb database and the myColl collection publishes to the topicOne topic.
  • A change stream document originating from the myDb database and the newColl collection publishes to the topicTwo.newColl topic.
  • A change stream document originating from dropping the myDb database, which does not bear a collection name, publishes to the topicTwo topic.
You can use the "*" wildcard character to match change stream document namespaces.
Example

The following mapping instructs the connector to publish all change stream documents to the topicThree topic:

topic.namespace.map={"*": "topicThree"}
Default: ""
Accepted Values: A valid JSON object
topic.separator
Type: string

Description:
Specifies the string the connector uses to concatenate the values used to create the name of your topic. The connector publishes records to a topic with a name formed by concatenating the values of the following fields in the following order:
  1. topic.prefix
  2. database
  3. collection
  4. topic.suffix
Example

The following configuration instructs the connector to publish change stream documents from the coll collection of the db database to the prefix-db-coll topic:

topic.prefix=prefix
database=db
collection=coll
topic.separator=-
Important
Topic Separator and Topic Namespace Map

When you use the topic.separator property, keep in mind that it does not affect how you define the topic.namespace.map property. The topic.namespace.map property uses MongoDB namespaces which you must always specify with a "." character to separate the database and collection name.


Default: "."
Accepted Values: A string
topic.mapper
Type: string

Description:
The Java class that defines your custom topic mapping logic.

Default: com.mongodb.kafka.connect.source.topic.mapping.DefaultTopicMapper
Accepted Values: Valid full class name of an implementation of the TopicMapper class.
←  MongoDB Source Connection PropertiesChange Stream Properties →

On this page

Give Feedback
© 2022 MongoDB, Inc.

About

  • Careers
  • Investor Relations
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2022 MongoDB, Inc.