Can I customize Kafka topic from MongoSourceConnector?

[Hyunsang_h]

I want to send Kafka messages to one topic from several MongoSourceConnectors.
That is, what I want: ( MongoSourceConnector : Kafka topic = N : 1 )

However, I know that Kafka messages from MongoSourceConnector are created automatically like {prefix}.{dbName}.{collectionName} , so it seems to be MongoSourceConnector : Kafka topic = 1 : 1 relation.

Can I customize the topic name from each MongoSourceConnector?

[Ross_Lawley]

Hi @Hyunsang_h ,

Good question! I’m pleased to let you know that you can configure the topic mapping.

By default the mapping is:

<prefix>.<dbName>.<collectionName>.<suffix>

However, you can configure overrides like so:

topic.prefix=""
topic.suffix=""
topic.namespace.map={"db1.coll1": "db1Collection1Topic", "db2": "everythingFromDb2", "*": "everyThingElseTopic"}

[1] https://docs.mongodb.com/kafka-connector/current/kafka-source/#topic-naming-examples

I realise that the documentation is lacking all the examples, so I have filed a ticket to get it updated.

Ross

[Hyunsang_h]

Thank you sooooooo much!!