How to copy a Mongo DB collection to a Kafka topic

I would like to copy a Mongo DB collection to a Kafka topic. Can this be done with the the MongoDB Kafka Connect source connector? Or it can only be used to capture changes that are made to the Mongo DB?

1 Like

Hi @Thomas_Bishop1

Yes, of course that’s possible. What you do is configure the source connector such that it will essentially take a snapshot of all data currently residing in a collection and after this finished, it will apply all changes that happened since the copy process started. Afterwards, it will continuously write further changes into the kafka topic.

See the copy.existing.* config properties in the documentation for the source connector https://docs.mongodb.com/kafka-connector/current/kafka-source/#source-connector-configuration-properties

Have fun!

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.