Kafka Connect source collection and sink the topic from source

There is n example of the Kafka Connector here : https://docs.mongodb.com/kafka-connector/master/kafka-docker-example/

basically, your source configuration will look something like:

curl -X POST -H “Content-Type: application/json” --data ’
{“name”: “mongo-source”,
“config”: {
“tasks.max”:“1”,
“connector.class”:“com.mongodb.kafka.connect.MongoSourceConnector”,
“key.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“key.converter.schemas.enable”:false,
“value.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“value.converter.schemas.enable”:false,
“publish.full.document.only”: true,
“connection.uri”:“mongodb://mongo1:27017,mongo2:27017,mongo3:27017”,
“topic.prefix”:“aprefix”,
“database”:“MyDB”,
“collection”:“SomeCollection”
}}’ http://localhost:8083/connectors -w “\n”

the sink would look something like:
curl -X POST -H “Content-Type: application/json” --data ’
{“name”: “mongo-atlas-sink”,
“config”: {
“connector.class”:“com.mongodb.kafka.connect.MongoSinkConnector”,
“tasks.max”:“1”,
“topics”:“aprefix.MyDB.SomeCollection”,
“connection.uri”:“'<>”,
“database”:“DestinationDBName”,
“collection”:“DestinationCollection”,
“key.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“key.converter.schemas.enable”:false,
“value.converter”:“org.apache.kafka.connect.json.JsonConverter”,
“value.converter.schemas.enable”:false
}}’ http://localhost:8083/connectors -w “\n”

1 Like