Hi, I’m running the kafka source connector version 1.8.1 on strimzi connect. However, the copy.existing doesn’t work.
Here is my configuration
{
"name":"mongodb-natvie-source-connector",
"config":{
"mongo.errors.deadletterqueue.topic.name":"mongodb.dev-api.dlq",
"connector.class":"com.mongodb.kafka.connect.MongoSourceConnector",
"errors.log.include.messages":"true",
"mongo.errors.log.enable":"true",
"publish.full.document.only":"true",
"tasks.max":"50",
"batch.size":"100",
"copy.existing.namespace.regex":"collection_x_namespace",
"heartbeat.interval.ms":"3000",
"collection":"collection_y_namespace",
"mongo.errors.tolerance":"none",
"key.converter.schemas.enable":"false",
"database":"dev-api",
"copy.existing.max.threads":"50",
"poll.await.time.ms":"100",
"connection.uri":"***",
"value.converter.schemas.enable":"false",
"name":"mongodb-natvie-source-connector",
"copy.existing":"true",
"value.converter":"org.apache.kafka.connect.json.JsonConverter",
"errors.log.enable":"true",
"key.converter":"org.apache.kafka.connect.json.JsonConverter",
"poll.max.batch.size":"1000"
},
"tasks":[
{
"connector":"mongodb-natvie-source-connector",
"task":0
}
],
"type":"source"
}
When I run the connector and check the metrics the getmore_commands_successful(copy.existing) metric is equal to 2 but there’s nothing written to the topic. The CDC works as expected.
There’s also no error. Is there anything missing or wrong in the configuration?
I’ve also tried running this with regex copy.existing.namespace.regex: “dev-api.(collection_x|collection_y)”
and a topic prefix, none of them works
I also tried replacing the copy.existing.namespace.regex with the collection whose related topic doesn’t exist. I expected that it’d throw a topic not found exception but it didn’t. It seems that these configurations are ignored altogether