Atlas private endpoint and debezium connector for kafka connect

Hi guys,

we started to use atlas mongodb in our project with combination of kafka connect + debezium source connector.

We use private endpoints for services and debezium connection string also uses private endpoints.
The way that debezium works is that it reads config database’s shards collection
But entries in shards collection contain adresses that are only available using public endpoints which are blocked in our setup.

Basically I am able to connect to shard directly using connection string in shards collection where it is allowed to access using public links. But obviously debezium source can’t connect to shard because we only allow private links for our services (debezium included).

Is it possible for Atlas mongo to have private endpoint accessible uris in shards collection in config database?

Thanks

Hi Yehor,

If the connector you’re connecting with needs direct shard (mongod) level access instead of working through the mongos (sharded cluster router) tier, you will need to connect via public IP or network peering. Note that you can connect via private endpoints and also one or both of those other options for different parts of your application. Note that I am not familiar with Debezium but as a general rule if the connection isn’t going through a mongos there is risk of accessing orphan data which could lead to data quality issues.

An alternative strategy could be to explore using MongoDB’s Kafka Connector which can connect via private endpoints through the mongos directly.

Cheers
-Andrew