I am attempting to use the official MongoDB kafka connector https://www.mongodb.com/docs/kafka-connector/current/sink-connector/ as a SINK to move data from an AWS hosted MSK cluster into an ATLAS hosted mongodb.
I’ve configured the client in a way that I THINK should work but I’m getting issues
[Worker-00cbe7dbb4d2fb6dc] [2022-04-27 13:30:21,449] INFO Exception in monitor thread while connecting to server [mydb].mongodb.net:27017 (org.mongodb.driver.cluster:76)
[Worker-00cbe7dbb4d2fb6dc] com.mongodb.MongoSocketOpenException: Exception opening socket
[Worker-00cbe7dbb4d2fb6dc] at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:70)
I’m wondering if anyone out there has successfully done this and if there is a trick somewhere that I’m missing.
This is a development level proof of concept at the moment, so I’ve got a fully-open kafka solution in AWS, the mongodb in atlas has a 0.0.0.0/0 setting so it should be open.
My connection.uri uses a mongodb+srv link
I am able to successfully connect to the mongo instance if I attempt to connect via mongosh from within an EC2 instance on the same AWS cluster, but the MSK Connect workers in AWS are failing.
Would love to hear any insights from others who have successfully done this in the past.