Connect to MongoDB Atlas through SSH Tunnel from Spark

MongoDB Atlas
, , ,
1

Hi, I can connect to MongoDB through SSH tunnel using MongoDB Compass, but when I try to connect from pyspark, setting up the tunnel using sshtunnel libraries it crashes.

Without the tunnel I can connect OK, but not using the tunnel. Although the tunnel works fine with other datasources like MySQL but it doesn’t work when try to connect to Altlas.

My code is:

    mypkey = paramiko.RSAKey.from_private_key_file(ssh_file)
    tunnel = SSHTunnelForwarder((ssh_host,int(ssh_port)), ssh_username=ssh_user, ssh_pkey=mypkey, remote_bind_address=(jdbcHostname, int(jdbcPort))) 
    tunnel.start()
        
    jdbcUrl = "mongodb+srv://{0}:{1}@{2}/?retryWrites=true&w=majority".format(jdbcUsername, jdbcPassword, jdbcHostname)

The error message is a Timeout… it seems like it doesn’t find the redirected port, but mongodb+srv does not allow port specification:

mongodb+srv://***:***@mongodb_host/?retryWrites=true&w=majority

com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]

Any help would be appreciated. Thanks in advance!

3

Hi @Abel_Martinez1 and welcome to the MongoDB community forum!!

Thank you for sharing the above information. However, to understand the issue better, could you help me with a few understandings related to the issue being observed:

Is this a local MongoDB deployment or an Atlas deployment ?
Could you also help to provide the actual error messages you observed if there’s any?

Is the MySQL deployment and on-prem or is this deployed on a cloud provider ?
Also, can you confirm if you are successfully able to connect without the SSH tunnelling.

Please refer to the official documentation on Spark Connector With MongoDB for more information.

Best Regards
Aasawari

4

Hi, sorry for the delay in the response.

  • It is a Altlas deployment
  • Error:
    com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]
  • The MySQL deployment is running in a AWS server managed by us
  • Yes I can connect without tunneling

BR,

Abel

1 Like