I am migrating data from MySQL to Mongo Atlas (One time migration).
Source URI:
jdbc:mysql://jamie:*****@localhost/hanndb?sslMode=disabled
Destination URI:
mongodb+srv://username:*****@somecluster.mongodb.net/database
The source URL is connected through a SSL Tunnel. I can connect to the MySQL db using MySQL Workbench and can execute queries.
The destination URL is on Mongo Atlas and I can connect using Compass.
In the migrator wizard I have confirmed using the “Test Connection” buttons. However once I execute the migration job I get the following error. I’m unsure what this error message means.
Hi @Jamie_Archibald, thanks for trying Relational Migrator, and sorry to hear you’re having problems.
When you say SSL tunnel, do you mean SSH tunnel? We haven’t tested this before, but if configured properly it should ideally be transparent to the drivers and workloads.
You could try looking in the migrator.log file to see if it has any additional details to help you troubleshoot.
There are also various SO articles describing problems and solutions with accessing databases over SSH tunnels that may be relevant (e.g. this one).
Tom
Hi Tom.
My database is an RDS MySQL instance hosted in AWS in a private subnet. I have a jump box in pubnet subnet which is internet facing.
I use the following command to setup and SSH tunnel (sorry typo, NOT SSL) to the jump server.
ssh -N -L 3306:hanndb-instance-1.caxokwegqpck.ca-central-1.rds.amazonaws.com:3306 -i “hann-ec2-server.pem” ec2-user@ec2-server.ca-central-1.compute.amazonaws.com
I’m able to connect both MySQL workbench and use telnet to connect to the database server on localhost:3306.
The Test connection works as well in Migrator but once the migration job runs it fails with that above error.
Thanks, can you send the full errors from the logfile?
Also this post suggests there can be issues with Java apps connecting over localhost - have you tried using the proper IP or 127.0.0.1?
1 Like
Hi Tom.
I would prefer not to post log files publicly. Is there a way I can post this privately for only you to review. Once the solution is found I’m happy to ensure it’s posted here.
I should mention the Relational Migrator was able connect to my RDS database and inspect the schema, where I was able to do the mapping to Mongo. However once the Sync Job was ran I got the error above. I also tried with both localhost and 127.0.0.1 and received the same error output.
I would like to mention to the product team that the DB Relational Migirator is a great tool but the Database connections setup is a bit confusing. It’s strange to me you can modify the Database connection string AND also the username / password database. In other apps it’s one or the other.
Kind Regards.
Jamie
Nevermind the logs aren’t sensitive.
FYI I can’t seem to post logs in here, getting a character limit and sometimes 403.
Here is a document with the logs
Thanks for sending the logs Jamie. It looks like this is a pretty low-level problem around the behaviour of the driver and database over the SSH tunnel. Since we don’t have the same setup we can’t reproduce this, but it does seem like there are a number of people reporting the same error in other apps and a few suggestions, e.g. MySQL SSH error: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure · Issue #3941 · metabase/metabase · GitHub
Regarding why you can change the database connection during sync, a lot of users connect to a development database when creating the project, but eventually want to migrate a staging or production database. As long as the same schema exists on all servers, you can use a different connection each time you run a job.
Tom
It seems odd to me that I was able to connect to the database and fetch the schema but not do a sync job. I’ll try another method (maybe VPN).
