Create a Sync Job
Sync jobs are the worker processes responsible for transferring your data and schemas from a relational database to MongoDB. Create a new sync job from the Data Migration tab.
This task also applies to Apache Kafka supported continuous sync jobs. There is no procedural difference introduced when using Kafka. For more details, see Integrating Relational Migrator with Kafka.
About this Task
The URI credentials you provide when creating a sync job do not need to be the same as the credentials used when creating your project.
Continuous sync jobs using Confluent Cloud Kafka may take additional time due to the provisioning of backend resources on Confluent Cloud. This process typically takes an additional two to five minutes the first time a CDC sync job runs.
You can recover a continuous sync job within 24 hours of failure. For details, see Recover a Sync Job.
Before you Begin
Create one or more mapping rules in your Relational Migrator project.
Prepare a MongoDB URI and credentials that have read/write permissions on your destination database.
Steps
Enter the relational database connection details
On the Connect Source DB form, enter the connection details to create the JDBC URI for your relational database.
2a. Select a database type from the Database type drop-down.
2b. Enter a host IP or DNS name in the Host text box.
2c. Enter a port number in the Port text box.
2d. Enter a database name in the Database text field.
Depending on your relational database, this behavior varies:
Database Type | Behavior |
|---|---|
Oracle | You must enter a database name and a Service ID or SID. |
SQL Server | Enter a database name, or leave blank to load all databases. |
MySQL | Enter a database name, or leave blank to load all databases. |
Postgres | Leaving database name blank loads schemas from the default database. |
Sybase ASE | Leaving database name blank loads schemas from the default database. |
DB2 | You must enter a database name. |
2e. Enter a username in the Username text box.
2f. Enter a password in the Password text box.
2g. (Optional) Click the Save a password for this session checkbox.
2h. Click the SSL toggle switch to enable or disable SSL and select an SSL mode.
2i. Click Connect
Note
Connection and SSL details depend on the database type you are connecting to. In addition to the generic connection properties listed above, you may also need to select an Identifier for Oracle and Authentication for SQL Server.
If you want to specify the JDBC URI manually, click the Enter URI manually toggle switch on the Connect SourceDB form. For details, see Relational Database Connection Strings.
Enter the source MongoDB connection details
To provide the MongoDB connection details, select an Atlas cluster or provide a MongoDB connection string (URI). If you're logged in with Atlas, the default option is to Select a cluster. If you aren't logged in with Atlas, the Select a cluster option is disabled.
On the Connect Destination DB form, select an Atlas Cluster from the drop-down. Clusters are displayed in a three-level hierarchy: Organization > Project > Cluster, organized alphabetically. Only the first 100 clusters you are authorized to access are shown.
If you leave any of the form fields for Database, Username, or Password blank, the values from the Atlas cluster's metadata are used.
Click Connect.
On the Connect Destination DB form, input your MongoDB connection string (URI).
If you leave any of the form fields for Database, Username, or Password blank, the values specified in the URI are used.
Click Connect.
Select your migration options
On the Migration Options form, select your Migration Options:
Migration Option | Description |
|---|---|
Mode | Defines the type of sync job. |
Drop destination collections before migration | Boolean. Indicates whether Relational Migrator drops a
destination collection before transferring data. |
Stop after errors | Integer. Indicates the number of errors after which Relational
Migrator stops the sync job. |
Verify migrated data | Boolean. If true, the sync engine checks the migrated data
against the source database. Only supported for snapshot mode. |
After you specify the job type on Migration Options form, Relational Migrator conducts various checks to ensure that the database is configured correctly. If a configuration is missing, a banner displays indicating improper database setup and provides a Generate Script button for SQL script download.
This script includes the required configuration statements and additional instructions in the form of comments.
Warning
Before proceeding with starting a sync job:
Download the script.
Carefully review its contents.
Execute the statements.
Follow any commented manual steps.
Review and start your sync job
On the Review Summary form, you can see details about your sync job. These details include the type, the number of tables and rows affected, and the estimated data size of the sync job. To start your sync job, click Start.
Note
For Kafka sync jobs, view deployment details by selecting the job in the History pane on the Data Migration screen and hovering over the icon next to the Create Migration Job button.
Next Steps
Learn More
For detailed information regarding the configuration requirements for each database, see the following: