/ /

Migrate a Self-Managed Deployment to Kubernetes

Docs Home

/ /

Deploy Database Resources

Migrate a Self-Managed Deployment to Kubernetes

Docs Home

Management

Controllers for Kubernetes Operator

Deploy Database Resources

Migrate a Self-Managed Deployment to Kubernetes

Migrate a Replica Set to Kubernetes

This procedure migrates any replica set that Ops Manager or Cloud Manager manages into Kubernetes under Kubernetes Operator, using normal MongoDB replication. This includes deployments that run on virtual machines or bare metal, and deployments already managed by a different Kubernetes Operator instance. The migration is live and incremental: you extend the replica set into Kubernetes, promote the Kubernetes members, and then prune the external members. There is no snapshot restore and no mongosync involved.

Before you Begin

Confirm the following before you migrate a replica set:

A Ops Manager or Cloud Manager connection ConfigMap with the keys baseUrl, orgId, and projectName.
An API key Secret with the keys publicKey and privateKey.
The Kubernetes Operator ServiceAccount has batch/jobs permissions (create, get, list, watch, and delete) for the dry-run connectivity Job.
The project contains exactly one deployment.

You also need:

An existing replica set that Ops Manager or Cloud Manager manages.
Exactly one deployment in the Ops Manager or Cloud Manager project.
A Kubernetes cluster with Kubernetes Operator installed.
The kubectl mongodb plugin at a version that matches Kubernetes Operator. Kubernetes Operator enforces this match on every reconcile, including the dry run.
Bidirectional network connectivity and hostname resolution between the virtual machine hosts and the Kubernetes Pods.
Pre-configured TLS certificates that cover all SANs, if the deployment uses TLS.
The SCRAM user passwords in hand. You cannot recover them from the automation config.
A backup taken before you start.
A deployment that is healthy and at goal state.

Keep at least 3 voting members at all times. Kubernetes Operator rejects more than 7 voting members. If you exceed 7, add non-voting Kubernetes members or remove voting external members.
Make only one kind of change at a time: add Kubernetes members, prune external members, or change the votes and priority of only one member at a time. Admission validation rejects mixed changes once migration has started, and also rejects removing Kubernetes members or adding external members mid-migration.
Act only when the deployment is at goal state.
Migrate the primary last. Shifting votes and priority onto a member can trigger an election, and writes can briefly fail while the replica set elects a new primary. Migrating the primary last avoids triggering that election, and the write downtime it causes, until the final step.

Note
You can also trigger a re-election by increasing the priority of a member in Kubernetes.
While spec.externalMembers is non-empty, Kubernetes Operator forces one-member-at-a-time scaling. This is why each change needs its own wait for goal state.

Procedure

Prepare Ops Manager or Cloud Manager access.

Create the project ConfigMap and the API key Secret described in the prerequisites.

Generate the MongoDB custom resource.

Run the kubectl mongodb migrate-to-mck mongodb command:

kubectl mongodb migrate-to-mck mongodb \
  --config-map-name <configmap> \
  --secret-name <secret> \
  --namespace <namespace> \
  -o mongodb-cr.yaml

Inspect the generated file. It contains:

spec.externalMembers, listing the source processes.
spec.members set to 0.
The mongodb.com/migration-dry-run: true annotation, which the plugin always adds.

The command also supports these optional flags:

--certs-secret-prefix: required when TLS is enabled. Sets spec.security.certsSecretPrefix.
--prometheus-secret-name: use when Prometheus is enabled. The Secret must already exist and have a password key.
--resource-name-override: sets metadata.name on the generated resource. The plugin normalizes the replica set name automatically when it is not a valid Kubernetes name, and sets spec.replicaSetNameOverride for you.

Generate the MongoDBUser custom resources.

Complete this step only if you want to migrate your database users. The migration does not require it.

Pre-create one Secret per SCRAM user, each with a password key. Run:

kubectl mongodb migrate-to-mck users \
  --config-map-name <configmap> \
  --secret-name <secret> \
  --namespace <namespace> \
  --users-secrets-file users.csv \
  -o users-cr.yaml

The CSV file maps users to Secrets, one per line, in the form username:database,secret-name. Omit --users-secrets-file to be prompted for each user instead.

The plugin generates X.509 and LDAP users against the $external database. It skips the automation agent user.

If the deployment uses TLS, issue the Kubernetes member certificates.

Issue the Kubernetes member certificates from the same certificate authority that signed the virtual machine certificates.

Kubernetes Operator expects a kubernetes.io/tls Secret named <certsSecretPrefix>-<resourceName>-cert and a CA ConfigMap named <resourceName>-ca that contains both a ca-pem key and an mms-ca.crt key. The certificates need SANs that cover the per-Pod and Service DNS names, and both server auth and client auth usages.

If the source deployment does not use TLS, set net.tls.mode to disabled in Ops Manager or Cloud Manager, on the existing virtual machine deployment, before you migrate. You do not need to set anything in the MongoDB custom resource for this case.

Note

A CA mismatch is caught by the dry run in the next step, by design.

Configure external access.

Before you run the dry run, configure spec.externalAccess so that the virtual machine members can reach the Kubernetes Pods. To let those members resolve the Pods by hostname, you can also set spec.externalAccess.externalDomain. To learn about the fields involved and the DNS requirements, see Network Requirements for Migration.

This setup is user-owned: you configure the LoadBalancers or NodePorts and the DNS records for your environment.

Important

Don't set externalDomain if you use MongoDB Search or Vector Search with this deployment. MongoDBSearch does not support a MongoDB resource that sets externalDomain, and you can't remove the field after you create the cluster. To learn more, see MongoDB Search Interaction with Migration.

Run the dry run and check connectivity.

Apply the generated resource with the mongodb.com/migration-dry-run annotation still present. While the annotation is set, Kubernetes Operator makes no automation config changes and only validates connectivity.

Kubernetes Operator creates a Job named <resourceName>-connectivity-check, which dials every external member and authenticates. The Job deletes itself using ttlSecondsAfterFinished, and the next reconcile re-creates it, so re-validation is automatic. You can fix problems in Kubernetes or in the Ops Manager UI and re-run freely.

The dry run checks Kubernetes-to-virtual-machine reachability (DNS, TLS, firewalls, and member addresses) and credentials, including the __system role on the local database, and the CA when TLS is enabled. It does not check virtual-machine-to-Kubernetes inbound connectivity.

Because that direction depends entirely on your networking setup, no single command can guarantee it. Instead, work through this checklist:

Confirm that the hostnames the Kubernetes members will have, following the pattern <metadata.name>-0.<spec.externalAccess.externalDomain>, are resolvable from the virtual machine members.
Confirm that the Kubernetes node or LoadBalancer IPs are reachable from the virtual machines.

Read the result from status.conditions[type=NetworkConnectivityVerified]:

Validator Job Exit Code	Condition Status	Reason	Meaning
Job still running	`Unknown`	`Running`	The `status.phase` is `ConnectivityCheckRunning`.
`0`	`True`	`NetworkValidationPassed`	All external members are reachable and authenticated.
`2`	`False`	`AuthenticationFailed`	Credentials, the authentication mechanism, or a missing `__system@local` role.
`3`	`False`	`NetworkFailed`	DNS, TLS, timeouts, or unreachable members. Check the Job Pod logs.
`1` or other	`False`	`UnknownError`	Unclassified failure. Check the Job Pod logs.

Failures that occur before the Job starts use the reasons OperatorImageUnknown, BuildStatefulSetOptions, AgentCertSecretFailed, and AgentCertSubject.

Kubernetes Operator removes the NetworkConnectivityVerified condition from status.conditions entirely once no external members remain.

Extend the replica set into Kubernetes.

Remove the dry-run annotation:

kubectl annotate mdb <resourceName> \
  mongodb.com/migration-dry-run-

This is the point at which Kubernetes Operator takes ownership of the Ops Manager project.

Raise spec.members and hand-write spec.memberConfig together.

Warning

Set spec.memberConfig before you raise the member count

By default, new Kubernetes members join as voting members. The CRD defaults are votes: 1 and priority: "1", which let a still-syncing member participate in an election before it has finished its initial sync.

MongoDB recommends that you write one spec.memberConfig entry per new Kubernetes member with votes: 0 and priority: "0" before you raise the member count, so that a still-syncing member cannot win an election. votes is an integer. priority is a string.

For example, to add three Kubernetes members as non-voting:

spec:
  memberConfig:
    - votes: 0
      priority: "0"
    - votes: 0
      priority: "0"
    - votes: 0
      priority: "0"

Wait for full initial sync and goal state. If you generated MongoDBUser resources in the optional step above, apply them now and confirm each one reaches a status.phase of Updated.

Promote the Kubernetes members.

Shift votes and priority onto the Kubernetes members by editing spec.memberConfig. votes is an integer. priority is a string containing a float.

External members keep whatever votes and priority they carry in the source automation config. spec.externalMembers has no votes or priority field.

Wait for goal state.

Prune the external members.

Important

Remove only one entry from spec.externalMembers at a time. Wait for goal state after each removal before you remove the next entry.

For example:

kubectl patch mdb <resourceName> --type=json \
  -p='[{"op":"remove","path":"/spec/externalMembers/0"}]'

You can remove entries but never add them once migration has started.

Watch status.conditions[type=Migrating].reason move through Extending, InProgress, and Pruning:

Reason	Status	Meaning
`Validating`	`True`	The dry-run annotation is set.
`Extending`	`True`	The desired Kubernetes member count exceeds the last reconciled count.
`Pruning`	`True`	The `externalMembers` count dropped below `status.migrationObservedExternalMembersCount`.
`InProgress`	`True`	External members exist but nothing is changing. This is also the reason on the first reconcile.
`MigrationComplete`	`False`	All external members have been removed.

Precedence is Validating > Extending > Pruning > InProgress. A prune that also grows the Kubernetes side reports Extending, which is another reason to make one change at a time.

Pruning and extending are not allowed at the same time.

To script against migration completion, use kubectl wait --for=condition=Migrating=False rather than polling status.phase.

Verify and finish.

Migration is complete when spec.externalMembers is empty, the Migrating condition is False with reason MigrationComplete, and all data lives in persistent volume claims:

kubectl wait --for=condition=Migrating=False mdb/<resourceName>

Update your application connection strings and retire the virtual machines. Kubernetes Operator does not decommission the virtual machines for you.

Connection String Secret

Kubernetes Operator automatically generates a connection string Secret containing no credentials, named <metadata.name>-cluster-connection-string, and keeps it in sync with the active nodes. Point your applications at this Secret instead of hardcoding a connection string.

Availability during Migration

Reads stay available throughout migration. Writes might briefly fail during elections, so your drivers must use retryable writes. Read latency can increase if a read target moves, so run the Kubernetes cluster in the same region as the virtual machines. Long-running cursors against a migrated secondary break, because there is no quiesce mode.

Back

Network Requirements

Sharded Cluster