Set Up Data Federation

This tutorial demonstrates how to create a federated database instance in Atlas from Kubernetes configuration files with Atlas Kubernetes Operator. The federated database instance in this tutorial connects an AWS S3 bucket and an Atlas cluster.

Prerequisites

This tutorial requires:

A running Kubernetes cluster with Atlas Kubernetes Operator deployed, including an AtlasProject Custom Resource.
An AWS user account with permissions to modify IAM roles.
The Atlas CLI.

Procedure

Important

Custom Resources Definitions Take Priority

Atlas Kubernetes Operator uses custom resource configuration files to manage your Atlas configuration. Each custom resource definition overrides settings specified in other ways such as in the Atlas UI. If you delete a custom resource, Atlas Kubernetes Operator deletes the object from Atlas unless you use annotations to skip deletion. To learn more, see the Create and Update Process and the Delete Process.

Create the AWS IAM role in Atlas

Run the following Atlas CLI command to create the new AWS IAM role in Atlas. Replace the following placeholder with your value:

Placeholder	Description
`PROJECT-ID`	Unique 24-character hexadecimal string that identifies the Atlas project to use.

atlas cloudProviders accessRoles aws create --projectId <PROJECT-ID>

Note the returned field values RoleID, Atlas AWS Account ARN, and Unique External ID:
```
AWS IAM role '<RoleID>' successfully created.
Atlas AWS Account ARN: <AtlasAWSAccountARN>
Unique External ID: <AtlasAssumedRoleExternalID>
```

Modify your AWS IAM role trust policy.

Log in to your AWS Management Console.
Navigate to the Identity and Access Management (IAM) service.
Select Roles from the left-side navigation.
Click on the existing IAM role you wish to use for Atlas access from the list of roles.
Select the Trust Relationships tab.
Click the Edit trust relationship button.

Edit the Policy Document. Add a new Statement object with the following content.

Note

Replace the highlighted lines with values returned in the previous step.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "AWS":"<atlasAWSAccountArn>"
         },
         "Action":"sts:AssumeRole",
         "Condition":{
            "StringEquals":{
               "sts:ExternalId":"<atlasAssumedRoleExternalId>"
            }
         }
      }
   ]
}

Click the Update Trust Policy button.

Create the `AtlasDataFederation` custom resource.

Run the following command to create the AtlasDataFederation Custom Resource. The roleId should match the value returned for RoleID in the previous step and the spec.projectRef.name should match the name of your AtlasProject Custom Resource:

cat <<EOF | kubectl apply -f -
apiVersion: atlas.mongodb.com/v1
kind: AtlasDataFederation
metadata:
  name: my-federated-deployment
spec:
  projectRef:
    name: my-project
    namespace: default
  cloudProviderConfig:
    aws:
      roleId: 12345678
      testS3Bucket: my-bucket
  dataProcessRegion:
    cloudProvider: AWS
    region: OREGON_USA
  name: my-fdi
  storage:
    databases:
    - collections:
      - dataSources:
        - allowInsecure: false
          collection: my-collection
          collectionRegex:
          database: my-database
          databaseRegex:
          defaultFormat: ".avro"
          path: /
          provenanceFieldName: string
          storeName: my-data-store
          urls:
          - string:
        name: my-collection-mdb
      maxWildcardCollections: 100
      name: my-database-mdb
      views:
      - name: my-view
        pipeline:
        source: my-source-collection
    stores:
    - name: my-store
      provider: S3
      additionalStorageClasses:
      - STANDARD
      bucket: my-bucket
      delimiter: /
      includeTags: false
      prefix: data-
      public: false
      region: US_WEST_1
EOF

Check the status of your federated database instance.

Run the following command until you recieve a True response, which indicates the database user is ready:

kubectl get atlasdatafederation my-federated-deployment -o=jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

Take the Next Steps

To configure private endpoints for your federated database instance, see Manage Private Endpoints.

Back

Import Projects

Atlas Search