Set Up Data Federation
On this page
This tutorial demonstrates how to create a federated database instance in Atlas from Kubernetes configuration files with Atlas Kubernetes Operator. The federated database instance in this tutorial connects an AWS S3 bucket and an Atlas cluster.
Prerequisites
This tutorial requires:
A running Kubernetes cluster with Atlas Kubernetes Operator deployed, including an
AtlasProject
Custom Resource.An AWS user account with permissions to modify IAM roles.
The Atlas CLI.
Procedure
Important
Custom Resources No Longer Delete Objects by Default
Atlas Kubernetes Operator uses custom resource configuration
files to manage your Atlas configuration, but as of Atlas Kubernetes Operator 2.0,
custom resources you delete in Kubernetes are no longer deleted in
Atlas. Instead, Atlas Kubernetes Operator simply stops managing those resources.
For example, if you delete an AtlasProject
Custom Resource
in Kubernetes, Atlas Kubernetes Operator no longer automatically deletes the corresponding project
from Atlas, preventing accidental or unexpected deletions. To learn more,
including how to revert this behavior to
the default used prior to Atlas Kubernetes Operator 2.0, see New Default: Deletion Protection in Atlas Kubernetes Operator 2.0.
Create the AWS IAM role in Atlas
Run the following Atlas CLI command to create the new AWS IAM role in Atlas. Replace the following placeholder with your value:
PlaceholderDescriptionPROJECT-ID
Unique 24-character hexadecimal string that identifies the Atlas project to use.atlas cloudProviders accessRoles aws create --projectId <PROJECT-ID> Note the returned field values
RoleID
,Atlas AWS Account ARN
, andUnique External ID
:AWS IAM role '<RoleID>' successfully created. Atlas AWS Account ARN: <AtlasAWSAccountARN> Unique External ID: <AtlasAssumedRoleExternalID>
Modify your AWS IAM role trust policy.
Log in to your AWS Management Console.
Navigate to the Identity and Access Management (IAM) service.
Select Roles from the left-side navigation.
Click on the existing IAM role you wish to use for Atlas access from the list of roles.
Select the Trust Relationships tab.
Click the Edit trust relationship button.
Edit the Policy Document. Add a new
Statement
object with the following content.Note
Replace the highlighted lines with values returned in the previous step.
{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Principal":{ "AWS":"<atlasAWSAccountArn>" }, "Action":"sts:AssumeRole", "Condition":{ "StringEquals":{ "sts:ExternalId":"<atlasAssumedRoleExternalId>" } } } ] } Click the Update Trust Policy button.
Create the AtlasDataFederation
custom resource.
Run the following command to create the
AtlasDataFederation
Custom Resource. The roleId
should
match the value returned for RoleID
in the previous step and the
spec.projectRef.name
should match the name of your
AtlasProject
Custom Resource:
cat <<EOF | kubectl apply -f - apiVersion: atlas.mongodb.com/v1 kind: AtlasDataFederation metadata: name: my-federated-deployment spec: projectRef: name: my-project namespace: default cloudProviderConfig: aws: roleId: 12345678 testS3Bucket: my-bucket dataProcessRegion: cloudProvider: AWS region: OREGON_USA name: my-fdi storage: databases: - collections: - dataSources: - allowInsecure: false collection: my-collection collectionRegex: database: my-database databaseRegex: defaultFormat: ".avro" path: / provenanceFieldName: string storeName: my-data-store urls: - string: name: my-collection-mdb maxWildcardCollections: 100 name: my-database-mdb views: - name: my-view pipeline: source: my-source-collection stores: - name: my-store provider: S3 additionalStorageClasses: - STANDARD bucket: my-bucket delimiter: / includeTags: false prefix: data- public: false region: US_WEST_1 EOF
Take the Next Steps
To configure private endpoints for your federated database instance, see Manage Private Endpoints.