Docs Menu

Atlas Cluster

On this page

  • Example Configuration for Atlas Data Store
  • Configuration Format
  • stores
  • databases

Atlas Data Federation supports Atlas clusters as federated database instance stores. You must define mappings in your federated database instance to your Atlas cluster to run queries against your data.

Important

Information in your storage configuration is visible internally at MongoDB and stored as operational data to monitor and improve the performance of Atlas Data Federation. So, we recommend that you do not use PII in your configurations.

Note

To use Atlas as a federated database instance store, Data Federation requires a serverless instance, or an M10 or higher cluster.

Example

Consider a M10 or higher Atlas cluster named myDataCenter containing data in the metrics.hardware collection. The metrics.hardware collection contains JSON documents with metrics derived from the hardware in a datacenter. The following configuration:

  • Specifies the Atlas cluster named myDataCenter in the specified project as a federated database instance store.
  • Maps documents from the metrics.hardware collection in the Atlas cluster to the dataCenter.inventory collection in the storage configuration.
{
"stores" : [
{
"name" : "atlasClusterStore",
"provider" : "atlas",
"clusterName" : "myDataCenter",
"projectId" : "5e2211c17a3e5a48f5497de3"
}
],
"databases" : [
{
"name" : "dataCenter",
"collections" : [
{
"name" : "inventory",
"dataSources" : [
{
"storeName" : "atlasClusterStore",
"database" : "metrics",
"collection" : "hardware"
}
]
}
]
}
]
}

Atlas Data Federation maps all the documents in the metrics.hardware collection to the dataCenter.inventory collection in the storage configuration.

Users connected to the federated database instance can use the MongoDB Query Language and supported aggregations to analyze data in the Atlas cluster through the dataCenter.inventory collection. When you run queries, the query first goes to Atlas Data Federation. Therefore, if you run aggregation queries that are supported by your Atlas cluster but not by Atlas Data Federation, the queries will fail. To learn more about supported and unsupported commands in Data Federation, see Supported MongoDB Commands.

Tip
See also:

The federated database instance configuration has the following format:

1{
2 "stores" : [
3 {
4 "name" : "<string>",
5 "provider": "<string>",
6 "clusterName": "<string>",
7 "projectId": "<string>",
8 "readPreference": {
9 "mode": "<string>",
10 "tagSets": [
11 [{"name": "<string>", "value": "<string>"},...],
12 ...
13 ],
14 "maxStalenessSeconds": <int>
15 }
16 }
17 ],
18 "databases" : [
19 {
20 "name" : "<string>",
21 "collections" : [
22 {
23 "name" : "<string>",
24 "dataSources" : [
25 {
26 "storeName" : "<string>",
27 "database" : "<string>",
28 "collection" : "<string>",
29 "collectionRegex" : "<string>"
30 }
31 ]
32 }
33 ],
34 "views" : [
35 {
36 "name" : "<string>",
37 "source" : "<string>",
38 "pipeline" : "<string>"
39 }
40 ]
41 }
42 ]
43}
stores
The stores object defines each data store associated with the federated database instance. The federated database instance store captures files in documents in Atlas cluster. federated database instance can only access data stores defined in the stores object.
databases
The databases object defines the mapping between each federated database instance store defined in stores and MongoDB collections in the databases.
1"stores" : [
2 {
3 "name" : "<string>",
4 "provider" : "<string>",
5 "clusterName" : "<string>",
6 "projectId": "<string>",
7 "readPreference": {
8 "mode": "<string>",
9 "tagSets": [
10 [{"name": "<string>", "value": "<string>"},...],
11 ...
12 ],
13 "maxStalenessSeconds": <int>
14 }
15 }
16]
stores

Array of objects where each object represents a data store to associate with the federated database instance. The federated database instance store captures files in an S3 bucket, documents in Atlas cluster, or files stored at publicly accessible URLs. Atlas Data Federation can only access data stores defined in the stores object.

stores.[n].name

Name of the federated database instance store. The databases.[n].collections.[n].dataSources.[n].storeName field references this value as part of mapping configuration.

Note

To use Atlas as a federated database instance store, Data Federation requires a serverless instance, or an M10 or higher cluster.

stores.[n].provider

Defines where the data is stored. Value must be atlas for a collection in an Atlas cluster.

stores.[n].clusterName

Name of the Atlas cluster on which the store is based. The cluster must exist in the same project as your federated database instance. The source field on the data partition is the name of the Atlas cluster.

stores.[n].projectId

Unique identifier of the project that contains the Atlas cluster on which the store is based.

stores.[n].readPreference

Optional. Cluster read preference, which describes how to route read requests to the cluster.

Example

The following readPreference setting specifies secondary mode and ANALYTICS nodeType.

{
...
"stores": [
{
"provider": "atlas",
"clusterName": <CLUSTER_NAME>,
"name": <STORE_NAME>,
"projectId": <PROJECT_ID>,
"readPreference": {
"mode": "secondary",
"tagSets": [
[
{
"name": "nodeType",
"value": "ANALYTICS"
}
],
...
]
}
}
]
}
stores.[n].readPreference.mode

Optional. Read preference mode that specifies which replica set member to route the read requests to. Value can be one of the following:

  • primary - to route all read requests to the replica set primary
  • primaryPreferred - to route all read requests the replica set primary and to secondary members only if primary is unavailable
  • secondary - to route all read requests to the secondary members of the replica set
  • secondaryPreferred - to route all read requests to the secondary members of the replica set and the primary on sharded clusters only if secondary members are unavailable
  • nearest - to route all read requests to random eligible replica set member, irrespective of whether that member is a primary or secondary
stores.[n].readPreference.tagSets

Optional. Arrays of tag sets or tag specification documents that contain name and value pairs for the replica set member. If specified, Atlas Data Federation routes read requests to replica set member or members that are associated with the specified tags. To learn more, Read Preference Tag Sets.

Note

Atlas Data Federation doesn't support tagSets for sharded clusters.

stores.[n].readPreference.maxStalenessSeconds

Optional. Maximum replication lag, or “staleness”, for reads from secondaries. To learn more about maxStalenessSeconds, see Read Preference maxStalenessSeconds.

1"databases" : [
2 {
3 "name" : "<string>",
4 "collections" : [
5 {
6 "name" : "<string>",
7 "dataSources" : [
8 {
9 "storeName" : "<string>",
10 "database" : "<string>",
11 "collection" : "<string>",
12 "collectionRegex" : "<string>"
13 }
14 ]
15 }
16 ]
17 }
18]
databases

Array of objects where each object represents a database, its collections, and, optionally, any views on the collections. Each database can have multiple collections and views objects.

databases.[n].name

Name of the database to which Atlas Data Federation maps the data contained in the data store. You can generate databases dynamically by specifying * for the database name. Dynamically generated databases:

  • Can exist alongside explicitly defined databases. However, Atlas Data Federation won't dynamically generate databases with names that conflict with explicitly defined databases in the storage configuration.
  • Can only be from a single Atlas cluster. Atlas Data Federation won't dynamically generate databases from multiple Atlas clusters or other data stores.
databases.[n].collections

Array of objects where each object represents a collection and data sources that map to a stores federated database instance store. For dynamically generated databases, you can define only one wildcard (*) collection object in the storage configuration.

databases.[n].collections.name

Name of the collection to which Atlas Data Federation maps the data contained in each databases.[n].collections.[n].dataSources.[n].storeName. Each object in the array represents the mapping between the collection and an object in the stores array.

You can generate collection names dynamically by specifying * for the collection name and omitting the collection field. To dynamically generate wildcard (*) collections for dynamically generated wilcard (*) databases, specify the databases.[n].collections.[n].dataSources.[n].storeName option and omit the databases.[n].collections.[n].dataSources.[n].database option. Note that for dynamically generated databases, you can define only one wildcard (*) collection object in the storage configuration.

For wildcard (*) collections, you can also define regex patterns using databases.[n].collections.[n].dataSources.[n].collectionRegex field to filter the collections only.

databases.[n].collections.[n].dataSources

Array of objects where each object represents a stores federated database instance store to map with the collection.

databases.[n].collections.[n].dataSources.[n].storeName

Name of a federated database instance store to map to the <collection>. Must match the name of an object in the stores array.

databases.[n].collections.[n].dataSources.[n].database

Name of the database on the Atlas cluster that contains the collection. You must omit this parameter to create a wildcard (*) collection for a wilcard (*) database.

databases.[n].collections.[n].dataSources.[n].collection

Name of the collection in the Atlas cluster on which the federated database instance store is based. You must omit this setting for:

  • Creating a wildcard (*) collection.
  • Creating wildcard collection names that match regex patterns.
databases.[n].collections.[n].dataSources.[n].collectionRegex

Optional. Regex pattern to use for creating the wildcard (*) collection. If you specify this field, the federated database instance instance only contains collections with names that match the specified regular expression. Collections in the federated database instance storage configuration use their original name in the Atlas cluster.

To use regex patterns for wildcard (*) collection names, you must not specify collection. To learn more about the regex syntax, see Go programming language.

databases.[n].views

Array of objects where each object represents an aggregation pipeline on a collection. To learn more about views, see Views.

databases.[n].views.[n].name

Name of the view.

databases.[n].views.[n].source

Name of the source collection for the view.

databases.[n].views.[n].pipeline

Aggregation pipeline stage(s) to apply to the source collection.

←  Data Federation AWS S3 LimitationsCreate From the UI →
Give Feedback
© 2022 MongoDB, Inc.

About

  • Careers
  • Investor Relations
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2022 MongoDB, Inc.