databases

array

required

Array of objects that define the mapping between each federated database instance store defined in stores and online archives. Each object represents a database, its collections, and, optionally, any views on the collections. Each database can have multiple collections and views objects.

databases.[n].name

string

required

Name of the database to which Atlas Data Federation maps the data contained in the data store.

databases.[n].collections

array

required

Array of objects where each object represents a collection and data sources that map to a stores federated database instance store.

databases.[n].collections.[n].name

string

required

Name of the collection to which Atlas Data Federation maps the data contained in each databases.[n].collections.[n].dataSources.[n].storeName. Each object in the array represents the mapping between the collection and an object in the stores array.

You can generate collection names dynamically by specifying * for the collection name. To dynamically generate collection names, you must also specify the following:

• databases.[n].collections.[n].dataSources.[n].datasetPrefix
• databases.[n].collections.[n].dataSources.[n].trimLevel

For wildcard collections, Atlas Data Federation maps a dataset name to a collection name first by splitting the namespace into a list of fields on the $ delimiter, then by trimming a number of fields from the left of the list, and finally by combining the remaining fields using _.

databases.[n].collections.[n].dataSources

array

required

Array of objects where each object represents a federated database instance store in the stores array to map with the collection. You can specify multiple dataSources for a wildcard collection only if all the dataSources for the collection map to the online archive stores.

databases.[n].collections.[n].dataSources.[n].storeName

string

required

Name of a federated database instance store to map to the <collection>. Must match the name of an object in the stores array.

databases.[n].collections.[n].dataSources.[n].datasetName

string

required

Name of the online archive dataset to map with the collection. The datasetName is in the following format:

<version>$<type>$<subtype>$<clusterName>$<dbName>$<collectionName>$<archiveId>

For example, consider the following online archive name.

v1$atlas$archive$clusterName$dbName$collections$archiveId

Here:

• v1 is the version

• atlas is the type of data source

• archive is the subtype

• clusterName is the name of the Atlas cluster

• dbName is the name of the database on the Atlas cluster

• collection is the the collections in the database

• archiveId is the unique identifier of the archived data

IMPORTANT: For a non-wildcard collection, you can't specify this option if you specify the databases.[n].collections.[n].dataSources.[n].datasetPrefix option because Atlas Data Federation automatically generates collections for the latest dataset using the name specified through databases.[n].collections.[n].dataSources.[n].datasetPrefix option.

databases.[n].collections.[n].dataSources.[n].datasetPrefix

string

Required for wildcard collections.

Dataset name prefix to match against the online archive dataset name. When you set this for wildcard collections, Atlas Data Federation maps collections to only to the online archive dataset names whose prefix match the value specified here.

If you specify this setting for a non-wildcard collection, Atlas Data Federation maps only the latest dataset (for the most recently captured snapshot) to the collection. You can't specify the databases.[n].collections.[n].dataSources.[n].datasetName also if you specify this option for a non-wildcard collection.

For example:

{
  ...,
  "name": "myFederatedDbCollection",
  "dataSources": [
  {
     "storeName": "aws-dl-store",
     "datasetPrefix": "v1$atlas$archive$MyCluster$MyDB$MyArchiveId"
     }
   ]
}

databases.[n].collections.[n].dataSources.[n].trimLevel

integer

required

Unsigned integer that specifies how many fields of the dataset name to trim from the left of the dataset name before mapping the remaining fields to a wildcard collection name. Value must be greater than 0 and less than 7. You can set this setting for wildcard collections only.

You can't configure this setting using the Visual Editor in the Atlas UI. Therefore, this setting defaults to trim level 5 for configurations using the Visual Editor.

databases.[n].collections.[n].dataSources.[n].provenanceFieldName

string

required

Name for the field that includes the provenance of the documents in the results. If you specify this setting in the storage configuration, Atlas Data Federation returns the following fields for each document in the result:

You can't configure this setting using the Visual Editor in the Atlas UI.

databases.[n].collections.[n].dataSources.[n].maxDatasets

integer

required

Unsigned integer that specifies the maximum number of datasets from which to dynamically generate collections for the data source. You must provide a value greater than 0. You can set this setting for wildcard collections only. Atlas Data Federation returns datasets in reverse lexicographical order.

IMPORTANT: You can't configure this setting using the Visual Editor in the Atlas UI. Therefore, Atlas Data Federation configuration doesn't include a limit on the number of datasets for configurations using the Visual Editor.