> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

# Query a Federated Database Instance

You can use the MongoDB Query Language (MQL) on Atlas Data Federation to query and analyze data on your data store. Atlas Data Federation supports most, but not all standard server commands. To learn more about the supported and unsupported MongoDB server commands and aggregation pipeline stages, see [Supported MongoDB Commands.](https://www.mongodb.com/docs/atlas/data-federation/supported-unsupported/mql-support.md#std-label-adf-mql-support)

To query data on your data store, your federated database instance storage [configuration](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf) must contain settings that define:

- Your federated database instance store.

- Federated Database Instance virtual databases and collections that map to your federated database instance store.

You can create or update your federated database instance [storage configuration](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf) for your data store using the Atlas UI Visual Editor or JSON Editor, Atlas Data Federation [CLI](https://www.mongodb.com/docs/atlas/data-federation/admin/manage-federated-database.md#std-label-adf-cli) commands, and Atlas Data Federation [API](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/group/endpoint-data-federation) endpoints. To learn more about federated database instance storage configuration, see [Configure Data Stores for a Federated Database Instance.](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf)

Atlas Data Federation creates the virtual databases and collections you specified in your federated database instance configuration for the data in your data store. When you [connect](https://www.mongodb.com/docs/atlas/data-federation/tutorial/connect.md#std-label-gst-connect-adf) to your federated database instance and [run queries](https://www.mongodb.com/docs/atlas/data-federation/tutorial/run-queries-sample-data.md#std-label-gst-run-queries), Atlas Data Federation processes your queries against the data and returns the query results. You can, optionally, configure [limits](https://www.mongodb.com/docs/atlas/data-federation/query/query-federated-database.md#std-label-adf-query-limits) on the amount of data that Atlas Data Federation processes for your queries to control costs.

**Note:**

Atlas Data Federation uses columnar storage that doesn't preserve the order of fields within documents. Therefore, Atlas Data Federation doesn't support queries that are field-order sensitive, such as an [embedded document equality query](https://www.mongodb.com/docs/manual/reference/operator/query/eq.md#std-label-eq-field-in-embedded-document) or sorting on a document field.

A database user must have one of the following roles to run queries against a federated database instance:

- [readWriteAnyDatabase](https://www.mongodb.com/docs/atlas/security-add-mongodb-users/#readWriteAnyDatabase)

- [readAnyDatabase](https://www.mongodb.com/docs/atlas/security-add-mongodb-users/#readAnyDatabase)

- A [custom role](https://www.mongodb.com/docs/atlas/security-add-mongodb-roles/) with the [`find`](https://www.mongodb.com/docs/manual/reference/privilege-actions.md#mongodb-authaction-find) privilege for the [namespace](https://www.mongodb.com/docs/manual/reference/limits.md#std-label-faq-dev-namespace) against which you want to run queries

You can run up to 30 simultaneous queries on your federated database instance against:

- Data in your S3 (Simple Storage Service) bucket or Azure Blob Storage container.

- Documents in your MongoDB Atlas cluster.

- Archived data in your Online Archive.

- Data in files hosted at publicly accessible URL (Uniform Resource Locator)s.

**See also:**

- [How to Connect to Your Federated Database Instance](https://www.mongodb.com/docs/atlas/data-federation/tutorial/connect.md#std-label-gst-connect-adf)

- [How to Run Queries Against Your Data Using Your Federated Database Instance](https://www.mongodb.com/docs/atlas/data-federation/tutorial/run-queries-sample-data.md#std-label-gst-run-queries)

The following sections contain information pertaining to running queries against data in your data store.

## Querying Data on S3

When [deploying](https://www.mongodb.com/docs/atlas/data-federation/tutorial/deploy.md#std-label-gst-deploy-adf) your federated database instance, if you specified an S3 (Simple Storage Service) bucket with both read and write permissions or AWS (Amazon Web Services) S3 (Simple Storage Service) [s3:PutObject](http://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-objects) permission, you can also save your query results in your S3 (Simple Storage Service) bucket using [$out](https://www.mongodb.com/docs/atlas/data-federation/supported-unsupported/pipeline/out.md#std-label-adf-out-stage) to S3 (Simple Storage Service).

If you successfully create or update an object on your S3 (Simple Storage Service) data store, Data Federation returns the latest version of that object for any subsequent read requests and all list operations of the objects also reflect the changes. If your query contains multiple stages, each stage receives the most recent data available from the data store as that stage is processed.

By default, Atlas Data Federation does not return documents in any specific order for queries on Data Federations for S3 (Simple Storage Service) data stores. Atlas Data Federation reads the partitions concurrently and the underlying storage response order determines which documents Atlas Data Federation returns first, unless you define order using [`$sort`](https://www.mongodb.com/docs/manual/reference/operator/aggregation/sort.md#mongodb-pipeline-pipe.-sort) in your query. For example, if you run the same `findOne()` query twice, you could see different documents, and if you use [`$skip`](https://www.mongodb.com/docs/manual/reference/operator/aggregation/skip.md#mongodb-pipeline-pipe.-skip), different documents might be skipped if [`$sort`](https://www.mongodb.com/docs/manual/reference/operator/aggregation/sort.md#mongodb-pipeline-pipe.-sort) is not used in the query.

You incur "Data Processed" costs for the amount of data that Atlas Data Federation processes to return results for your queries in addition to the "Data Returned" cost for the amount of data that Atlas Data Federation returns. For example, for a 10 GB file, you incur the following "Data Processed" cost in addition to the "Data Returned" cost:

- If you have no partitions, Atlas Data Federation reads the entire file to return results for the query. Therefore, you incur 10 GB of "Data Processed" cost.

- If you have 10 partitions of 1 GB each, Atlas Data Federation targets and reads a single partition. Therefore, you incur 1 GB of "Data Processed" cost.

You can configure [query limits](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits) per federated database instance and for all federated database instances in your project to limit the amount of [processed data](https://www.mongodb.com/docs/atlas/billing/data-federation.md#std-label-atlas-data-federation-billing). To learn more, see [Manage Atlas Data Federation Data Processing Query Limits.](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits)

**Note:**

Data partitioning doesn't guarantee reduced data processing cost. For example, if you run a blank [`$match`](https://www.mongodb.com/docs/manual/reference/operator/aggregation/match.md#mongodb-pipeline-pipe.-match) query, which queries all the data, Atlas Data Federation needs to read the entire collection to return the results for the query regardless of the number of partitions.

## Querying Data on Azure Blob Storage

When [deploying](https://www.mongodb.com/docs/atlas/data-federation/tutorial/deploy.md#std-label-gst-deploy-adf) your federated database instance, you can specify an Azure Blob Storage container with both read and write permissions.

You incur "Data Processed" costs for the amount of data that Atlas Data Federation processes to return results for your queries in addition to the "Data Returned" cost for the amount of data that Atlas Data Federation returns. For example, for a 10 GB file, you incur the following "Data Processed" cost in addition to the "Data Returned" cost:

- If you have no partitions, Atlas Data Federation reads the entire file to return results for the query. Therefore, you incur 10 GB of "Data Processed" cost.

- If you have 10 partitions of 1 GB each, Atlas Data Federation targets and reads a single partition. Therefore, you incur 1 GB of "Data Processed" cost.

You can configure [query limits](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits) per federated database instance and for all federated database instances in your project to limit the amount of [processed data](https://www.mongodb.com/docs/atlas/billing/data-federation.md#std-label-atlas-data-federation-billing). To learn more, see [Manage Atlas Data Federation Data Processing Query Limits.](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits)

## Querying Data on Google Cloud Storage

When [deploying](https://www.mongodb.com/docs/atlas/data-federation/tutorial/deploy.md#std-label-gst-deploy-adf) your federated database instance, you can specify a Google Cloud Storage bucket with both read and write permissions.

You incur "Data Processed" costs for the amount of data that Atlas Data Federation processes to return results for your queries in addition to the "Data Returned" cost for the amount of data that Atlas Data Federation returns. For example, for a 10 GB file, you incur the following "Data Processed" cost in addition to the "Data Returned" cost:

- If you have no partitions, Atlas Data Federation reads the entire file to return results for the query. Therefore, you incur 10 GB of "Data Processed" cost.

- If you have 10 partitions of 1 GB each, Atlas Data Federation targets and reads a single partition. Therefore, you incur 1 GB of "Data Processed" cost.

You can configure [query limits](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits) per federated database instance and for all federated database instances in your project to limit the amount of [processed data](https://www.mongodb.com/docs/atlas/billing/data-federation.md#std-label-atlas-data-federation-billing). To learn more, see [Manage Atlas Data Federation Data Processing Query Limits.](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits)

## Querying Data in Your Atlas Cluster

When you run queries against your Atlas cluster through your federated database instance, Atlas Data Federation sets the `appName`, when querying your cluster, based on the `appName` that you used to connect to your federated database instance. For example, if you connect to your federated database instance with `appName` set to `myApp` (i.e. `appName = "myApp"`), Atlas Data Federation sets the `appName` when connecting to your cluster to the following:

```shell
atlas-data-federation|myApp
```

If you query a collection in Atlas Data Federation that is mapped to only one Atlas collection, Atlas Data Federation acts as a proxy and forwards your query to Atlas. When acting as a proxy, Atlas Data Federation doesn't scan data into its virtual collection to process the query thus improving performance and reducing cost. This optimization is not available for queries on Atlas Data Federation collections that are mapped to multiple Atlas collections.

**Example:**

Consider the following federated database instance storage configuration:

```json
{
  "stores" : [
    {
      "name" : "atlas-store",
      "provider": "atlas",
      "clusterName": "myCluster",
      "projectId": "5e2211c17a3e5a48f5497de3"
    }
  ],
  "databases" : [
    {
      "name" : "atlas-db",
      "collections" : [
        {
          "name" : "foo",
          "dataSources" : [
            {
              "storeName" : "atlas-store",
              "database" : "myFooData",
              "collection" : "foo"
            }
          ]
        },
        {
          "name" : "barbaz",
          "dataSources" : [
            {
              "storeName" : "atlas-store",
              "database" : "myBarData",
              "collection" : "bar"
            },
            {
              "storeName" : "atlas-store",
              "database" : "myBazData",
              "collection" : "baz"
            }
          ]
        }
      ]
    }
  ]
}
```

For the above storage configuration, Atlas Data Federation acts as a proxy for queries on `foo` collection and forwards the queries to Atlas. This performance and cost optimization is not available for queries on `barbaz` collection because `barbaz` is mapped to multiple Atlas collections.

You can also save your query results in your Atlas cluster using [$out](https://www.mongodb.com/docs/atlas/data-federation/supported-unsupported/pipeline/out.md#std-label-adf-out-stage) to Atlas.

If you successfully create or update a document in your collection on the Atlas cluster, Data Federation returns the latest version of that document for any subsequent read requests and all list operations of the collection also reflect the changes. If your query contains multiple stages, each stage receives the most recent data available from the data store as that stage is processed.

Atlas logs your queries against your cluster data in the Atlas cluster audit logs. The log entry for a database user is in the following format:

```sh
<SERVICE_NAME>-<ATLAS_DATA_FEDERATION_INSTANCE_NAME>-<DATABASE_USER_NAME>
```

For example, for a database user configured in Atlas as `"user" : "CN=atlasDataFederation-DataFederation0-test_datafederation0"`, a log entry in the Atlas cluster audit log looks similar to the following:

```sh
{
  "atype" : "authenticate",
  "ts" : { "$date" : "2022-04-29T13:17:54.020+00:00" },
  "local" : { "ip" : "XXXX", "port" : 27017 },
  "remote" : { "ip" : "XXXXX", "port" : 10844 },
  "users" : [ { "user" : "CN=atlasDataFederation-DataFederation0-test_datafederation0", "db" : "$external" } ],
  "roles" : [ { "role" : "backup", "db" : "admin" }, { "role" : "readWriteAnyDatabase", "db" : "admin" }, { "role" : "clusterMonitor", "db" : "admin" }, { "role" : "enableSharding", "db" : "admin" }, { "role" : "atlasAdmin", "db" : "admin" }, { "role" : "dbAdminAnyDatabase", "db" : "admin" } ],
  "param" : { "user" : "CN=atlasDataFederation-DataFederation0-test_datafederation0", "db" : "$external", "mechanism" : "MONGODB-X509" },
  "result" : 0
}
```

**Note:**

The connection mechanism is always `MONGODB-X509` in the Atlas cluster audit logs.

## Querying Data in Your Online Archives

For queries, Atlas Data Federation uses the partitions that you created on fields during the creation of the Atlas Online Archive. The order of fields in the partitions is important, as data is optimized for queries by the first field, followed by the second field, and so on. To query the Online Archive effectively, use all defined partition fields in the order in which they're defined. Queries using all partition fields in the order they were defined target specific partitions and minimize the amount of data they scan.

Atlas Data Federation is less performant in supporting queries on fields that don't have partitions.

## Querying Data at a HTTP (HyperText Transport Protocol) or HTTPS (Secure HyperText Transport Protocol) URL (Uniform Resource Locator)

Data Federation also creates one partition for each URL (Uniform Resource Locator) in your collection. When you [connect](https://www.mongodb.com/docs/atlas/data-federation/tutorial/connect.md#std-label-gst-connect-adf) to your federated database instance and run queries, Data Federation processes your queries against the data and returns the query results.

## Running Federated Queries

You can use Atlas Data Federation to query and analyze a unified view of data in your Atlas cluster, S3 (Simple Storage Service) bucket or Azure Blob Storage container, HTTP (HyperText Transport Protocol) URL, and online archives. For federated queries, your federated database instance storage [configuration](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf) must contain the settings that define:

- Your S3 (Simple Storage Service) or Azure (Microsoft Azure), Atlas, Online Archive, and HTTP (HyperText Transport Protocol) stores.

  **Note:**

  Atlas Data Federation doesn't support federated queries across cloud providers. Therefore, you can't run federated queries against data stored on AWS (Amazon Web Services) S3 (Simple Storage Service) buckets and Azure Blob Storage containers. Regardless of the cloud provider backing your Atlas cluster, you can run federated queries against data on your Atlas cluster and AWS (Amazon Web Services) S3 (Simple Storage Service) bucket or Azure Blob Storage container simultaneously.

- Federated Database Instances with virtual collections that map to your S3 (Simple Storage Service) bucket or Azure Blob Storage, Atlas cluster, Online Archive, and HTTP (HyperText Transport Protocol) stores.

You can create or update your federated database instance [storage configuration](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf) using the Atlas UI Visual Editor or the JSON Editor, Atlas Data Federation [CLI](https://www.mongodb.com/docs/atlas/data-federation/admin/manage-federated-database.md#std-label-adf-cli) commands, and Atlas Data Federation [API](https://www.mongodb.com/docs/api/doc/atlas-admin-api-v2/group/endpoint-data-federation) endpoints. To learn more about federated database instance storage configuration, see [Configure Data Stores for a Federated Database Instance.](https://www.mongodb.com/docs/atlas/data-federation/config/config-data-stores.md#std-label-config-adf)

When you [connect](https://www.mongodb.com/docs/atlas/data-federation/tutorial/connect.md#std-label-gst-connect-adf) to your federated database instance and run federated queries, Data Federation combines data from your Atlas cluster, S3 (Simple Storage Service) bucket or Azure Blob Storage container, and HTTP (HyperText Transport Protocol) URL (Uniform Resource Locator)s in virtual databases and collections and returns a union of data in the results.

## Configuring Query Limits

You can limit the amount of data that Atlas Data Federation processes for your queries to control costs. To limit the amount of data that Atlas Data Federation processes for your queries, you can configure query limits per federated database instance or for all federated database instances in your project. When the amount of processed data reaches any applicable configured limit, Atlas Data Federation won't execute any new queries and returns an error to the client application that a limit has been reached. To learn more, see [Manage Atlas Data Federation Data Processing Query Limits.](https://www.mongodb.com/docs/atlas/data-federation/query/manage-query-limits.md#std-label-adf-manage-query-limits)

## Troubleshooting

**Error:** We are currently experiencing increased query processing wait times for Atlas Data Federation. Our Engineering team is investigating. Normal service will resume shortly, please try again.

Atlas Data Federation returns this error only when Atlas Data Federation can't execute queries because of resource contention. We recommend that you run your queries again.