MongoDB Atlas Online Archive for Data Tiering is now GA

We’re thrilled to announce that MongoDB Atlas Online Archive is now Generally Available. With Online Archive, you can seamlessly tier your data across Atlas clusters and fully managed cloud object stores, gaining the flexibility to set the perfect price to performance ratio across your data.

Eliminate the need to manually migrate or delete valuable data. Simply set a rule on your Atlas cluster to automate data archival while retaining easy access to query all your data using a single connection string. With this capability, you can bring new and previously cost-prohibitive use cases onto MongoDB Atlas, our first-class managed offering, and manage your entire data lifecycle without replicating or migrating it across multiple systems.

Atlas Online Archive

What is Atlas Online Archive?

Online Archive is a fully managed data tiering solution that allows you to tier data across your "hot" database storage layer and "colder" cloud object storage to maintain queryability while optimizing on cost and performance.

Online Archive is a good fit for many different use cases, including:

  • Insert heavy workloads, where data is immutable and has lower performance requirements as it ages
  • Historical log keeping and time-series datasets
  • Storing valuable data that would have otherwise been deleted using TTL indexes

We’ve received amazing feedback from the community over the past few months while the feature was in beta and we’re now confident in supporting your production workloads. Our users have put the feature through a variety of use cases in production and development workloads which has enabled us to make a wide range of improvements.

Autonomous Archival Management

It's easy to get started with Online Archive and it requires no ongoing maintenance once it’s been set up. In order to activate the feature, you can follow these simple steps:

  1. Navigate to the “Online Archive” tab on your cluster card and begin the setup flow.
  2. Set an archiving rule by selecting a date field, with dot-notation if it’s nested, or creating a custom filter.
  3. Choose commonly queried fields that you want your archival queries to be optimized for, with a few things in mind:
    • Your data will always be “partitioned” by the date field in your archive, but can be partitioned by up to two additional fields as well.
    • The fields that you most commonly query should be towards the top of the list (date can be moved to the top or bottom).
    • Query fields should be chosen carefully as they cannot be changed after the fact and will have a large impact on query performance. Avoid choosing a field that has unique values as it will have negative performance impacts for queries that need to scan lots of data.
  4. And you’re done! MongoDB Atlas will automatically move data off of your cluster and into a more cost-effective storage layer that can still be queried with a single connection string that combines cluster and archive data, powered by Atlas Data Lake.
Atlas Online Archive

What's Next?

Along with announcing Online Archive as Generally Available, we’re excited to share a few additional product enhancements which should be available in the coming months:

  • Custom filters for your archival rules using a non-date based field
  • Support for BYO Key Encryption on your archival data
  • A dedicated connection string for archive-only queries
  • Support for additional time formats
  • Improved performance and stability

Try Atlas Online Archive

Online Archive allows you to right-size your Atlas clusters by storing hot data that is regularly accessed in live storage and moving colder data to a cheaper storage tier. Billing for this feature will include the cost to store data in our fully managed cloud object storage and usage based pricing for querying archive data. We can’t wait to see what new workloads you’ll bring onto MongoDB Atlas with the new flexibility provided by Online Archive!

To get started, sign up for an Atlas account and deploy any dedicated cluster (M10 or higher).

Have questions? Check out the documentation or head over to our community forums to get answers from fellow developers. And if we’re missing a feature you’d like to see, please let us know!

Safe Harbor Statement

The development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB's sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.

Try MongoDB in the Cloud

Create a free account and launch a cluster in minutes!