Atlas Online Archive
New Online Archive with Performance Improvements and Enhanced Metrics
We are excited to announce several new features coming to Atlas Online Archive. With these improvements, customers will observe higher performance, be able to choose which region hosts their archival data, and have greater insight into the data stored in the archive through improved metrics. New storage engine We have optimized Online Archive through the introduction of a new storage engine that captures metadata during archival to power faster query performance. The new underlying storage service minimizes the overall data scan when querying the Online Archive. Additionally, the new storage engine delivers additional benefits in the form of storage optimizations like sorting as well as ongoing rebalancing to deliver consistent performance over time. With this feature, you will experience faster Online Archive querying performance while your overall costs are decreased when querying against the archives. Ability to choose a storage region With the new Online Archives, the users will be able to select the storage region at the time of creating an Online Archive. There will be a feature to choose one of the supported Data Federation regions from the dropdown during the creation process. In addition, we display the closest region so the users can make an informed decision about their storage region. Enhanced metrics With these new enhanced and improved metrics, we are helping customers better understand what data is in their archive such that they can connect to the appropriate endpoint for their use case and have insight into the performance of archival jobs. With these enhanced metrics, users can see a dashboard including the total # of documents in the archive and overall Archive data size (with a tooltip to explain metrics), what rate are archival jobs archiving, Min/Max Date Fields in the archive, and other statistics that are helpful and important for the users to increase the visibility of the archives. Newly created archives Note that only the newly created Online Archives on or after 06/07/2023 will see the performance improvements and enhancements initially. All existing archives created before 06/07/2023 will continue to function as-is (without the new improvements and enhanced metrics). Over the coming year, we will migrate all existing archives created before 06/07/2023 to the new backend storage service. See the documentation for additional information about creating new Online Archives and Enhanced Metrics .
MongoDB Atlas Online Archive for Data Tiering is now GA
We’re thrilled to announce that MongoDB Atlas Online Archive is now Generally Available. Online Archive allows you to seamlessly tier your data across Atlas clusters and fully managed cloud object stores, while retaining the ability to query it through a single endpoint. Reduce storage costs. Set the perfect price to performance ratio on your data. Automate data tiering. Eliminate the need to manually migrate or delete valuable data. Queryable archives. Easily federate queries across live and archival data using a unified connection string. With Online Archive, you can bring new use cases to MongoDB Atlas that were previously cost-prohibitive such as high volume time-series workloads, data archival for auditing purposes, historical log keeping and more. Manage your entire data lifecycle on MongoDB Atlas without replicating or migrating it across multiple systems. What is Atlas Online Archive? Online Archive is a fully managed data tiering solution that allows you to tier data across your "hot" database storage layer and "colder" cloud object storage to maintain queryability while optimizing on cost and performance. Online Archive is a good fit for many different use cases, including: Insert heavy workloads, where data is immutable and has lower performance requirements as it ages Historical log keeping and time-series datasets Storing valuable data that would have otherwise been deleted using TTL indexes We’ve received amazing feedback from the community over the past few months while the feature was in preview and we’re now confident in supporting your production workloads. Our users have put the feature through a variety of use cases in production and development workloads which has enabled us to make a wide range of improvements. Online Archive gives me the flexibility to store all of my data without incurring high costs, and feel safe that I won't lose it. It's the perfect solution. Ran Landau, CTO, Splitit Autonomous Archival Management It's easy to get started with Online Archive and it requires no ongoing maintenance once it’s been set up. In order to activate the feature, you can follow these simple steps: Navigate to the “Online Archive” tab on your cluster card and begin the setup flow. Set an archiving rule by selecting a date field, with dot-notation if it’s nested, or creating a custom filter. Choose commonly queried fields that you want your archival queries to be optimized for, with a few things in mind: Your data will always be “partitioned” by the date field in your archive, but can be partitioned by up to two additional fields as well. The fields that you most commonly query should be towards the top of the list (date can be moved to the top or bottom). Query fields should be chosen carefully as they cannot be changed after the fact and will have a large impact on query performance. Avoid choosing a field that has unique values as it will have negative performance impacts for queries that need to scan lots of data. And you’re done! MongoDB Atlas will automatically move data off of your cluster and into a more cost-effective storage layer that can still be queried with a single connection string that combines cluster and archive data, powered by Atlas Data Lake . What's Next? Along with announcing Online Archive as Generally Available, we’re excited to share a few additional product enhancements which should be available in the coming months: Private Link support when querying your archive Incremental deletes of data from your archive Support for BYO key encryption on your archival data Improved performance and stability Try Atlas Online Archive Online Archive allows you to right-size your Atlas clusters by storing hot data that is regularly accessed in live storage and moving colder data to a cheaper storage tier. Billing for this feature will include the cost to store data in our fully managed cloud object storage and usage based pricing for querying archive data. We can’t wait to see what new workloads you’ll bring onto MongoDB Atlas with the new flexibility provided by Online Archive! To get started, sign up for an Atlas account and deploy any dedicated cluster (M10 or higher). Have questions? Check out the documentation or head over to our community forums to get answers from fellow developers. And if we’re missing a feature you’d like to see, please let us know ! Safe Harbor Statement The development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB's sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.