Enhancing Atlas Online Archive With Data Expiration and Scheduled Archiving
June 7, 2022
Atlas's Online Archive feature allows you to set archiving rules to move data that is not frequently accessed from your Atlas cluster to a MongoDB-managed object store. It also allows you to query both your Atlas and Online Archive data in a unified manner without having to worry about the tier in which the data resides.
We are enhancing this feature by adding two new capabilities: data expiration and scheduled archiving (in preview).
Data expiration: Online Archive makes it easy to tier data out of a live database into an object store, but what if you want to set a second threshold to delete data from the archive entirely? Perhaps you don’t want to store data indefinitely due to costs, or maybe you have a compliance requirement for data expiration that means you need to ensure deletion on a schedule.
Previously, there was no option to remove data from the archive except deleting the archive completely, which is not a preferred option for most use cases. With our new data expiration feature, you can specify for how many days data should be stored in the online archive before being deleted. You can set an expiration from the archive as low as seven days and as high as 9,125 days; you can set the archive expiration time through either the Atlas UI or the Admin API. Expiration rules can be edited after creation, if needed.
Note that once the data is deleted from the archive there is no way to recover it, so you must define your rules carefully.
Scheduled archiving: Previously, the archiving process ran every five minutes. Though in most cases this is acceptable, some customers were concerned that this process could affect cluster performance when, for example, a cluster is running close to capacity during a specific time period. If an archiving window overlaps with this period, it may overload the cluster and lead to stability issues. These customers requested the ability to schedule archiving during an off-peak time, when the clusters have spare capacity. With this requirement in mind, we are thrilled to introduce the scheduled archiving feature.
You can configure the scheduled window by setting rules. The window can be scheduled to repeat every day, every week, or every month, depending on your preference. To ensure that the archive process is able to work through any backlog that’s accrued, there is a minimum window requirement of two hours.
You are also able to edit the archive rule and define when you want to archive your data and when you want to delete it.
Data expiration and scheduled archiving will provide operational efficiency to Atlas customers. Both are in preview mode and will be generally available soon. See the documentation for additional information about data expiration and scheduled archiving.
Working Together: MongoDB's Partner of the Year Awards
Organizations face more pressure than ever to modernize, to innovate, and to become more data-driven. Whether it's AstraZeneca doing next-generation genome sequencing to develop drugs to fight cancer and other diseases, Toyota connecting more than 100,000 warehouse vehicles with integrated telematics to reshape industrialization, or Forbes developing new ways to create and deploy content, many of the world's top companies have chosen to modernize with MongoDB. Today, more than 35,000 customers trust the MongoDB Atlas data platform with their most critical workloads. But we don’t do it alone. MongoDB's partner ecosystem has scaled to every vertical, industry, and geography, delivering MongoDB to tens of thousands of customers. MongoDB partners play many roles in the ecosystem, from extending MongoDB's developer data platform with complementary functionality to building new solutions on MongoDB to offering expertise to implement and modernize databases. Now, we are pleased to announce the 2022 MongoDB Partners of the Year. Although we are grateful for all of the many partners that help MongoDB succeed with customers, this year at MongoDB World we are recognizing a select few that have worked especially closely with us to drive joint success in the past year. The 2022 MongoDB Partners of the Year are Accenture , Alibaba Cloud , AWS , BigID , Capgemini , Carahsoft , commercetools , Confluent , Exafluence , Google Cloud , HCL Technologies , IBM , PeerIslands , Tecnotree , Temenos , and Unqork . We will honor these companies at our partner VIP reception on June 6, 2022. Global System Integrator Partner of the Year Accenture is a leading global professional services company, providing a broad range of services and solutions across strategy, consulting, applied intelligence, technology, and operations. A core tenet of Accenture’s “Cloud Continuum” approach indicates that the companies who achieve the most value from their cloud transformation journeys are those who go beyond “lift and shift” migrations and drive to “move and improve” their applications onto the cloud. Thus, Accenture is investing heavily in upskilling their consultants on MongoDB, as well as building assets to accelerate their clients’ MongoDB transformations. One example is Smart Data Mover, an Accenture asset designed to simplify and accelerate the process of modernizing legacy applications from relational database systems to MongoDB Atlas on the cloud. As a result, Accenture has been an essential partner in helping enterprises modernize with MongoDB. Learn more about the partnership. Boutique System Integrator Partner of the Year PeerIslands is a service consulting firm focused on providing high-velocity digital transformation. PeerIslands won the Boutique Systems Integrator Partner of the Year in 2021 as well, in recognition of their expertise and outstanding work in the MongoDB ecosystem. The firm consults on cloud transformation strategy and partners on product and data engineering, application transformation, and data science and AI/ML. In addition, PeerIslands has developed a suite of tools that allow for easy adoption of MongoDB and seamless migration of workloads to MongoDB within enterprises. Learn more about the partnership. Rising Star System Integrator Partner of the Year Capgemini is one of the world's foremost providers of consulting, technology and outsourcing services. Capgemini and MongoDB have partnered to help re-envision data and advanced business processes so its clients can focus on innovation. Traditional relational database management systems often inhibit performance, falter under heavy volumes, and slow deployment. Capgemini’s new Database Convert & Compare (DCC) tool, built on MongoDB, helps customers develop a new database or migrate from legacy systems to MongoDB. CapGemini has helped grow MongoDB’s footprint in some of the largest Fortune 100 financial services customers contributing large revenue numbers through the partnership, and it has been instrumental in driving MongoDB adoption by identifying new workloads and implementing new use cases for digital transformation, legacy modernization, and app modernization for financial services customers. Learn more about the partnership. System Integrator Certification Partner of the Year HCL Technologies (HCL) is a leading global technology company, helping enterprises reimagine their businesses for the digital age. Their technology products and services are built on four decades of innovation, with a world-renowned management philosophy, a strong culture of invention and risk-taking, and a relentless focus on customer relationships. HCL is partnering with MongoDB across Cloud Migration, Digital Business, and Engineering Services. This award honors partners with the highest number of MongoDB certified champions across the globe, who are skilled to modernize and transform customers' application and infrastructure leveraging MongoDB technologies. Learn more about the partnership. Cloud (Co-Sell) Partner of the Year AWS provides a massive global cloud infrastructure that allows customers to rapidly innovate and iterate. In 2022, AWS and MongoDB entered into a six-year collaboration agreement , which has placed the partnership into hypergrowth. MongoDB and AWS sellers have come together to help customers modernize, accelerate consumption of AWS customer commits, win new workloads, and close on major cross-industry and cross-vertical deals around the world. MongoDB and AWS have also partnered to help customers build modern, event-driven serverless applications, while releasing integrations for Atlas with Amazon EventBridge, Amazon Kinesis, AWS App Runner, AWS PrivateLink, AWS Wavelength, and more in the past year. The launch of Pay-as-You-Go option on AWS Marketplace made it easier for customers to pay for MongoDB Atlas using their AWS account. With additional integrations in the pipeline, there’s so much more to come. Learn more about the partnership. Cloud (Marketplace) Partner of the Year Google Cloud views the data platform as an essential building block of cloud infrastructure. Since launching MongoDB Atlas on Google Cloud Marketplace over the past three years, this partnership has seen rapid adoption and acceleration across industries such as gaming, retail, healthcare, financial services, and automotive. We have integrated MongoDB Atlas with more Google services, including DataStream, BigQuery, DataFlow, Cloud Run, App Engine, Apigee, and Cloud Functions, while also introducing a Pay-as-You-Go option on the Google Cloud Console and Marketplace . More and more Google Cloud customers are choosing to run MongoDB Atlas for a variety of needs, including managing large-scale product catalogs of popular e-commerce websites, building great customer experiences by unifying disparate pieces of data, and building modern global web and mobile applications. Learn more about the partnership. Cloud (Emerging Markets) Partner of the Year Alibaba Cloud is one of MongoDB’s largest Cloud OEM partners. In our third year of partnership with Alibaba Cloud as an authorized MongoDB-as-a-service solution, MongoDB has seen some of its strongest adoption numbers in mainland China, especially in the gaming and automotive industries. With this partnership, Alibaba Cloud ensures end-to-end management and support for customers on current and future versions of MongoDB, with the ability to escalate bug fixes and support issues on their behalf. Users of Alibaba Cloud’s platform offering receive easy access to the latest MongoDB features and capabilities, backed by comprehensive support from Alibaba Cloud and MongoDB. We remain thrilled to further scale this partnership. Try MongoDB on Alibaba Cloud . Independent Software Vendor (ISV) Partner of the Year (Financial Services) Temenos offers cloud-native, cloud-agnostic, API-first digital banking, core banking, payments, fund management, and wealth management software products, enabling banks to deliver consistent, frictionless customer journeys and achieve market-leading cost/income performance. More than 3,800 banks across the globe rely on Temenos to process the client interactions and daily transactions of more than 500 million customers. MongoDB underpins Temenos Transact, which is the most successful and widely used digital core-banking solution in the world. Temenos also selected MongoDB as foundational technology for their microservices transformation journey as they deliver their flagship product, Infinity Digital Banking. Together, we help banks and financial institutions comply with stringent data sovereignty requirements and scale seamlessly. Learn more about the partnership. Independent Software Vendor (ISV) Partner of the Year (Telco) Tecnotree is a digital business support systems (BSS) and digital partner ecosystem provider with more than 40 years of deep domain knowledge, proven delivery, and transformation capability. The company offers digital solutions for Communication & Digital Service Providers across the globe. Leveraging MongoDB technology, Tecnotree delivers world-class BSS and partner ecosystem solutions for telcos, empowering them to deliver insightful and connected customer experiences to their enterprises, consumers and partners. MongoDB powers Tecnotree’s Digital Customer Lifecycle Management Suite, empowering communications service providers (CSPs) to transform their business towards a marketplace of digital services. Learn more about the partnership. Independent Software Vendor (ISV) Partner of the Year (Healthcare) Exafluence is a domain-centric data and analytics company that utilizes modern digital technologies to enhance the competitive advantage of its customers in their markets. Exafluence brings in low-code platforms, accelerators, and expertise across a range of digital technologies to reduce time by 45% to 60% and cost by 25% to 30%, while delivering significantly higher business value. Exafluence has built a variety of technical and business solutions that can be assembled together and customized for specific business scenarios. Some of its key platform offerings include ExfDigital (Data), ExfInsights (Analytics), ExfHealth (Healthcare), and ExfIndustry (IoT). Learn more about the partnership. Independent Software Vendor (ISV) Partner of the Year (Retail) Commercetools ’ world-leading commerce platform, built on modern MACH principles (microservice-based, API-first, cloud-native and headless), is driving the future of digital commerce. Fully powered by MongoDB Atlas, the Commercetools suite allows brands to work with, not around, their commerce solution to tailor experiences to the exact needs of the business and its customers. Through this partnership, Commercetools has made MongoDB the optimal system of record for an even broader range of business-critical applications. Learn more about the partnership. Independent Software Vendor (ISV) Partner of the Year (Security) BigID’s data intelligence and privacy platform allows customers to rely on actual data discovered across the enterprise and cloud, rather than surveys and interviews, to ensure privacy compliance. To generate data insights at scale, automate advanced discovery and classification, and accommodate complex enterprise requirements, BigID chose MongoDB to power their platform. MongoDB’s data platform provides a flexible, reliable and scalable solution to customers across any vertical. Learn more about the partnership. Technology Partner of the Year Founded by the original developers of Apache Kafka, Confluent delivers the most complete distribution of Kafka with Confluent Platform. Real-time event streaming (data-in-motion) from Confluent complements the modern general-purpose distributed document database platform (for data-at-rest) from MongoDB, enabling organizations to run their business in real-time and to build fast-moving applications enriched with historical context. Learn more about the partnership. Distributor Partner of the Year Carahsoft is a trusted, long-standing MongoDB partner, and it continues to be one of our largest resellers in the world. Carahsoft’s depth and reach in the public sector market helps government agencies leverage open technologies to drive innovation, maximize cost efficiencies, and achieve success for their digital modernization initiatives. The company has been critical to growing MongoDB’s government agency and civilian business, and we’re excited for what’s next. Learn more about the partnership. Hybrid Cloud Partner of the Year IBM is a global technology company that provides infrastructure, software, hybrid cloud services, and cognitive computing to global enterprise clients. IBM played an instrumental role in helping one of the world’s largest banks launch their internal MongoDB-as-a-service on the IBM LinuxONE platform for their most critical financial applications. Our strategic partnership has helped customers powering applications on IBM Power platform and those requiring a fully managed database-as-a-service on IBM Cloud. Our OEM Partnership with Data & AI solutions and integration with IBM Cloud Pak for Data (ICP) has helped create success stories at large enterprises across the globe. By leveraging the flexibility of MongoDB on IBM Cloud Pak for Data, customers can get to market faster, iterate faster, and attract new customers faster, without sacrificing security or control. Learn more about the partnership. MongoDB for Startups Program to Unicorn Partner of the Year Unqork ’s enterprise no-code platform helps leading organizations build, deploy, and manage complex software without having to think about code. Unqork created the codeless architecture standard to free the world’s largest enterprises from the pitfalls of legacy code and allow them to focus on innovation to drive business and maintain a competitive edge. In 2017, also the year of its founding, Unqork joined the MongoDB for Startups program, which gave them free access to get started on MongoDB Atlas. Since then, Unqork has raised $414 million in funding and is valued at $2 billion. MongoDB is thrilled for Unqork’s success and honored to be a part of their journey. Learn more about the partnership. Learn more about being a MongoDB partner. Visit our list of partners to see all of our partners.
Security in Government Solutions: Why Secure By Default is Essential
Data security in government agencies is table stakes at this point. Everyone knows it’s essential, both for compliance and data protection purposes. However, most government agencies are working with solutions that require frequent security patches or built-on tools to protect their data. Today, the federal government is pushing its agencies to move to modernize their solutions and improve their security posture. For example, the DHS and Cybersecurity and Infrastructure Security Agency’s recently issued technical rule for modernization of the Protected Critical Information Infrastructure program – a program that provides legal protections for cyber and physical infrastructure information submitted to DHS. “The PCII Program is essential to CISA’s ability to gather information about risks facing critical infrastructure,” said Dr. David Mussington, Executive Assistant Director for Infrastructure Security. “This technical rule modernizes and clarifies important aspects of the Program, making it easier for our partners to share information with DHS. These revisions further demonstrate our commitment to ensuring that sensitive, proprietary information shared with CISA remains secure and protected.” So how can government agencies modernize their data infrastructure and find solutions that not only protect data but also power innovation? Let’s look into a few different strategies. 1. Why secure by default is key Secure by default means that any piece of software uses default security settings that are configured for the highest possible security out of the box. CISA Director Jen Easterly has addressed how using solutions that are secure by default is critical for public sector organizations. “We have to have [multi-factor authentication] by default. We can't charge extra for security logging and [single sign-on],” Easterly said . “We need to ensure that we're coming together to really protect the technology ecosystem instead of putting the burden on those least able to defend themselves.” “The American people have accepted the fact that they’re constantly going to have to update their software,” she said. “The burden is placed on you as the user and that’s what we have to collectively stop.” Easterly is right. Secure-by-design solutions are vital to the success of data protection. The expectation should alway be that solutions have built-in, not bolt-on security features. One approach that’s gaining traction both in the public and private sectors is zero trust environments. In a zero trust environment, the perimeter is assumed to have been breached. There are no trusted users, and no user or device gains trust simply because of its physical or network location. Every user, device, and connection must be continually verified and audited. As the creator of zero trust, security expert John Kindervag, summed it up: “Never trust, always verify.” For government agencies, that means the underlying database must be secure by default, and it needs to limit users’ opportunities to make it less secure. 2. Security isn't just on-prem anymore; cloud is secure, too Cloud can be a scary word for public sector organizations. Trusting your sensitive data to the cloud might feel risky for those who handle some of the country’s most sensitive data. But, cloud providers are stepping up to meet the security needs of government agencies. There is no need to fear the cloud anymore. Government agencies and other public sector organizations nationwide are navigating cloud modernization through the lens of increased cybersecurity requirements outlined in the 2021 Executive Order on Improving the Nation’s Cybersecurity . “The Federal Government must adopt security best practices; advance toward Zero Trust Architecture; accelerate movement to secure cloud services, including Software as a Service (SaaS), Infrastructure as a Service (IaaS), and Platform as a Service (PaaS); centralize and streamline access to cybersecurity data to drive analytics for identifying and managing cybersecurity risks; and invest in both technology and personnel to match these modernization goals.” Also, the major cloud providers are well established, purpose-built options for government users. AWS GovCloud, for example, is more than a decade old and was “ the first cloud provider to build cloud infrastructure specifically designed to meet U.S. government security and compliance needs.” This push by the federal government toward cloud modernization and increased cybersecurity will be a catalyst in upcoming years for rapid cloud adoption and greater dependence on cloud solutions designed specifically for government users. 3. Security features purpose-built for goverment needs are essential Government agencies are held to a higher standard than those in the private sector. From data used in sometimes life-or-death missions to data for students building their futures in educational institutions (and everything in between), security has real-world consequences. Today, security is non-negotiable and like we explored above, it’s especially crucial that public sector entities have built-in security measures to keep data protected. So, what built-in features should you look for? Network isolation and access It’s critical that your data and underlying systems are fully isolated from other organizations using the same cloud provider. Database resources should be associated with a user group, which is contained in its own Virtual Private Cloud (VPC), and access should be granted by IP access lists, VPC peering, or private endpoints. Encyption in flight, at rest, and in use Encryption should be the standard. For example, when using MongoDB Atlas, all network traffic is encrypted using Transport Layer Security (TLS). Encryption for data at rest is automated using encrypted storage volumes. Customers can use field-level encryption to encrypt sensitive workloads which enables you to encrypt data in your application before you send it over the network to MongoDB clusters. Users can bring their own encryption keys for an additional level of control. Granular database auditing Granular database auditing allows administrators to answer detailed questions about systems activity by tracking all commands against the database. This ensures you always know who has access to what data and how they’re using it. Multi-factor authentication User credentials should always be stored using industry-standard and audited one-way hashing mechanisms, with multi-factor authentication options including SMS, voice call, a multi-factor app, or a multi-factor device, ensuring only approved users have access to your data. MongoDB Atlas for Government: Purpose-built for public sector As we’ve discussed, solutions that are purpose-built with built-in security are ideal for government agencies, and choosing the right one is the best way to keep sensitive data protected. MongoDB Atlas for Government on AWS GovCloud recently secured its FedRAMP Moderate authorization thanks to these security measures built into the solution. FedRAMP is a government-wide program that provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services. To ensure the utmost levels of security, Atlas for Government is an independent, dedicated environment for the U.S. public sector, as well as ISVs looking to build U.S. public sector offerings. Public Sector organizations carry a heavy burden when it comes to keeping data protected. However, with the right data platform underpinning modern applications – a platform with built-in security features – progress doesn’t mean you have to compromise on security. Want to learn more about data protection best practices for public sector organizations? Attend our upcoming webinar on April 12 for deeper insight .