Karthic Subramanian

Launching Rockets Doesn’t Need to be Rocket Science with MongoDB Atlas: Building Smarter Applications Using Application-Driven Analytics

This article is a preview of our upcoming livestream at 10 am ET on March 15, 2023, which uses a space rocket launch to demo how teams can use MongoDB Atlas to handle real-time analytics for high volumes of data. Data shouldn’t be the limiting factor in any system. Take for example, space rocket launches. Did you know a typical rocket launch can generate more than one million metrics per second? This includes readings from thousands of IoT sensors, observation notes added in real-time by engineers, and critical weather data. Integrating such large volumes of distinct data types into a single platform used to take a significant development effort — and that's even before factoring in real-time analytics and alerts. Status quo architecture: complicated and fragile Teams used to have to do a lot of custom engineering work in order to make this happen. How much? Here’s a basic rundown of just some of that work: Stitching together multiple databases to handle different data structures (i.e., documents, tables, time series measurements, key-values, search), each accessed with its own unique query API. Building ETL data pipelines to transform data in required analytics formats, and tier it from the live database to lower-cost object storage. Spinning up a federated query engine to work across each data tier, again using its own unique query API. Integrating serverless functions to react to real-time data changes. Standing up their own API layers to expose data to consuming applications. All of this complexity places enormous overhead on teams. It results in a multitude of operational and security models to deal with, a ton of data integration work, and lots of data duplication. But for certain critical use cases, like monitoring a space rocket launch, this complicated custom engineering has traditionally been the only solution. Why? A typical rocket launch happens over the course of 30 minutes. There's no time for batch analysis processes that send data from one system to another and separate application events from analytical actions. Added to that, data warehouses and centralized analytics stores are built for historical analyses, not analytics on live, fast changing operational data. Now there is an alternative, though. Analytics processing can now be “shifted left” to the source of the data, to the applications themselves. We call this shift application-driven analytics . Successful modern applications are defined by their ability to drive better customer experiences, surface insights, and take intelligent action directly within the application on live operational data — in real-time. Atlas architecture: enabling app-driven analytics MongoDB Atlas enables application-driven analytics by providing an integrated set of data and application services that put powerful analytics capabilities into the hands of developers in ways that fit their workflows. They can land data of any structure, index, query, and analyze it in any way they want, and then archive it. All while working with a unified API and without having to build their own data pipelines or duplicate data. At the same time, analytics teams get access to live data with their preferred tools without interrupting the application, and with the ability to share insights with the business teams that need it. Some of the advanced capabilities Atlas provides include: Time series collections allow for storing and analyzing time-stamped data. Query API powers analytics with in-database transformations, making it easy to analyze data without complex ETL. Atlas Data Federation enables easily querying data across multiple MongoDB clusters and cloud object storage, providing the ability to combine data from different sources into one unified view. Atlas Charts provides native data visualization capabilities, enabling quick creation of interactive charts and dashboards. Analytics nodes allow for workload isolation ensuring your application performance isn’t impacted by complex analytics. And high availability, automatic failover, end-to-end encryption, and VPC peering in the database , ensures your data remains available and secure. This integrated set of data and application services, along with a unified developer experience, makes MongoDB Atlas the developer data platform for teams looking to build smarter, modern applications for a wide variety of use cases. Like, for instance, monitoring space rocket launches. To learn more about how you can build analytics into your application, join us on LinkedIn Live or YouTube at 10 a.m. ET on March 15, 2023, for Part One of our three-part demo . Jay Runkel, Distinguished Solutions Architect at MongoDB, will simulate a rocket launch with actual launch data from several devices producing one million metrics per second.