Launching Rockets Doesn’t Need to be Rocket Science with MongoDB Atlas: Building Smarter Applications Using Application-Driven Analytics
This article is a preview of our upcoming livestream at 10 am ET on March 15, 2023, which uses a space rocket launch to demo how teams can use MongoDB Atlas to handle real-time analytics for high volumes of data. Data shouldn’t be the limiting factor in any system. Take for example, space rocket launches. Did you know a typical rocket launch can generate more than one million metrics per second? This includes readings from thousands of IoT sensors, observation notes added in real-time by engineers, and critical weather data. Integrating such large volumes of distinct data types into a single platform used to take a significant development effort — and that's even before factoring in real-time analytics and alerts. Status quo architecture: complicated and fragile Teams used to have to do a lot of custom engineering work in order to make this happen. How much? Here’s a basic rundown of just some of that work: Stitching together multiple databases to handle different data structures (i.e., documents, tables, time series measurements, key-values, search), each accessed with its own unique query API. Building ETL data pipelines to transform data in required analytics formats, and tier it from the live database to lower-cost object storage. Spinning up a federated query engine to work across each data tier, again using its own unique query API. Integrating serverless functions to react to real-time data changes. Standing up their own API layers to expose data to consuming applications. All of this complexity places enormous overhead on teams. It results in a multitude of operational and security models to deal with, a ton of data integration work, and lots of data duplication. But for certain critical use cases, like monitoring a space rocket launch, this complicated custom engineering has traditionally been the only solution. Why? A typical rocket launch happens over the course of 30 minutes. There's no time for batch analysis processes that send data from one system to another and separate application events from analytical actions. Added to that, data warehouses and centralized analytics stores are built for historical analyses, not analytics on live, fast changing operational data. Now there is an alternative, though. Analytics processing can now be “shifted left” to the source of the data, to the applications themselves. We call this shift application-driven analytics . Successful modern applications are defined by their ability to drive better customer experiences, surface insights, and take intelligent action directly within the application on live operational data — in real-time. Atlas architecture: enabling app-driven analytics MongoDB Atlas enables application-driven analytics by providing an integrated set of data and application services that put powerful analytics capabilities into the hands of developers in ways that fit their workflows. They can land data of any structure, index, query, and analyze it in any way they want, and then archive it. All while working with a unified API and without having to build their own data pipelines or duplicate data. At the same time, analytics teams get access to live data with their preferred tools without interrupting the application, and with the ability to share insights with the business teams that need it. Some of the advanced capabilities Atlas provides include: Time series collections allow for storing and analyzing time-stamped data. Query API powers analytics with in-database transformations, making it easy to analyze data without complex ETL. Atlas Data Federation enables easily querying data across multiple MongoDB clusters and cloud object storage, providing the ability to combine data from different sources into one unified view. Atlas Charts provides native data visualization capabilities, enabling quick creation of interactive charts and dashboards. Analytics nodes allow for workload isolation ensuring your application performance isn’t impacted by complex analytics. And high availability, automatic failover, end-to-end encryption, and VPC peering in the database , ensures your data remains available and secure. This integrated set of data and application services, along with a unified developer experience, makes MongoDB Atlas the developer data platform for teams looking to build smarter, modern applications for a wide variety of use cases. Like, for instance, monitoring space rocket launches. To learn more about how you can build analytics into your application, join us on LinkedIn Live or YouTube at 10 a.m. ET on March 15, 2023, for Part One of our three-part demo . Jay Runkel, Distinguished Solutions Architect at MongoDB, will simulate a rocket launch with actual launch data from several devices producing one million metrics per second.
What the C-Suite Should Know About Data Strategy for 2023
Trying to predict the future is obviously fraught with difficulty. Anything can happen. Just look at the past few years, where it seemed like everything and anything did happen. With us now in the second month of 2023 and the rest of the year shaping up to be one of potentially big changes and disruptions, the only clear indicator of what’s to come is what we’ve seen trending in the months, weeks, and days preceding this new year. So, with that said, here are five things the C-suite is likely to see more of as 2023 progresses. And what it all means for building a resilient, enduring, and innovative data strategy. 1. Software may still be eating the world, but developers are eating all the work Almost 12 years ago, Marc Andreessen proclaimed, “software is eating the world.” And while that sentiment still holds true today, the biggest beneficiaries of software’s global appetites will continue to be developers. In an interview with The Cube at last year’s AWS re:Invent, MongoDB CEO Dev Ittycheria put it this way: “It’s almost a cliche to say now that software is eating the world. Because every company’s value proposition is driven by software. But what that really means is developers are eating all the work.” One of the best examples of developers “eating all the work” is DevOps. At the advent of DevOps, we saw software development teams incorporate the previously separate domain of IT operations into their work, while turning infrastructure into a programmable interface and creating a continuous feedback loop that improved developer agility. But DevOps was just the start. We’re now seeing developers embedding other previously separate domains into their work, such as security, data science, and data analytics (more on that below). The business implications of embedding these previously disparate domains into software development are quite huge. It means rapid innovation, faster time-to-market, better fraud detection and prevention, A/B testing — the list goes on and on. With software continuing to eat the world, developers are continuing to eat all the work while also taking massive bites out of silos. 2. Builder teams will require less and less complexity With software development teams taking on more work, we’re also going to see the need to reduce complexity. Particularly when it comes to bolt-on solutions. Search is a good example here. For a lot of teams out there, database operations and search have traditionally been two separate systems that are then glued together. Which doesn’t usually decrease complexity. In fact, the opposite happens. Such as having to manage dependencies across systems. But when teams have access to a single, unified, and fully-managed platform that integrates the database, search engine, and sync mechanism, you remove the need for glue and the complexity goes way down. As SVP of products at MongoDB Andrew Davidson said on a recent episode of The Cloudcast : “...Search as a bolt-on [and] entirely different system… has such a profoundly inconsistent experience that if you can bring it in to have near consistency in line with the database, that's a game changer…” And with development teams taking on more and more work previously associated with separate domains, like analytics (described above), they’re needing to use other systems that have also been traditionally glued together. So the question facing many organizations this year and beyond will be: Why spend time moving data between separate glued-together solutions for things like search, visualization, and analytics, when a single data platform can handle it all? 3. Apps are going to get a lot smarter If you were to go back 15 years to 2008 — which, wow, can’t believe that was 15 years ago, but anyway… — you’d notice just how radically the technology landscape has really changed. Cloud computing wasn’t quite yet a thing back then. And mobile was really just getting off the ground. Today, an equally sizable shift is happening. In an interview with SiliconAngle this past November, MongoDB CEO Dev Ittycheria said: “I believe the next big platform shift is moving from dumb apps to smart apps that incorporate machine learning, AI, and very sophisticated automation.” As mentioned previously, development teams are taking on more work associated with previously separate domains. This is also happening with data analytics, which previously lived outside the application development process. But now analytics is “ shifting left ” directly into app development. The results for businesses are: the ability for applications to process and analyze real-time data much, much faster and at a lower cost, and to both understand trends and make more informed predictions based on those trends. The results for customers are greater personalization and richer digital experiences. Building smarter applications is the future. But how quickly and effectively organizations do that is still dependent on their data platforms . Not all can bring analytics into app development in the same ways. In this respect, the future may be smarter applications; but for different businesses — to paraphrase author William Gibson — that future isn’t evenly distributed. Yet. Encryption, encryption, [$a&*9Qd] Encryption will not only continue to be critical for how organizations store their data, it will also revolutionize how data is used in the application development process. Ask a lot of software veterans about data encryption and they’re likely to tell you how important it is. They’ll also likely say that encryption, particularly in-use encryption, can have scalability issues and/or complexity problems . But in 2023 and beyond, new advancements will make those issues a thing of the past. With new technologies, like Queryable Encryption , the ability to build smarter applications that use end-to-end encrypted data can move at the speed that development teams and businesses require. The added benefit is that this increases end-user trust. As MongoDB’s chief information security officer Lena Smart said in an interview with SiliconAngle in December 2022 : “By giving people things like Queryable Encryption, for example, you’re going to free up a whole bunch of headspace. [Their customers] don’t have to worry about their data being … harvested from memory or harvested while at rest or in motion.” The name of the game in 2023 will be 8QTwZm* *encrypted for demonstration purposes. 5. Bottom line: Your data strategy is your business strategy When we get to the end of December 2023, we’ll probably look back on the intervening months between now and then and see a lot of stuff we didn’t expect. What we do know is that data is going to play an increasingly important role in how businesses operate. Why do we know this? Well, because this has been a trend in each and every year since organizations first started using data to build better software and richer digital experiences. Software might be eating the world, and developers might be eating the work, but data is eating business. So in 2023, it’s incumbent for business leaders to set the table accordingly. To get started building your data strategy with MongoDB, get in touch with our experts .