Build Event-Driven Apps Locally with MongoDB Atlas Stream Processing

Gautam Mulchandani
July 1, 2025

Building event-driven architectures (EDAs) often poses challenges, particularly when you’re integrating complex cloud components with local development services. For developers, working directly from a local environment provides convenience, speed, and flexibility. Our demo application demonstrates a unique development workflow that balances local service integration with cloud stream processing, showcasing portable, real-time event handling using MongoDB Atlas Stream Processing and ngrok.

With MongoDB Atlas Stream Processing, you can streamline the development of event-driven systems while maintaining all the components locally. Using this service’s capabilities alongside ngrok, this demo application shows a secure way to interact with cloud services directly from your laptop, ensuring you can build, test, and refine applications with minimal friction and maximum efficiency.

Using MongoDB Atlas Stream Processing

MongoDB Atlas Stream Processing is a powerful feature within the MongoDB Atlas modern database that enables you to process data streams in real time using the familiar MongoDB Query API (and aggregation pipeline syntax). It integrates seamlessly with MongoDB Atlas clusters, Apache Kafka, AWS Lambda, and external HTTP endpoints.

Key takeaway #1: Build event-driven apps more easily with MongoDB Atlas Stream Processing

One of the primary goals of MongoDB Atlas Stream Processing is to simplify the development of event-driven applications. Instead of managing separate stream processing clusters or complex middleware, you can define your processing logic directly within MongoDB Atlas. This means:

A unified platform: Keep your data storage and stream processing within the same ecosystem.
Familiar syntax: Use the MongoDB Query API and aggregation pipelines you already know.
Managed infrastructure: Let MongoDB Atlas handle the underlying infrastructure, scaling, and availability for your stream processors.

Key takeaway #2: Develop and test locally, deploy globally

A significant challenge in developing event-driven systems is bridging the gap between your local development environment and cloud-based services. How do you test interactions with services running on your laptop? You can configure MongoDB Atlas Stream Processing to connect securely to HTTP services and even Apache Kafka instances running directly on your development machine!

You can typically achieve this using a tunneling service like ngrok, which creates secure, publicly accessible URLs for your local services. MongoDB Atlas Stream Processing requires HTTPS for HTTP endpoints and specific Simple Authentication and Security Layer protocols for Apache Kafka, making ngrok an essential tool for this local development workflow.

Introducing the real-time order fulfillment demo

To showcase these capabilities in action, we’ve built a full-fledged demo application available on GitHub.

Figure 1. High-level architecture diagram.

Diagram showing the high-level architecture. The diagram is split into two boxes, one on the left and one on the right. On the left, the box is labeled local developer, and includes sub-boxes for shopping cart event generator, kafka broker, and order processing service. This left box, connects over to the right box, which is labeled MongoDB Atlas, and contains sub-boxes for Atlas Cluster and Atlas Stream Processor.

This demo simulates a real-time order fulfillment process using an event-driven architecture orchestrated entirely by MongoDB Atlas Stream Processing.

What the demo features

A shopping cart service: Generates events when cart items change.
An order processing service: Handles order creation and validation (running locally as an HTTP service).
A shipment service: Manages shipment updates.
Event source flexibility: Can ingest events from either a MongoDB capped collection or an Apache Kafka topic (which can also run locally).
Processors from Atlas Stream Processing: Act as the central nervous system, reacting to events and triggering actions in the different services.
An order history database: Centralizes status updates for easy tracking.

Figure 2. High-level sequence diagram of a flow.

How the demo uses MongoDB Atlas Stream Processing and local development

Event orchestration: MongoDB Atlas Stream Processing instances listen for shopping cart events (from MongoDB or Kafka).
Local service interaction: An ASP processor calls the Order Processing Service running locally on localhost via an ngrok HTTPS tunnel.
Kafka integration (optional): Demonstrates ASP connecting to a local Kafka broker, also tunneled via ngrok.
Data enrichment & routing: Processors enrich events and route them appropriately (e.g., validating order, triggering shipments).
Centralized logging: All services write status updates to a central MongoDB collection that functions as a continuously materialized view of order status and history.

This demo practically illustrates how you can build sophisticated, event-driven applications using ASP while performing key development and testing directly on your laptop, interacting with local services just as you would in a deployed environment.

What the demo highlights

Real-world EDA: Provides a practical example of asynchronous service communication.
Orchestration powered by MongoDB Atlas Stream Processing: Shows how this service manages complex event flows.
- Local development workflow: Proves the concept of connecting this service to local HTTP / Apache Kafka via ngrok.
- Flexible event ingestion: Supports both MongoDB and Apache Kafka sources.
- Centralized auditing: Demonstrates easy status tracking via a dedicated history collection.

Get started with the demo!

MongoDB Atlas Stream Processing significantly lowers the barrier to entry for building robust, real-time EDAs. Its ability to integrate seamlessly with MongoDB Atlas, external services, and, crucially, your local development environment (thanks to tools like ngrok) makes it a powerful addition to the developer toolkit.

Explore the demo project, dive into the code, and see for yourself how ASP can simplify your next event-driven architecture, starting right from your own laptop!

Ready to see it in action? Head over to the GitHub repository!

The repository’s README.md file contains comprehensive, step-by-step instructions to get you up and running. In summary, you’ll:

Clone the repository.
Set up a Python virtual environment and install dependencies.
Crucially, set up ngrok to expose your local order-processing service (and Apache Kafka, if applicable) via secure tunnels. (Details in the README.md appendix!)
Configure your .env file with MongoDB Atlas credentials, API keys, and the ngrok URLs.
Run scripts to create necessary databases, collections, and the MongoDB Atlas Stream Processing instance/connections/processors.
Start the local order_processing_service.py.
Run the shopping_cart_event_generator.py to simulate events.
Query the order history to see the results!

For detailed setup guidance, especially regarding ngrok configuration for multiple local services (HTTP and TCP / Apache Kafka), please refer to the appendix of the project's README.md.

← Previous

Data Modeling Strategies for Connected Vehicle Signal Data in MongoDB

Today’s connected vehicles generate massive amounts of data. According to an article from S&P Global Mobility, a single modern car produces nearly 25GB of data per hour. To put that in perspective: that’s like each car pumping out the equivalent of six full-length Avatar movies in 4K—every single day! Now scale that across millions of vehicles, and it’s easy to see the challenge ahead. Of course, not all of that data needs to be synchronized to the cloud—but even a fraction of it puts significant pressure on the systems tasked with processing, storing, and analyzing it at scale. The challenge isn’t just about volume. The data is fast-moving and highly diverse—from telematics and location tracking to infotainment usage and driver behavior. Without a consistent structure, this data is hard to use across systems and organizations. That’s why organizations across the industry are working to standardize how vehicle data is defined and exchanged. One such example is the Connected Vehicle Systems Alliance or COVESA , which developed the Vehicle Signal Specification (VSS)—a widely adopted, open data model that helps normalize vehicle signals and improve interoperability. But once data is modeled, how do you ensure it's persistent and available at all times in real-time? To meet these demands, you need a data layer that's flexible, reliable, and performant at scale. This is precisely where a robust data solution designed for modern needs becomes essential. In this blog, we’ll explore data strategies for connected vehicle systems using VSS as a reference model, with a focus on real-world applications like fleet management. These strategies are particularly effective when implemented on flexible, high-performance databases like MongoDB, a solution trusted by leading automotive companies . Is your data layer ready for the connected car era? Relational databases were built in an era when saving storage space was the top priority. They work well when data fits neatly into tables and columns—but that’s rarely the case with modern, high-volume, and fast-moving vehicle data. Telematics, GPS coordinates, sensor signals, infotainment activity, diagnostic logs—data that’s complex, semi-structured, and constantly evolving. Trying to force it into a rigid schema quickly becomes a bottleneck. That’s why many in the automotive world are moving to document-oriented databases. A full-fledged data solution, designed for modern needs, can significantly simplify how one works with data, scale effortlessly as demands grow, and adapt quickly as systems evolve. A solution embodying these capabilities, like MongoDB, supports the demands of complex connected vehicle systems. Its features include: Reduced complexity: The document model mirrors the way developers already structure data in their code. This makes it a natural fit for vehicle data, where data often comes in nested, hierarchical formats. Scale by design: MongoDB’s distributed architecture and flexible schema design help simplify scaling. It reduces interdependencies, making it easier to shard workloads without performance headaches. Built for change: Vehicle platforms are constantly evolving, and MongoDB makes it easy to update data models without costly migrations or downtime, keeping development fast and agile. AI-ready: MongoDB supports a wide variety of data types—structured, time series, vector, graph—which are essential for AI-driven applications. This makes it the natural choice for AI workloads, simplifying data integration and accelerating the development of smart systems. Figure 1. The MongoDB connected car data platform. These capabilities are especially relevant in connected vehicle systems. Companies like Volvo Connect use MongoDB Atlas to track 65 million daily events from over a million vehicles, ensuring real-time visibility at massive scale. Another example is SHARE NOW , which handles 2TB of IoT data per day from 11,000 vehicles across 16 cities, using MongoDB to streamline operations and deliver better mobility experiences. It’s not just the data—it’s how you use it Data modeling is where good design turns into great performance. In traditional relational systems, modeling starts with entities and relationships to focus on minimizing data duplication. MongoDB flips that mindset. You still care about entity relationships—but what really drives design is how the data will be used. The core principle? Data that is accessed together should be stored together. Let’s bring this to life. Take a fleet management system. The workload includes vehicle tracking, diagnostics, and usage reporting. Modeling in MongoDB starts by understanding how that data is produced and consumed. Who’s reading it, when, and how often? What’s being written, and at what rate? Below, we show a simplified workload table that maps out entities, operations, and expected rates. Table 1. Fleet management workload example. Now, to the big question: how do you model connected vehicle signal data in MongoDB? It depends on the workload. If you're using COVESA’s VSS as your signal definition model, you already have a helpful structure. VSS defines signals as a hierarchy: attributes (rarely change, like tank size), sensors (update often, like speed), and actuators (reflect commands, like door lock requests). This classification is a great modeling hint. VSS’s tree structure maps neatly to MongoDB documents. You could store the whole tree in a single document, but in most cases, it’s more effective to use multiple documents per vehicle. This approach better reflects how the data is produced and consumed—leading to a model that’s better suited for performance at scale. Now, let’s look at two examples that show different strategies depending on the workload. Figure 2. Sample VSS tree. Source: Vehicle Signal Specification documentation . Example 1: Modeling for historical analysis For historical analysis—like tracking fuel consumption trends—time-stamped data needs to be stored efficiently. Access patterns may include queries like “What was the average fuel consumption per km in the last hour?” or “How did the fuel level change over time?” Here, separating static attributes from dynamic sensor signals helps minimize unnecessary updates. Grouping signals by component (e.g., powertrain, battery) allows updates to be scoped and efficient. MongoDB Time Series collections are built for exactly this kind of data, offering optimized storage, automatic bucketing, and fast time-based queries. Example 2: Modeling for the last vehicle state If your focus is real-time state—like retrieving the latest signal values for a vehicle—you’ll prioritize fast reads and lightweight updates. Common queries include “What’s the latest coolant temperature?” or “Where are all fleet vehicles right now?” In this case, storing a single document per vehicle or update group with only the most recent signal values works well. Updating fields in place avoids document growth and keeps read complexity low. Grouping frequently updated signals together and flattening nested structures ensures that performance stays consistent as data grows. These are just two examples—tailored for different workloads—but MongoDB offers the flexibility to adapt your model as needs evolve. For a deeper dive into MongoDB data modeling best practices, check out our MongoDB University course and explore our Building with Patterns blog series . The right model isn't one-size-fits-all—it’s the one that matches your workload. How to model your vehicle signal data At the COVESA AMM Spring 2025 event, the MongoDB Industry Solutions team presented a prototype to help simplify how connected vehicle systems adopt the Vehicle Signal Specification. The concept: make it easier to move from abstract signal definitions to practical, scalable database designs. The goal wasn’t to deliver a production-ready tool—it was to spark discussion, test ideas, and validate patterns. It resonated with the community, and we’re continuing to iterate on it. For now, the use cases are limited, but they highlight important design decisions: how to structure vehicle signals, how to tailor that structure to the needs of an application, and how to test those assumptions in MongoDB. Figure 3. Vehicle Signal Data Model prototype high-level architecture. This vehicle signals data modeler is a web-based prototype built with Next.js and powered by MongoDB Atlas. It’s made up of three core modules: Schema builder: This is where it starts. You can visually explore the vehicle signals tree, select relevant data points, and define how they should be structured in your schema. Use case mapper: Once the schema is defined, this module helps map how the signals are used. Which signals are read together? Which are written most often? These insights help identify optimization opportunities before the data even hits your database. Database exporter: Finally, based on what you’ve defined, the tool generates an initial database schema optimized for your workload. You can load it with sample data, export it to a live MongoDB instance, and run aggregation pipelines to validate the design. Together, these modules walk you through the journey—from signal selection to schema generation and performance testing—all within a simple, intuitive interface. Figure 4. Vehicle signal data modeler demo in action. Build smarter, adapt faster, and scale more confidently Connected vehicle systems aren’t just about collecting data—they’re about using it, fast and at scale. To get there, you need more than a standardized signal model. You need a data solution that can keep up with constant change, massive volume, and real-time demands. That’s where MongoDB stands out. Its flexible document model, scalable architecture, and built-in support for time series and AI workloads make it a natural fit for the complexities of connected mobility. Whether you're building fleet dashboards, predictive maintenance systems, or next-gen mobility services, MongoDB helps you turn vehicle data into real-world outcomes—faster. To learn more about MongoDB-connected mobility solutions, visit the MongoDB for Manufacturing & Mobility webpage. You can also explore the vehicle signals data modeller prototype and related resources on our GitHub repository .

July 1, 2025

Next →

Innovating with MongoDB | Customer Successes, July 2025

How time flies! Summer is in full swing, and it’s already time for another MongoDB customer success roundup. This month, we’re focusing on customers who have combined the flexibility of MongoDB Atlas with cutting-edge AI advancements to unlock insights and fuel innovation. Let’s be honest—AI is everywhere. We’re intrigued, inspired, and maybe a little overwhelmed by its possibilities. But the hype exists for a good reason: AI is a groundbreaking technology that’s poised to transform every industry, job, and task, and it’s fundamentally changing how software interacts with data. We’re quickly learning, though, that delivering meaningful outcomes with AI requires the right infrastructure. With MongoDB Atlas , companies are leveraging vector search, seamless document modeling, and large language model (LLM) integrations to make smarter use of their data in real time. Whether that means enhancing engagement, simplifying decision-making, or enabling more efficient processes, MongoDB is helping organizations redefine how they leverage AI to solve critical challenges and create lasting impact. In this issue, I’m particularly excited to share the highlight of an impactful platform developed by CentralReach in the Autism care space – a cause near and dear to my family. They, along with customers like the Financial Times, Ubuy, and Base39, are demonstrating AI’s possibilities and transforming how data powers success. Ubuy Ubuy , an e-commerce platform serving customers in over 180 countries, needed a faster, more scalable solution to manage its catalog of over 300 million products. They were facing significant search performance bottlenecks, which impacted user experience and limited growth potential. By migrating from MySQL to MongoDB Atlas and leveraging Atlas Search and Atlas Vector Search , Ubuy reduced search response times from 4–5 seconds to milliseconds and enabled intent-driven product discovery powered by AI. Now, Ubuy easily handles over 150 million searches annually while delivering personalized recommendations and seamless scalability. AI-driven search enhancements have boosted customer engagement and SEO visibility, transforming global e-commerce and redefining how Ubuy customers access international products. Financial Times The Financial Times (FT) , a global leader in business journalism, wanted to deliver a hybrid search experience that combined traditional keyword precision with AI-driven discovery. With over a million daily searches, scaling this innovative solution quickly was critical. Using MongoDB Atlas—including Atlas Vector Search—the FT developed its AI-powered hybrid search in just 18 weeks. By blending full-text and semantic search capabilities, the solution delivers relevant recommendations instantly, enhancing content discovery for time-strapped readers. Partnering with MongoDB streamlined deployment, enabling the FT to surface hyper-relevant results while positioning itself as a leader in media innovation. With plans to roll out hybrid search across mobile apps and specialist titles next, the FT continues redefining how readers engage with trusted journalism in an AI-enabled world. Through our partnership with The Stack, learn how our customers are achieving extraordinary results with MongoDB. This exclusive content could spark the insights you need to drive your business forward. CentralReach CentralReach , a global leader in autism and intellectual and developmental disability (IDD) care technology, faced the challenge of managing 4 billion clinical data points annually while reducing administrative burdens for behavioral analysts. By building its Care360 platform on MongoDB Atlas and joining the MongoDB AI Applications Program (MAAP), CentralReach unified data across 62 million service appointments per year. With flexible document modeling, vector search, and advanced AI pipelines, the platform enables seamless access to patient records and intelligent querying, reducing manual workflows and improving care consistency. CentralReach’s AI-powered solution has streamlined processes, reduced documentation errors, and helped expand access to care for hundreds of thousands globally. With MongoDB Atlas’s scalability and powerful AI integrations, CentralReach is redefining autism care delivery. Base39 Base39 , a Brazil-based fintech, set out to streamline complex credit analysis using AI-driven insights. Manual processes and data scarcity limited efficiency and accuracy, often delaying loan assessments by up to 10 days. By leveraging MongoDB Atlas on AWS , as well as Atlas Vector Search and LLM integrations, Base39 transformed its workflow. With agentic AI and predictive algorithms, loan applications are now assessed in minutes, achieving 96% cost reductions and improved data insights. MongoDB’s flexible schema and native vector search capabilities helped boost productivity while cutting infrastructure costs by 84%. By empowering developers to focus on innovation instead of management, Base39 has set a new standard in AI-powered credit analysis. Video spotlight: Cisco Before you go, check out how Cisco is redefining innovation with generative AI while prioritizing security. Omar Santos, Distinguished Engineer at Cisco, shares how MongoDB Atlas Vector Search accelerated development and saved millions through smarter, safer AI applications. Want to get inspired by your peers and discover all the ways we empower businesses to innovate for the future? Visit MongoDB’s Customer Success Stories hub to see why these customers, and so many more, build modern applications with MongoDB.

July 17, 2025