client-side-field-level-encryption

2226 results

Simplifying IoT Connectivity with myDevices and MongoDB

In the highly competitive era of Industry 4.0, companies that are able to adopt emerging Internet of Things (IoT) technologies and shift from traditional offerings to digitally differentiated ones are moving to the forefront of their respective industries. McKinsey & Company estimates that by 2030, IoT could enable $5.5 trillion to $12.6 trillion in value globally, including the value captured by consumers and customers of IoT products and services. From smart thermostats to smart factories, IoT already connects billions of devices worldwide. Figure 1 shows potential areas where IoT solutions make a difference. Figure 1:   IoT applications by industry (non-exhaustive). All of these IoT applications and solutions require technologies that can offer low-power operation, low-cost, and low complexity in setting up and maintaining end devices. End devices that are able to communicate wirelessly over large distances with low-power consumption are key. The data generated by IoT devices is time series and high frequency, placing a unique strain on the underlying data infrastructure. Because of the polymorphic nature of IoT sensor data, the database must support flexible data schemas, making it easy for developers to work with the data. It must also ensure that the IoT applications are resilient to future changes. MongoDB embraces the variety and volume of IoT data without compromising on performance. Through its document model, MongoDB eliminates data movement and blends time series with the rest of the enterprise data in a single developer data platform. In this article, we’ll describe how myDevices leverages the MongoDB developer data platform for IoT. Overview of myDevices myDevices is a U.S.-based IoT solutions company that empowers system integrators, MSPs, ISVs, VARS, and enterprise customers to quickly deploy IoT solutions to their customers. The company has more than 1000 plug and play sensors and multiple Long Range Wide Area Network (LoRaWAN) gateway options to create IoT solutions for a variety of use cases. Over time, myDevices has created the world’s most extensive IoT device catalog from more than 150 hardware manufacturers around the globe. LoRaWAN offers unique IoT benefits, such as long range and coverage, which may reach up to 15 kilometers in line of sight (LOS). It offers ultra-low power consumption for end devices, low-cost infrastructure, and high capacity, which makes it possible to link thousands of devices to one single gateway. myDevices understands that connecting devices from disparate manufacturers can be very challenging; thus, they have created a no-code solution that includes plug-and-play templates to connect sensors to the gateway just by scanning a QR code. After the sensor is connected to the gateway, users can perform remote monitoring and device management from a single-view interface. They can also get alerts through text and email and set up charts for visualization of sensor data. The alert rules can be configured as time based or threshold based in the myDevices platform. The myDevices IoT platform is secure from the edge to the application layer through the cloud. The security is composed of LoRaWAN network security at the edge, TLS to the cloud, and SAML at the application layer. Figure 2 shows the architecture of the myDevices platform and how it connects to the sensors. Figure 2:   MyDevices architecture. myDevices also has multiple ready-to-go solutions for a variety of IoT use cases and applications. From machine health predictive maintenance to soil moisture detection, there are sensors that just work with the IoT in a box application. It takes only minutes to set up connectivity between the sensor and myDevices cloud, and myDevices enhances productivity because you don’t have to worry about writing code to extract data from the sensors and establishing secure connectivity with the gateway. As LoRaWAN enables hundreds, if not thousands, of sensors sending data to a single gateway, it requires a database that can easily and automatically scale. When it comes to publishing data out of myDevices cloud to MongoDB Atlas, myDevices provides a webhook integration functionality that can be set up in minutes to establish connectivity between the two systems. Database requirements for IoT and MongoDB Atlas MongoDB and MongoDB Atlas are ideal partners for any IoT deployment, offering: Deployment flexibility (on-premises, in-field, cloud) Multi-cloud flexibility (AWS, Azure, GCP) Schema flexibility (frequent changes and additions) The ability to blend different data (time series, operational) Real-time analytics readiness Automated data tiering As a result, IoT data platforms and service providers, such as Bosch and Software AG, as well as some of the world’s most intensive IoT users, including Toyota, Mercedes-Benz, and Vodafone, choose MongoDB for their IoT platforms and services. MongoDB’s developer data platform supports the entire IoT data life cycle, from ingestion, storage, querying, real-time analytics, and visualization to online archiving (Figure 3). MongoDB Atlas brings the core components of real-time analytics into one developer data platform. Figure 3:   MongoDB Developer Data Platform for IoT. Let's talk about a few features that directly support IoT applications: Native time series platform: MongoDB supports native time series collections with hands-free schema optimization supporting high-efficiency storage and low-latency queries. This is an extremely important feature for IoT applications. Change streams: MongoDB change streams allow applications to access real-time data changes in the database without any complexity or risk. IoT applications can use change streams to subscribe to all data changes on a single collection, a database or an entire deployment and immediately react to them. This approach enables quick response time and fast decision making. Aggregation framework: By using the built-in aggregation framework in MongoDB, users are able to do real-time analytics without having to move the data to another platform. By using the aggregation framework, the work is done inside MongoDB, and the final results can be sent to the application, typically resulting in a smaller amount of data being moved around. For IoT applications, this can be a powerful tool to only transmit the filtered data to the Cloud or central storage resulting in improved security and reduced cost. Data Lake: As data is ingested, Atlas Data Lake automatically optimizes and partitions the data in a format and structure best for analytical queries. This capability significantly reduces the complexity of transforming data for the data scientist tasked with building machine learning models for analytical use cases and applications Data Federation: Atlas Data Federation provides the ability to federate queries across data stored in various supported storage formats, including Atlas Clusters, Data Lake Datasets, AWS S3 buckets, and HTTP stores. This feature reduces complexity of bringing data together for analytical model testing purposes. Data API: Companies can use Atlas Data API to integrate Atlas into any apps and services that support HTTPS requests. Leveraging this feature, the data from the myDevices cloud can be sent to Atlas and then used for storage and for analytical purposes using the aggregation framework or via the Atlas ecosystem connectors with third-party analytical software. Ecosystem integration: MongoDB Spark Connector opens up access to all Spark libraries for use with MongoDB datasets: Datasets for analysis with SQL (benefiting from automatic schema inference), streaming, machine learning, and graph APIs. Charts: MongoDB Charts is the best way to visualize IoT data stored in MongoDB. Charts is built specifically for the document model, no ETL, no time loss to data manipulation or duplication required to visualize rich JSON data. Using Charts, powerful engaging data experiences can be created for the use case stakeholders in no time. Integrating Atlas and myDevices using Webhooks and Data API myDevices offers a variety of no-code integrations for its clients to quickly get started by sending data to the platform of their choice. For MongoDB Atlas clients, this is great news because, by using myDevices Webhook integrator and payload transformation feature, MongoDB Atlas clients can receive and store LoRa sensor data into the specified collection. Let’s run through the methodology to perform this integration: Step 1: Log into your Atlas Cluster and set up Data API and API key. The MongoDB Atlas Data API lets you read and write data in Atlas with standard HTTPS requests. To use the Data API, all you need is an HTTPS client and a valid API key. It is important to understand that the Data API is not a direct connection to the MongoDB database. Instead, it routes requests through a fully managed middleware layer, called Atlas App Services, that sits between your cluster and client apps. This layer handles user authentication and enforces data access rules to ensure that the data is secure. The Data API supports two types of endpoints: Data API endpoints are automatically generated endpoints that each represent a MongoDB operation. You can use the endpoints to create, read, update, delete, and aggregate documents in a MongoDB data source. Custom endpoints are app-specific API routes handled by functions that you write. You can use custom endpoints to run your app's backend logic or as webhooks that integrate with external services. In this example, we are using a data API endpoint. You can follow these easy steps to enable Data API and create a Data API Key. Step 2: Log in your myDevices Console and set up integrations After you log in, click on new webhook creation through the INTEGRATIONS option on the right-hand panel (Figure 4). For the purpose of this article, we are assuming that you have already created an organization in myDevices and added sensors and gateways to it. If you have not, please refer to myDevices API docs to get started. Figure 4:   Set up integrations in myDevices. Step 3: Click on Webhook integration to open up the new Webhook creation panel. In this step, choose Webhook as the desired integration option, as shown in Figure 5. Figure 5:   Choose Webhook as the integration option. Step 4: Add key information. In this step, you’ll want to include key information, such as Url, which is your Data API endpoint, Webhook Header, which will include the api-key at the very minimum, and the payload transform script, where you can specify the cluster, database, and collection where this sensor data needs to be stored (Figure 6). Figure 6:   Paste the endpoint generated by Data API in Atlas. An example payload transformation script looks like the following. This is according to Data API requirements where you have to specify the cluster, database and collection name in the raw body data. function Transform(event, metadata) { return { dataSource: "my_cluster", database: "my_database", collection: "current_sensor", document: event, }; } Step 5: Save your webhook. Once you save your webhook, you can observe sensor data flowing into your MongoDB Atlas collection from the actual device using MongoDB Compass or Atlas Charts (Figure 7). For more details on how to create Charts, please visit the Atlas Charts documentation . Figure 7: Visualize sensor data using Atlas Charts. Conclusion We have shown how easy it is to connect myDevices IoT platform with MongoDB using the Data API . The overall architecture is shown in Figure 8. Figure 8: End-to-end architecture of myDevices and MongoDB Atlas integration. Simplifying IoT connectivity is of paramount importance for any organization looking to embark on a digital transformation journey. Fortunately, both myDevices and MongoDB Atlas provide platforms that simplify management of the full life cycle of an IoT device from provisioning to connectivity to data storage and archival. To learn more about how MongoDB enables IoT for our customers, please visit our IoT use cases page .

December 6, 2022

What’s New in Atlas Charts: Easy Organization-Wide Sharing

We’re excited to announce improvements to sharing dashboards in MongoDB Atlas Charts . Data visualization is a powerful tool for discovering insights, and sharing visualizations across your team helps amplify those insights to propel businesses forward. With organization-wide sharing in Atlas Charts, we’re making it even easier to share the insights you discover from your application data across your entire organization. Sharing dashboards Atlas Charts has always made it possible to share visualizations with either individual members or everyone inside your Atlas project. Assuming a user had access to a given data source in Atlas, adding a user to a Charts project was effectively a one-click process. However, many teams do not broadly share database access unless an individual specifically needs it. And, if you want to share data with many members of your team, provisioning users one by one is tedious. Once users are in a Charts project, however, sharing a dashboard with everyone inside the project becomes relatively easy — you can invite all users in your project to view your dashboard with a single action. There are probably scenarios in which some members of your organization have Atlas access and others do not. In this case, if your team has enabled Federated Authentication and uses a third-party authentication provider, such as Google or Okta, Charts now makes it simple to turn on sharing dashboards across your entire organization. Granting access This approach makes sharing company-wide information quick and easy. For example, you can keep employees aware of product or platform growth or other key business metrics. Any members of your organization can be granted access to view these dashboards with a single click, as shown in Figure 1. Figure 1:   A look at a dashboard shared across an organization. Note that, with these changes to dashboard sharing, your ability to maintain the security of your data remains unchanged. New dashboard viewers still need at least viewer access to any data source behind the charts in a shared dashboard, thereby ensuring that your company's sensitive data remains private. Additionally, project owners can now manage data source access at a deployment level, which means they can give access to their clusters or federated database instances . This capability is in addition to the already available granular control of data source access at a collection level, which was introduced as part of recent improvements we made to data sources. You can read more about managing access to data sources in your organization in our documentation . We hope you find these sharing improvements valuable and start leveraging this capability to share additional insights across your organization. New to Atlas Charts? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.

December 5, 2022

4 Ways MongoDB Enhances Your Google BigQuery Experience

MongoDB and Google Cloud continue to build on their partnership, with MongoDB enhancing Google Cloud with pay-as-you-go abilities, unified billing, and integrations with multiple different GC features, including BigQuery . And, when it comes to data architecture, BigQuery and MongoDB are two products that are better together. Google BigQuery and MongoDB are better together Google’s serverless data warehouse, BigQuery, was launched in 2011 with an aim to enhance business agility as their cloud-native data warehouse. BigQuery allows for fast queries that can uncover insights using familiar SQL. When MongoDB is added to the database technology stack as a complementary technology, it enhances the breadth of capabilities for the developer across a variety of use cases, including the following four examples. Combined impact of the Enterprise Data Warehouse and the Operational Data Store BigQuery is best suited as an Enterprise Data Warehouse (EDW), meaning it is designed to optimize long-running analytics. MongoDB Atlas , on the other hand, is best suited as an Operational Data Store (ODS), designed to optimally support high throughput and highly concurrent real-time operational applications that demand random access to an entity’s data in native JSON. This combination means that BigQuery and MongoDB are complementary technologies that can jointly deliver more value — each delivering on their strongest qualities. BigQuery excels at long-running queries, while Atlas handles the real-time operational application needs with thousands of concurrent sessions and millisecond response times. Enriched end-customer experiences BigQuery enables data scientists and analysts with machine learning (ML) models and BI tools for structured and semi-structured data at scale. For roles that need results with a turnaround time of a day or more, BigQuery is a strong tool for big data queries. With MongoDB Atlas, engineers and development teams can build applications faster and handle highly diverse schema, query, and update patterns, adapting to demanding user needs and competition. Atlas can also deliver the real-time or less than 24-hour queries that are necessary to keep your business operational. Additionally, data can easily move back and forth between the two platforms, creating a prime combination for running analytics on operational data. Being able to unlock the full potential of your data across your organization means that everyone has the insight into the business metrics they need, when they need it. This allows quicker decision making, as well as stronger and more accurate reporting. Extensibility to MongoDB Atlas features On top of the value and synergy that can be realized by a BigQuery+Atlas combination, other Atlas features can help enhance the usefulness and sophistication of a data architecture, such as: Atlas Charts can be leveraged to create rich visualizations of any data stored within Atlas. Atlas Triggers and Alerts can apply database logic in response to events or on a predefined schedule. Atlas Search brings full-text search at scale to all data across MongoDB and BigQuery alike. Atlas Data Federation enables aggregating data across multiple data sources, such as Atlas clusters and HTTPS endpoints, and transforming it into analytical formats (e.g., Parquet). This means you can not only access data in real-time, but you can also analyze it in a visual, user-friendly way. This functionality makes your data more actionable, allowing you not only to answer questions about your business data but also make better predictions and future adjustments based on it. Furthermore, being alerted to certain data-based events and triggering new actions based on that information means you can have your data working more efficiently for you, freeing up time to innovate and focus on core business competencies. Lastly, this approach simplifies your data lifecycle, so JSON data from various applications and endpoints can easily be transformed and consumed for rich analytics. Deeper understanding of your customer Businesses can use fully managed MongoDB Atlas to store customer 360 profiles. A 360-degree view of a customer allows businesses to track an individual customer’s journey across multiple channels, devices, purchases, and interactions, and improves customer satisfaction. With the combination of Atlas and BigQuery, businesses can also use compiled data — such as, transactional data, behavioral data, user profile and segmentations, and business analytics — to match user profiles with products and services using Artificial Intelligence (AI). Vertex AI , a managed machine learning platform, provides all the Google cloud services in one place to deploy and maintain AI models. Being able to easily access a 360 view for each customer and have automation around their customer journey helps with customer engagement and loyalty by improving customer satisfaction and retention through personalization and targeted marketing communications. It also enables retailers to aggregate customer interactions across all channels and identify valuable new customers. Google BigQuery and MongoDB Atlas in the real world Current , a leading U.S. challenger bank, uses innovative approaches, services, and technologies to serve people overlooked by traditional banks, regardless of age or income level, to help improve their financial outcomes. To help create customer experiences that cannot exist in traditional systems, Current chose to leverage Google Cloud, including BigQuery, with MongoDB layering the platform to achieve their goals. Read Full Current Story Are you a Google BiqQuery customer that is curious about how MongoDB Atlas can amplify your existing data warehouse or data lake architecture? Try MongoDB Atlas for free today and spin up your first workload in minutes. Try pay-as-you-go Atlas on GC Marketplace

December 1, 2022

Achieving Industrial Connectivity at Scale with Wimera and MongoDB

Industry 4.0 (I4.0) represents the beginning of the Fourth Industrial Revolution. It includes the current trend of automation technologies in the manufacturing industry as well as disruptive technologies and concepts, such as cyber-physical systems (CPS), Industrial Internet of Things (IIoT), cloud computing, and immersive visualization. Through Industry 4.0, embedded systems, semantic machine-to-machine communication, IIoT, and CPS technologies are integrating the virtual space with the physical world. These technologies are enabling a new generation of industrial systems, such as smart factories, to deal with the complexity of fast-paced and hyper-personalized production. In this article, we’ll explore Wimera’s unique solutions to the challenges of I4.0 and IIoT, built with MongoDB. Information and insights With IIoT, existing industrial systems will be modernized to drive digital transformation and unlock tomorrow's smart enterprise. IIoT has been finding its way into products and sensors while revolutionizing existing manufacturing systems; thus, it is considered a key enabler for the next generation of advanced manufacturing. Industry 4.0 generally comprises many complex components and has broad applications in all manufacturing sectors. The first challenge faced by manufacturing companies when embarking on the I4.0 journey is to sensorize and connect their manufacturing equipment in order to collect, store, and analyze data for information and insights. Wimera Systems is solving this challenge as an I4.0 enablement company offering IIoT solutions using their unique hardware, software application, and AI/ML-based analytics engine. Wimera’s Smart Factory Suite has seen tremendous growth, with 2500+ global installations across 50+ customers. MongoDB has been pivotal to that growth, acting as the core component of the IIoT suite and enabling the company to offer its services at scale without having to worry about managing the complexity of an IIoT database. Bringing AI-powered IIoT to the manufacturing shop floor Manufacturing companies are emerging from the pandemic with a renewed focus on digital transformation and smart factories investment. COVID-19 has heightened the need for IIoT technology and innovation, forcing manufacturers to compete in a digitalized business environment. Many manufacturers still operate using legacy technologies and systems; on most shop floors, equipment and operator efficiency are manually calculated and tracked using spreadsheets. The machines are maintained using time-based rather than condition-based maintenance strategies. And, no real-time visibility exists on consumables and tools usage. All these practices result in increased maintenance costs, suboptimal production, and ultimately, customer dissatisfaction. Wimera understands these challenges all too well, which is why they created the Smart Factory Suite supporting both on-premise and cloud deployments. The Smart Factory Suite provides insights for managing the entire production landscape through interconnected devices and machines, operations, and facilities. It can predict and make real-time adjustments for increased production efficiency and less downtime. The suite is primarily utilized for empowering manufacturing operations, equipment maintenance, warehouse operations, and inventory management. With Smart Factory Suite, Wimera serves a wide range of manufacturing industry sectors including, but not limited to, automotive, electronics, chemical, and food processing companies. Deploy and run anywhere with MongoDB MongoDB, with its freedom to run anywhere, lets Wimera offer both on-premises and cloud deployment options for its customers. In both cases, the suite is directly connected with machine controllers using Wimera libraries for all popular Programmable Logic Controller (PLC) brands. The suite is also connected to legacy machines through external sensors installed by the Wimera team. Data is extracted via the Wimera ReMON Data Acquisition (DAQ) device (Figure 1) that utilizes the MongoDB database as the persistent data storage. MongoDB’s flexible data model makes it easy to combine and enrich this data and enables live dashboards and instant alerts for factory personnel. The data collected and optimized by ReMON DAQ is further fed to ReMON AI , an advanced analytics engine. ReMON AI provides advanced analytics through AI/ML models and leverages MongoDB to deliver application-driven analytics in real time. Figure 1: ReMON DAQ and ReMON AI (source: Wimera ReMON ). Whether through on-premises or cloud deployment (Figures 2 and 3), Wimera’s customers have benefited from MongoDB’s capabilities that are critical for IIoT applications, such as time series collections and the flexible, intuitive document data model. Figure 2: Wimera IoT architecture on premises. Figure 3: Wimera IoT architecture on cloud (using MongoDB on AWS). In one customer example, while deploying IIoT at a multinational CNC machine shop, the customer preferred to use their existing production monitoring application enriched with IoT data coming from Wimera’s Smart Factory Suite. In this case, MongoDB enabled easy and seamless integration of the IoT application with the customer's application via a simple API. Additionally, high-speed data coming from a vibration sensor was handled effectively by MongoDB time series collections, resulting in real-time alerts sent to maintenance teams for instant corrective actions on the shop floor. In another example, a multinational automotive manufacturer wanted a single platform that could collect and combine data coming from vendors in different formats and contexts. MongoDB's flexible document model helped manage the varied data types easily, allowing the customer to benefit from a single application capable of managing multiple vendors in parallel. This flexibility offered by MongoDB enables the customer to keep adding new vendors instantly without changing the underlying cloud infrastructure or tweaking schemas. Interested readers can check out additional case studies on Wimera’s website. Building better together Wimera and MongoDB’s partnership gives customers confidence with validated architectures to ensure successful, optimized, and scalable deployments at their facilities. Wimera’s continued partnership with MongoDB also helps guide the company’s product roadmap as we expand in the IIoT, Smart Factory market together. MongoDB is the only enterprise grade database chosen by the Wimera development team due to easy handling of the large volume of data generated from machines and sensors while maintaining a high performance… If we want to insert thousands of records in a second, then MongoDB is the best choice for that given our solutions are for Industrial IoT. Also, horizontal scaling (adding new columns) is not an easy process in any RDBMS system. But in the case of MongoDB, it is very easy Nagarajan Narayanasamy, CEO, Wimera Systems Private Limited A bright future ahead Since 2019, Wimera has been an early adopter of MongoDB for their Industrial IoT application for discrete manufacturing industries and process industries on multiple domains. “Currently, Narayanasamy says, “Wimera’s Industrial IoT solutions are matured, and we are focused on scaling globally.” Wimera now targets expansion in India, APAC, EU, and USA for the discrete manufacturing and process industries and also for select OEMs and machine builders. “As MongoDB continues to scale itself globally through its multi-cloud data distribution strategy, we see a good synergy partnering with MongoDB for the mutual benefit of both companies and the community as a whole. We also would like to work with MongoDB on the technology roadmap and solve some of the real-life challenges faced by manufacturing industries,” Narayanasamy says. Wimera has recently started their MongoDB Atlas journey, and the adoption will grow as their customers demand more cloud solutions compared to current on-premises deployments. MongoDB will continue to help IoT companies like Wimera take their product offering to the next level and enable their customers to digitally transform their manufacturing operations. To learn more about MongoDB’s role in industrial connectivity and IIoT, please visit our Manufacturing and Industrial IoT page.

December 1, 2022

Choosing the Right Tool for the Job: Understanding the Analytics Spectrum

Data-driven organizations share a common desire to get more value out of the data they're generating. To maximize that value, many of them are asking the same or similar questions: How long does it take to get analytics and insights from our application data? What would be the business impact if we could make that process faster? What new experiences could we create by having analytics integrated directly within our customer-facing apps? How do our developers access the tools and APIs they need to build sophisticated analytics queries directly into their application code? How do we make sense of voluminous streams of time-series data? We believe the answer to these questions in today's digital economy is application-driven analytics. What is Application-Driven Analytics? Traditionally, there's been a separation at organizations between applications that run the business and analytics that manage the business. They're built by different teams, they serve different audiences, and the data itself is replicated and stored in different systems. There are benefits to the traditional way of doing things and it's not going away. However, in today's digital economy, where the need to create competitive advantage and reduce costs and risk are paramount, organizations will continue to innovate upon the traditional model. Today, those needs manifest themselves in the demand for smarter applications that drive better customer experiences and surface insights to initiate intelligent actions automatically. This all happens within the flow of the application on live, operational data in real time. Alongside those applications, the business also wants faster insights so it can see what's happening, when it's happening. This is known as business visibility, and the goal of it is to increase efficiency by enabling faster decisions on fresher data. In-app analytics and real-time visibility are enabled by what we call application-driven analytics. Find out why the MongoDB Atlas developer data platform was recently named a Leader in Forrester Wave: Translytical Data Platforms, Q4 2022 You can find examples of application-driven analytics in multiple real-world industry use cases including: Hyper-personalization in retail Fraud prevention in financial services Preventative maintenance in manufacturing Single subscriber view in telecommunications Fitness tracking in healthcare A/B testing in gaming Where Application-Driven Analytics fits in the Analytics Ecosystem Application-driven analytics complements existing analytics processes where data is moved out of operational systems into centralized data warehouses and data lakes. In no way does it replace them. However, a broader spectrum of capabilities are now required to meet more demanding business requirements. Contrasting the two approaches, application-driven analytics is designed to continuously query data in your operational systems. The freshest data comes in from the application serving many concurrent users at very low latency. It involves working on much smaller subsets of data compared to centralized analytics systems. Application-driven analytics is typically working with hundreds to possibly a few thousand records at a time. And it's running less complex queries against that data. At the other end of the spectrum is centralized analytics. These systems are running much more complex queries across massive data sets — hundreds of thousands or maybe millions of records, and maybe at petabyte scale — that have been ingested from many different operational data sources across the organization. Table 1 below identifies the required capabilities across the spectrum of different classes of analytics. These are designed to help MongoDB’s customers match appropriate technologies and skill sets to each business use case they are building for. By mapping required capabilities to use cases, you can see how these different classes of analytics serve different purposes. If, for example, we're dealing with recommendations in an e-commerce platform, the centralized data warehouse or data lake will regularly analyze vast troves of first- and third-party customer data. This analysis is then blended with available inventory to create a set of potential customer offers. These offers are then loaded back into operational systems where application-driven analytics is used to decide which offers are most relevant to the customer based on a set of real-time criteria, such as actual stock availability and which items a shopper might already have in their basket. This real-time decision-making is important because you wouldn't want to serve an offer on a product that can no longer be fulfilled or on an item a customer has already decided to buy. This example demonstrates why it is essential to choose the right tool for the job. Specifically, in order to build a portfolio of potential offers, the centralized data warehouse or data lake is an ideal fit. Such technologies can process hundreds of TBs of customer records and order data in a single query. The same technologies, however, are completely inappropriate when it comes to serving those offers to customers in real time. Centralized analytics systems are not designed to serve thousands of concurrent user sessions. Nor can they access real-time inventory or basket data in order to make low latency decisions in milliseconds. Instead, for these scenarios, application-driven analytics served from an operational system is the right technology fit. As we can see, application-driven analytics is complementary to traditional centralized analytics, and in no way competitive to it. The benefits to organizations of using these complementary classes of analytics include: Maximizing competitive advantage through smarter and more intelligent applications Out-innovating and differentiating in the market Improving customer experience and loyalty Reducing cost by improving business visibility and efficiency Through its design, MongoDB Atlas unifies the essential data services needed to deliver on application-driven analytics. It gives developers the tools, tech, and skills they need to infuse analytics into their apps. At the same time, Atlas provides business analysts, data scientists, and data engineers direct access to live data using their regular tools without impacting the app. For more information about how to implement app-driven analytics and how the MongoDB developer data platform gives you the tools needed to succeed, download our white paper, Application-Driven Analytics: Defining the Next Wave of Modern Apps .

November 30, 2022

Built With MongoDB: Inspirit Helps Kids Learn Science Through Immersive Technology

Immersive technologies like AR and VR have tremendous potential to transform learning outcomes for students. By representing complicated, often intangible concepts in an interactive, 3D platform and VR system, they encourage engagement and improve memory recall in STEM (science, technology, engineering, and mathematics) subjects. Palo Alto-based startup, Inspirit , has developed an immersive, interactive VR platform that combines the best of both 3D and VR instruction to allow students in middle school and high school to experience science and learning through virtual reality instead of reading. Inspirit is also part of the MongoDB for Startups program , which helps startups build faster and scale further with free MongoDB Atlas credits, one-on-one technical advice, co-marketing opportunities, and access to a vast partner network. Birth of a Startup Inspirit Co-founders Amrutha Vasan and Aditya Vishwanath were conducting research at Georgia Tech and trying to understand how to bring virtual reality into education when they had a critical insight. "A lot of students tend to lose sight or just lose interest in science by the time they hit high school," Vasan says. "What we realized very quickly was that you can give teachers the tools that they need in order to do things that are way too expensive or just simply impossible to do in the real world." As students get into higher order physics, calculus, biology, and chemistry, they struggle with visualizing difficult concepts. "What we aim to do is provide interactive 3D models and interactive simulations that actually teach them core science concepts so that they build that curiosity and actually engage with the material in order to keep them in STEM programs and moving forward," Vasan says. Company evolution Inspirit has evolved since its early days. The founders initially focused on VR headsets and 3D goggles that students would share in the classroom. Once the global COVID-19 pandemic hit, it was clear that students wouldn't be going back to the classroom anytime soon. "We had to very quickly pivot into building a web platform that students could use at home and teachers could use for hybrid and online teaching," Vasan says. "Through the pandemic, we learned very quickly exactly how the education system would actually be changing. And so through that adaptation, we've now built a cross-platform product." Building a cross-platform solution aligns with one of Inspirit's core values, which is to be accessible. Inspirit is now available on the web in addition to virtual reality headsets so students can access the tools on different platforms. Inspirit isn't just an extracurricular activity. One of the biggest differentiators between Inspirit VR classroom and other platforms is that it addresses core content. It's not something that students need to use independently. "You still need a really great teacher," Vasan says. "There is no way for you to show a student a eukaryotic cell other than by drawing it on a whiteboard or giving them a diagram or a video," she says. "You still need a great teacher in order to use our platform. But they can now use this to help their students visualize difficult science concepts." Inspirit helps students who love science immerse themselves in it more while at the same time helping kids who dislike science experience it in a new, more engaging way. Building with MongoDB Inspirit chose MongoDB over other platforms because of its support and flexibility. As a startup, the founders knew their database structure would be changing constantly. "MongoDB allows for a lot of flexibility," Vasan says, "which is really important because we're probably not going to have the perfect database going into a startup. Allowing us to consistently change it as we're scaling and growing has just been very helpful." Regarding MongoDB support, which is one of the benefits of the MongoDB for Startups program, Vasan says the company got the most support from MongoDB than any of the other databases they were looking at. "That's ultimately why we ended up choosing it for our company," she says. "The things that we love the most about MongoDB are Atlas , Charts , and search indexes . Atlas has been really cool for visualizing all the data and for helping us grow and scale quickly as well." "We've had a really good partnership with the MongoDB for Startups program in particular," Vasan says. Inspirit uses AWS, and Vasan cites how easily Atlas integrates with AWS as another key benefit that she appreciates. "AWS integrates very easily with MongoDB. So to be honest, it's been pretty easy scaling our backend." Compliance was another critical requirement. "We actually need to be COPPA compliant," Vasan says, referring to the Children's Online Privacy Protection Rule. "There are a lot of data regulations for what we're using since we have sensitive student, parent, and teacher data. And so both MongoDB and AWS actually make that a lot easier." If you're interested in learning more about Inspirit's immersive learning tools, visit their website . Are you part of a startup and interested in joining the MongoDB for Startups program? Apply now . For more startup content, check out our previous blog on Qubitro .

November 30, 2022

MACH Aligned for Retail: Headless

The MACH Alliance is a non-profit organization fostering the adoption of composable architecture principles, namely Microservices , API-First , Cloud-Native SaaS , and Headless. MongoDB, among many other technology companies, is a member of this Alliance, enabling developers to adopt these principles in their applications. In this article, we’ll focus on the fourth principle championed by the MACH Alliance: Headless. Let’s dive in. What is headless? A headless architecture is one where the layers or components of the architecture are decoupled. The “heads” (i.e., frontends) operate independently from the backend logic or “core body” microservices and share data via API. This concept is key to a successful shift toward microservices — without decoupling the architectural layers, you’re running on a modern monolith. Looser coupling also leads to an increase in frontend change and flexibility, reusability of core features, less downtime because there’s no single point of failure, and promotes reusability of key features. Headless applied to retail Retail was one of the first industries to embrace headless architectures, with the term coined in 2012 by Dirk Hoerig, founder of commercetools . These concepts were originally applied to building modern ecommerce solutions and are now being expanded to any application in the IT stack. In this model, the head can be an ecommerce web frontend, or mobile app, or an internal frontend system for stock management. The core body components support the heads (Figure 1). They can be a payment system, a checkout solution, a product catalog, or a warehouse management application. Figure 1:   The “head” and “core body” components, sharing data as part of APIs. Customers and their experiences are at the heart of retail. Adopting headless principles can greatly help companies meet rapidly changing customer requirements and stand out from the competition. Customers require a seamless journey between mobile, web applications, and in-store with data and logic consistent across channels. New channels might also need to be added such as integration with social media, to reach a younger customer base. Retailers might need to be able to sell in multiple regions or across product lines, requiring them to adopt multiple frontends to serve different customer groups without having to rewrite or duplicate the whole IT stack. New features might need to be added quickly to reflect competitors’ moves without tracing changes back through every component of the stack or experiencing downtime. Internal workforce systems can follow similar principles. The common denominators of these example use cases include speed of change and frontend flexibility, avoiding downtime, and reusability of the backend components. Headless solutions enable developers to avoid duplicating efforts by reusing the core capabilities of applications and adapting them to various target systems and use cases. Those principles save developers’ time and can be leveraged to provide a seamless experience to customers, as the underlying data layer and workflows are shared across multiple services offering similar functionalities. Headless architectures also come with the following advantages. Bring new features to market faster New features and MVPs can be introduced with minimal impact on other application components. Release cycles can be managed efficiently via a microservice architecture relying on different squads, and new releases can be pushed to production when ready, independently of the work of other squads. For example, a retailer can expand into a new country quickly by developing a country-specific frontend that reuses existing core components and requires no backend downtime. Scale to meet seasonal demand Companies can independently scale application components where and when required. For example, increased user traffic might require more resources to support frontend components, leaving the backend untouched and vice versa. In an ecommerce scenario, this can take the form of expected deviations from a seasonality standpoint (e.g., end-of-month transactions following salary distribution, holiday shopping) or unplanned variations (e.g., influencer marketing). Thus, this model can result in: Cost savings: Achieve cost reductions as a headless architecture running on the cloud enables to further decouple its pay-as-you-go model, by only paying for the infrastructure required by each front/backend component. Improved customer experience: Develop highly available and responsive applications so that customer experience is not affected by computing resources. Leverage best-of-breed technologies Headless architectures can help companies gain greater flexibility in deploying and managing the IT stack, allowing them to: Focus on value-add development: A composable headless architecture enables companies to choose to build or buy individual components in the stack. As the components are decoupled, it becomes easier to unpick than if the stack is fully integrated — as the APIs can be redirected to the new solution more easily. This approach lets companies put their development activity into value-added functionality should a best-of-breed vendor solution arrive on the market delivering core functionality. Avoid vendor lock-in: This also allows for more seamless technology switches should companies decide to bring development back in-house or switch vendors. Improve talent acquisition and retention: Deploying in a flexible and composable manner lets development teams choose the programming languages and tools they feel best match the requirements, allowing companies to attract and retain top talent. Less downtime with faster troubleshooting A headless architecture also makes it easier to pinpoint which single layer/component is the root cause of issues, as opposed to troubleshooting in monolithic applications where dependencies can be difficult to map. Fewer dependencies mean less downtime; when a change or failure occurs to one component, it doesn't affect the whole stack. For ecommerce retailers, any downtime can have a direct impact on revenue, so an architecture that supports a move towards 24/7 uptime is ideal. Removing data silos and sharing data across multiple journeys also enables companies to implement truly omnichannel experiences and leverage the datasets for other downstream processes, such as user personalization and analytics. Learn how Boots is using MongoDB Atlas to standardize their infrastructure via an API and microservice-driven approach . How can MongoDB help? Headless architectures require a strong data layer to reap all the above-mentioned benefits. MongoDB includes several key features that enable developers to speed up the pace of delivery of new features and bug fixes, scale with minimal effort, and leverage APIs to share data with the different components of the stack. Deliver faster with no downtime MongoDB provides a flexible document model that easily adapts to the needs of different microservices and supports adding new features and data fields without having to rethink the underlying data schema or experience downtime. Let’s consider a product catalog microservice that uses a particular API to read data from certain fields. A second microservice can be developed requiring the same set of fields as the first along with a few new ones connecting via a new API. MongoDB allows the change to be made with no downtime of the product catalog microservice and related API. Scale effortlessly Adding new features and services will likely require scaling the data layer to cater to higher storage and workload. MongoDB, through its sharding capabilities , enables a distributed architecture by horizontally scaling the data layer and by distributing data across multiple servers. This approach can provide better efficiency than a single high-speed, high-capacity server (vertical scaling), to build highly responsive retail solutions. Support composable architectures MongoDB also possesses strong API capabilities to support a microservice-based backend architecture and make data accessible and shareable across components (Figure 2). These capabilities include APIs and drivers supporting a dozen programming languages on the market, such as C, Python, Node.js, and Scala. The MongoDB Unified Query API allows working with data of any type, including time series, arrays, and geospatial. MongoDB Atlas, MongoDB’s Developer Data Platform, comes with the Atlas Data API allowing to programmatically create, read, update, and delete data stored on Atlas clusters as part of standard HTTPS requests. The Atlas GraphQL API allows fine-tuning of API requests by returning only the required data (e.g., information about a particular customer or product). Figure 2:   MongoDB supports a headless architecture via APIs. Data availability and resiliency should also be considered when adopting headless architectures. MongoDB Atlas clusters are highly available and backed by an industry-leading uptime SLA of 99.995% across all cloud providers. If a primary node becomes unavailable, MongoDB Atlas will automatically failover in seconds. Clusters can be also deployed across multiple cloud regions to weather the unlikely event of a total region outage, or in multiple cloud platforms together. Summary Adopting a headless architecture is paramount for retailers wanting to enhance customer experience and build more resilient applications. MongoDB, with its leading database offering, API layer, and high availability is strongly suited to meet the requirements of modern applications. Read our previous blog posts in the MACH series covering Microservices , API-First , and Cloud-Native SaaS .

November 30, 2022

MongoDB Named as a Leader in The Forrester Wave™: Translytical Data Platforms, Q4 2022

In The Forrester Wave™: Translytical Data Platforms, Q4 2022, translytical data platforms are described by Forrester as being “designed to support transactional, operational, and analytical workloads without sacrificing data integrity, performance, and analytics scale.” Characterized as next-generation data platforms, the Forrester report further notes that “Adoption of these platforms continues to grow strongly to support new and emerging business cases, including real-time integrated insights, scalable microservices, machine learning (ML), streaming analytics, and extreme transaction processing.” To help users understand this emerging technology landscape, Forrester published its previous Translytical Data Platforms Wave back in 2019. Three years on, Forrester has named MongoDB as a Leader in its latest Translytical Data Platforms Wave. We believe MongoDB was named a Leader in this report due to the R&D investments made in further building out capabilities in MongoDB Atlas , our multi-cloud developer data platform. These investments were driven by the demands of the developer communities we work with day-in, day-out. You told us how you struggle to bring together all of the data infrastructure needed to power modern digital experiences – from transactional databases to analytics processing, full-text search, and streaming. This is exactly what our developer data platform offers. It provides an elegant, integrated, and fully-managed data architecture accessed via a unified set of APIs. With MongoDB Atlas, developers are more productive, they ship code faster and improve it more frequently. Translytics and the Rise of Application-Driven Analytics Translytics is part of an important shift that we at MongoDB call application-driven analytics . By building smarter apps and increasing the speed of business insights, application-driven analytics gives you the opportunity to out-innovate your competitors and improve efficiency. To do this you can no longer rely only on copying data out of operational systems into separate analytics stores. Moving data takes time and creates too much separation between application events and actions. Instead, analytics processing has to be “shifted left” to the source of your data – to the applications themselves. This is the shift MongoDB calls application-driven analytics . It’s a shift that impacts both the skills and the technologies developers and analytics teams use every day. This is why understanding the technology landscape is so important. Overall, MongoDB is good for customers that are driving their strategy around developers who are tasked with building analytics into their applications. The Forrester Wave™: Translytical Data Platforms, Q4 2022 Evaluating the top vendors in the Translytic Data Platforms Wave Forrester evaluated 15 of the most significant translytical data platform vendors against 26 criteria. These criteria span current offering and strategy through to market presence. Forrester gave MongoDB the highest possible scores across eleven criteria, including: Number of customers Performance Scalability Dev Tools/API Multi-model Streaming Cloud / On-prem / distributed architecture Commercial model The report cites that “MongoDB ramps up its translytical offering aggressively”, and that “Organizations use MongoDB to support real-time analytics, systems of insight, customer 360, internet of things (IoT), and mobile applications.” Access your complimentary copy of the report here . Customer Momentum Many development teams start out using MongoDB as an operational database for both new cloud-native services as well as modernized legacy apps. More and more of these teams are now improving customer experience and speeding business insight by adopting application-driven analytics. Examples include: Bosch for predictive maintenance using IoT sensor data. Keller Williams for relevance-based property search and sales dashboarding. Iron Mountain for AI-based information discovery and intelligence. Volvo Connect for fleet management. Getting started on your Translytics Journey The MongoDB Atlas developer data platform is engineered to help you make the shift to Translytics and application-driven analytics – leading to smarter apps and increased business visibility. The best way to get started is to sign up for an account on MongoDB Atlas . Then create a free database cluster, load your own data or our sample data sets, and explore what’s possible within the platform. The MongoDB Developer Center hosts an array of resources including tutorials, sample code, videos, and documentation organized by programming language and product. Whether you are a developer or a member of an analytics team, it's never been easier to get started enriching your transactional workloads with analytics!

November 29, 2022

MongoDB and AWS: How a decade-old collaboration got even better in 2022

Developers select MongoDB because it makes building with data for almost any class of application easy and fast for them. They select Amazon Web Services (AWS) because it offers a comprehensive and broadly adopted cloud platform, offering more than 200 fully featured services. Bringing together MongoDB Atlas on AWS helps developers build and ship higher quality applications faster and scale them further. MongoDB has collaborated with AWS for close to a decade now, but 2022 has seen dramatic growth in both the quantity and quality of our joint activities, resulting in a strategic collaboration agreement announced earlier this year. Our collaboration spans joint product engineering and integration so MongoDB Atlas is a first-party service on AWS, and also extends to making it easy for customers to procure MongoDB Atlas on AWS. In 2022, we have worked more closely together than ever before. In this post, we'll cover what we've achieved, and how our customers benefit. If at any point you want to stop reading about the partnership and experience it in action, we invite you to get started for free with MongoDB's fully managed, pay-as-you-go listing on the AWS Marketplace . Delivering an outstanding customer experience Since re:Invent 2021, MongoDB and AWS have jointly seen an explosion in customer success, with MongoDB for Startups becoming one of the most widely used offerings in the AWS Activate program after we launched in July. And, since launching in the AWS Marketplace with pay-as-you-go pricing in December 2021, MongoDB Atlas has become one of the most popular self-service listings, with well over 1,000 customers. More broadly, we've seen our AWS Marketplace business show triple-digit growth through significant, mutual investments across engineering, sales, and marketing. We've also found great success working with AWS' Workload Migration and Proof of Concept programs, helping many new customers accelerate their migration to MongoDB Atlas on AWS over the past 12 months. Additionally, while MongoDB works closely with AWS across the globe, we devoted increased attention to Europe this past year, resulting in a considerable increase in customer adoption. As a result, AWS named us their AWS Marketplace Partner of the Year - EMEA in November 2022. One way that we've helped to accelerate such customer success is by making it easier to procure MongoDB Atlas on AWS. Over the past year, MongoDB and AWS have significantly simplified the purchasing experience for customers. We did this across a few key areas. One thing customers love about buying through AWS Marketplace is how seamless it makes the purchasing experience. However, historically this has been slowed somewhat for MongoDB customers by the need to agree to separate legal terms. Starting in November 2022, however, all Atlas on AWS customers purchasing through the AWS Marketplace Self Service listing use AWS Marketplace’s Standard Contract for Marketplace (SCMP) terms and conditions rather than MongoDB Cloud Terms of Service, thereby further reducing friction to getting productive, faster, with MongoDB. In 2022, we also helped customers buy MongoDB with confidence through AWS Marketplace Vendor Insights . AWS Vendor Insights "simplif[ies] third-party software risk assessments by compiling security and compliance information in a unified dashboard." It's an important way we're working together to increase customer confidence, ensuring they can buy MongoDB in AWS Marketplace with security and control. Close product collaboration Behind these improvements to our joint purchasing experience were significant improvements to how MongoDB Atlas integrates with key AWS services. MongoDB has long worked seamlessly with core AWS services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3), and more recently has collaborated with AWS to ensure tight integration with AWS container services like Amazon Elastic Kubernetes Service (Amazon EKS) and Amazon Elastic Container Service (Amazon ECS), AWS serverless technologies like AWS Lambda, Amazon Eventbridge, and AWS Fargate; and edge computing services like AWS Wavelength . Over the past year, however, we've delved more deeply into AWS machine learning services (Amazon Comprehend, Amazon Kendra, Amazon Lex, etc.), AWS AppSync, Amazon Forecast, AWS Elastic Beanstalk, and more. In addition to direct integrations with AWS services, we made it simpler for customers to use MongoDB with important joint partners such as Datadog, Databricks, and Confluent. For Datadog, we improved MongoDB Atlas App Service to support forwarding logs on AWS to Datadog, thereby improving observability through real-time log analytics. With Databricks, we announced MongoDB as a data source within a Databricks notebook, thereby offering data practitioners an easier, more curated experience for connecting Databricks to MongoDB Atlas data. And with Confluent, we strengthened our integrations to help developers easily build robust, reactive data pipelines that stream events between applications and services in real time. Through innovations to the purchasing process and the product experience, we've helped make thousands of customers successful running MongoDB on AWS. Some joint customers, like Unqork , are upending entire industries with innovative approaches to technology and business. Others, like Volvo's Connected Solutions business , rely on MongoDB and AWS to scale their fleet management solution from tens of millions to billions of daily events. Other recent customers include Verizon , Marsello , GLS , and Shopline . Get started with MongoDB Atlas on AWS You needn't take our word for it, however. With just a few clicks — and no risk — you can get started for free with MongoDB Atlas on AWS . There's no upfront commitment, and if you choose to continue to build with MongoDB on AWS, you only pay for what you use.

November 28, 2022

3 Key Characteristics of Modernization

Analyst and research firm TDWI released its latest report on IT modernization: Maximizing the Business Value of Data: Platforms, Integration, and Management . The report reveals the modernization strategies, objectives, and experiences of more than 300 IT executives, data analysts, data scientists, developers, and enterprise architects. Within the survey itself lies the deeper, fundamental question of what is IT modernization in today's digital economy? It's an important question because it gets at the heart of why organizations want and need to modernize in the first place. Considering the effort, expense, and risks of modernizing, there needs to be a compelling purpose guiding the process in order to keep it on track and ensure its success. By dissecting the TDWI survey questions and responses, we can deduce what the three key characteristics of modernization are. #1: Modernization capabilities If we were to examine the elements and components that comprise modernized architecture, we would get a sense of what modernization looks like but not the purpose behind its deployment. So instead, let's start by looking at the capabilities modern architecture enables so we can get a clearer view of its characteristics and why they matter. Seventy-three percent of survey respondents reported that data democratization and self-service functionality are either extremely or very important. We've heard from numerous organizations that the task of managing data access at companies is slowing down innovation. Ben Herzberg, chief data scientist for data access company, Satori, recently told us , "The majority of organizations are still managing access to data in a manual way. Everyone is feeling the bottleneck. The data analyst who wants to do their job in a meaningful way just wants to understand what data sets they can use and get access to it fast." Getting access to data can be challenging without some sort of self-service data access capability. "Sometimes you have to go through three or four different teams to get access to data," Herzberg says. "It can take a week or two." The TDWI report also indicated a long-standing trend toward easier, more intuitive experiences extending to data integration, data pipelines, data catalog interaction, and monitoring. Survey respondents' top priorities over the next 12 months support this trend. In addition to migrating and consolidating data in the cloud, they intend to prioritize the following key capabilities: Enabling better data management for data science, AI, and ML Supporting development and deployment of data driven applications Supporting expansion in self service Business intelligence (BI) and analytics users Unifying management of data across distributed systems BI and analytics platforms remain one of the fastest growing software markets. The capabilities necessary to power these systems are in high demand: self-service analytics, faster discovery, predictions based on real-time operational data, and integration of rich and streaming data sets. The survey responses also showed that handling an increase in data volume and the number of concurrent users are modernization priorities. And there's pressure to reduce data latency and increase the frequency of updates. The survey showed that one of the most challenging capabilities organizations are dealing with is enabling low latency querying, search, and analytics. Giving users the right data at the right time to answer business questions, solve problems, and innovate with data is critical today and it depends on these capabilities. #2: Modernization outcomes The capabilities organizations seek only serve their modernization goals as far as they enable specific outcomes. And it's outcomes that are ultimately driving modernization initiatives. According to the survey, the number one outcome organizations seek to bring about is gaining fuller value from the data they store and capture. Forty-six percent of respondents cited it as their top challenge. Automating decision-making is another outcome organizations are seeking. Thirty-two percent of respondents rated automating decisions in operations in processes as very important. But it relies on the timely flow of insights into apps, one of the key capabilities identified earlier. Other key modernization outcomes cited in the survey include: Increase efficiency and effectiveness Generate new business strategies and models using analytics Make faster decisions Strengthen relationships via data sharing Improve trust and data quality Increase reuse and flexibility Reduce costs Provide authorized access to live data sets Consolidate data silos Developers in the survey said they were seeking to embed richer, personalized application experiences, with 52% saying they wanted seamless access to diverse data sets and sources. But first, they'll have to overcome several challenges that so far have proved difficult to solve. Sixty-eight percent of respondents said they face challenges processing streaming data and change data capture updates, and 64% struggle to integrate streaming with fast, high volume queries, and the same percentage said they struggle with combining historical and real-time analytics. #3: Modernization platform Modernized problems require modernized solutions. And the one most most commonly cited by respondents was a data platform , which they believe is the key to maximizing value from data. A data platform solves the issue of consolidating unnecessary data silos and ensuring access to data without the hassle of manual intervention or the risk of unauthorized access. Flexibility in the data platform is critical since data environments will continue to evolve, even after modernization milestones have been met. A data platform is one of the key elements that comprise modernized architecture. The TDWI survey cited several other advantages of unifying distributed data within a data platform: Simplifying and accelerating access Discovering data relationships easier and faster Creating a logical layer for single point of access Unifying data governance Reducing unnecessary data movement Modernized architecture Fifty-four percent of respondents said they were in the process of modernizing, and 29% were planning on doing so. The most frequently cited architectural feature by those modernizing or planning to was cloud migration from on-premises systems, with the goal being to change the dimensions of what was possible. But it wasn't just shifting to the cloud that respondents mentioned. The survey also indicated the prevalence of hybrid multi-cloud architectures as well, with data integration and management that span distributed data environments. Distributed architectures can lead to higher performance by putting data closest to where it's being used. It also solves data sovereignty issues by putting data where it's required to be due to regulatory jurisdiction. The report also mentions serverless architecture due to its pay-as-you-go computing model and improved business alignment. With serverless architecture , developers can build applications without thinking about infrastructure or traditional server management. Read the full TDWI report, Maximizing the Business Value of Data: Platforms, Integration, and Management .

November 28, 2022

10 years of MongoDB customers at AWS re:Invent

MongoDB has attended AWS re:Invent since its inception in 2012. A key reason for this is, of course, to help strengthen our partnership with AWS, which really began in 2015 and significantly expanded in March 2022 with a global, strategic collaboration agreement. But an even more fundamental reason for MongoDB's continued presence at AWS re:Invent over the years is the opportunity to engage with our many joint customers. Several MongoDB customers have been featured in re:Invent keynotes over the years. In fact, looking back at the customers AWS chose to feature in its keynotes, it's hard to find examples that are not MongoDB customers. Earlier this year, AWS celebrated 10 years of re:Invent by showcasing an equal number of "memorable customer moments" from the re:Invent mainstage. It was a great way to reaffirm AWS Leadership Principle #1 (Customer Obsession). It was also a great way to shine a light on the great things MongoDB's customers are doing. Rather than rewind on the many MongoDB customers spotlighted at re:Invent, let's look at those AWS called "most memorable" in its blog. All in on cloud Back in 2015, Capital One CIO Rob Alexander took to the re:Invent stage to discuss Capital One's "all in" approach to cloud. "We’re either using or experimenting with nearly every AWS service," Alexander said. What he didn't say, but which the company has been quite public about over the years, was how Capital One uses MongoDB in tandem with AWS services. A few months after Alexander's re:Invent comments, Capital One's Oron Gill Haus spoke at MongoDB World on Hygieia , the company's open source DevOps dashboard. Hygieia, built on MongoDB, provides the foundation for the company's attempts to reimagine banking. Haus detailed why MongoDB is so critical to Capital One's need to innovate quickly on customers' behalf, stressing how the variety and velocity of data makes MongoDB an ideal solution: We get data in from all different kinds of sources and formats, and we get it at different times. Now, what we have to do is predict the future and how you're planning on using the data. That's where traditional databases fall down. That's where you'll see MongoDB. We want to have the ability to find insights and be able to react quickly to those insights. Years later, Capital One advertises hundreds of jobs for those with MongoDB experience. (Hint: You may need to know how to roll back a MongoDB query for some of those jobs.) Capital One is doing impressive work with MongoDB, but it's not alone in its use of MongoDB for financial services. Goldman Sachs, Citi, Barclays, BBVA, Charles Schwab, FICO, HSBC, and Intuit are just a few MongoDB customers that have spoken publicly of how and why they use MongoDB. And, yes, some of these companies you may remember from the re:Invent main stage over the years. MongoDB to the Moon! Years before NASA Jet Propulsion Laboratory (JPL) took to the re:Invent stage (2016), the U.S. public agency was running MongoDB throughout NASA . By 2018, MongoDB was involved in the hugely interesting NASA Deep Space Network (DSN), a primary resource for communications and navigation for NASA's and partner agencies' interplanetary space missions. NASA had recently upgraded its decades-old infrastructure to base its modern Loading Analysis and Planning Software (LAPS) on Linux and MongoDB. LAPS, as a scientific paper details , "is responsible for long-term planning and forecasting, including studies and analysis of new missions, changed mission requirements, downtime, and new or changed antenna capabilities." Around the same time, and a key part of DSN operations, NASA was also looking for ways to improve the efficiency of operating antennas across the globe. The heart of this initiative was NASA's Link Complexity and Maintenance software (LCM), which stores all pertinent data in MongoDB. Hence, while it might not be accurate to say that MongoDB runs on the Moon, it would be true to say MongoDB helps NASA manage space missions to the Moon—and beyond. Can you hear me now? "[I]t’s just a massive moment for us at Verizon,” declared Hans Vestberg, chairman and CEO of Verizon, at re:Invent in 2019. He was talking about the company's partnership with AWS to deliver 5G network edge computing using AWS Wavelength. What wasn't said in the keynote, but that Robert Belson, Principal Engineer, Corporate Strategy, Verizon, explained , is that the vision was incomplete without MongoDB. “Verizon 5G Edge is a mobile edge computing platform, which embeds popular hyperscaler compute and storage, such as AWS Wavelength, at the edge of our 4G and 5G networks so application builders can extend existing workloads using the same popular services they know and love," he explained. “However, certain services, such as databases, are not natively supported, which is where [MongoDB] Atlas and Realm come into play by creating unprecedented flexibility for the developer and the end customer.” As we've described, Verizon decided that a comprehensive data platform was needed to make its 5G edge computing dream a reality. So Verizon integrated Atlas Functions with the Verizon edge discovery service to help direct 5G mobile clients to the topologically closest database instance across a customer’s edge deployment. In tandem, Verizon has overlaid a data persistence layer using MongoDB Realm, thereby enabling personalized experiences to extend to the network edge. Verizon is also using Atlas Device Sync and Realm to ensure the seamless synchronization of data between devices, the cloud and edge-of-network, online and offline. Customers love MongoDB + AWS Beginning to see a pattern here? While not every customer highlighted by AWS at re:Invent is a MongoDB customer, many are, including the few for which we've been able to provide some detail. Others include Epic Games, which runs its wildly popular game Fortnite on MongoDB ; or Volkswagen, which uses MongoDB throughout its web applications and in its Car Net service ; or Siemens, which runs MongoDB at the heart of its Monet system to provide monitoring, controlling, and remote management of field devices for advanced energy management services. This year while watching the various customers take the stage in re:Invent keynotes, keep in mind that they're also very likely a MongoDB customer, because customers that seek the agility and performance of AWS also tend to like how MongoDB's flexible data model enables them to do much more with their data. Interested in learning more? The best way might be to try fully managed MongoDB Atlas for free. You can get started now .

November 28, 2022

Modernize your GraphQL APIs with MongoDB Atlas and AWS AppSync

Modern applications typically need data from a variety of data sources, which are frequently backed by different databases and fronted by a multitude of REST APIs. Consolidating the data into a single coherent API presents a significant challenge for application developers. GraphQL emerged as a leading data query and manipulation language to simplify consolidating various APIs. GraphQL provides a complete and understandable description of the data in your API, giving clients the power to ask for exactly what they need — while making it easier to evolve APIs over time. It complements popular development stacks like MEAN and MERN , aggregating data from multiple origins into a single source that applications can then easily interact with. MongoDB Atlas: A modern developer data platform MongoDB Atlas is a modern developer data platform with a fully managed cloud database at its core. It provides rich features like native time series collections, geospatial data, multi-level indexing, search, isolated workloads, and many more — all built on top of the flexible MongoDB document data model. MongoDB Atlas App Services help developers build apps, integrate services, and connect to their data by reducing operational overhead through features such as hosted Data API and GraphQL API. The Atlas Data API allows developers to easily integrate Atlas data into their cloud apps and services over HTTPS with a flexible, REST-like API layer. The Atlas GraphQL API lets developers access Atlas data from any standard GraphQL client with an API that generates based on your data’s schema. AWS AppSync: Serverless GraphQL and pub/sub APIs AWS AppSync is an AWS managed service that allows developers to build GraphQL and Pub/Sub APIs. With AWS AppSync, developers can create APIs that access data from one or many sources and enable real-time interactions in their applications. The resulting APIs are serverless, automatically scale to meet the throughput and latency requirements of the most demanding applications, and charge only for requests to the API and by real-time messages delivered. Exposing your MongoDB Data over a scalable GraphQL API with AWS AppSync Together, AWS AppSync and MongoDB Atlas help developers create GraphQL APIs by integrating multiple REST APIs and data sources on AWS. This gives frontend developers a single GraphQL API data source to drive their applications. Compared to REST APIs, developers get flexibility in defining the structure of the data while reducing the payload size by bringing only the attributes that are required. Additionally, developers are able to take advantage of other AWS services such as Amazon Cognito, AWS Amplify, Amazon API Gateway, and AWS Lambda when building modern applications. This allows for a severless end-to-end architecture, which is backed by MongoDB Atlas serverless instances and available in pay-as-you-go mode from the AWS Marketplace . Paths to integration AWS AppSync uses data sources and resolvers to translate GraphQL requests and to retrieve data; for example, users can fetch MongoDB Atlas data using AppSync Direct Lambda Resolvers. Below, we explore two approaches to implementing Lambda Resolvers: using the Atlas Data API or connecting directly via MongoDB drivers . Using the Atlas Data API in a Direct Lambda Resolver With this approach, developers leverage the pre-created Atlas Data API when building a Direct Lambda Resolver. This ready-made API acts as a data source in the resolver, and supports popular authentication mechanisms based on API Keys, JWT, or email-password. This enables seamless integration with Amazon Cognito to manage customer identity and access. The Atlas Data API lets you read and write data in Atlas using standard HTTPS requests and comes with managed networking and connections, replacing your typical app server. Any runtime capable of making HTTPS calls is compatible with the API. Figure 1:   Architecture details of Direct Lambda Resolver with Data API Figure 1 shows how AWS AppSync leverages the AWS Lambda Direct Resolver to connect to the MongoDB Atlas Data API. The Atlas Data API then interacts with your Atlas Cluster to retrieve and store the data. MongoDB driver-based Direct Lambda Resolver With this option, the Lambda Resolver connects to MongoDB Atlas directly via drivers , which are available in multiple programming languages and provide idiomatic access to MongoDB. MongoDB drivers support a rich set of functionality and options , including the MongoDB Query Language, write and read concerns, and more. Figure 2:   Details the architecture of Direct Lambda Resolvers through native MongoDB drivers Figure 2 shows how the AWS AppSync endpoint leverages Lambda Resolvers to connect to MongoDB Atlas. The Lambda function uses a MongoDB driver to make a direct connection to the Atlas cluster, and to retrieve and store data. The table below summarizes the different resolver implementation approaches. Table 1:   Feature comparison of resolver implementations Setup Atlas Cluster Set up a free cluster in MongoDB Atlas. Configure the database for network security and access. Set up the Data API. Secrect Manager Create the AWS Secret Manager to securely store database credentials. Lambda Function Create Lambda functions with the MongoDB Data APIs or MongoDB drivers as shown in this Github tutorial . AWS AppSync setup Set up AWS Appsync to configure the data source and query. Test API Test the AWS AppSync APIs using the AWS Console or Postman . Figure 3:   Test results for the AWS AppSync query Conclusion To learn more, refer to the AppSync Atlas Integration GitHub repository for step-by-step instructions and sample code. This solution can be extended to AWS Amplify for building mobile applications. For further information, please contact partners@mongodb.com .

November 23, 2022