MongoDB 3.3.8 has been released. As a reminder, 3.3.8 is a development release and is not intended for production use. The 3.3 series will evolve into 3.4, which will be for production.
New/fixed in this release:
- SERVER-1393 Support decimal numbers
- SERVER-23043 Community and Enterprise builds on Ubuntu 16.04 LTS (Xenial Xerus)
- SERVER-23115 Include the prefixes of the indexed fields that cause index to be multikey in explain output
- SERVER-23644 Add additional tests for validate()
- SERVER-23697 Release shell as separate download
- SERVER-23725 Implement $graphLookup
- SERVER-23938 Include startup warning if running without access control
As always, please let us know of any issues.
-- The MongoDB Team
Building Modern Applications with Microservices: Part 2
In the previous post , I discussed the background behind microservices and their advantages. In this post, I will talk about how MongoDB enables microservices, as well as considerations to keep in mind before implementing a microservices project. How MongoDB Enables Microservices There are some fundamental technology principles that are required to ensure companies can reap the advantages of microservices, specifically around a flexible data model, redundancy, automation, and scalability. Flexible Data Model: MongoDB’s dynamic schema is ideal for handling the requirements of microservices and continuous delivery. When rolling out a new feature that changes the data model, there’s no requirement to update all of the existing records, something that can take weeks for a relational database. Developers can quickly iterate and model data against an ever changing environment, resulting in faster time to market and greater agility. Redundancy: Due to the distributed nature of microservices, there are more potential failure points, such as more network links, and thus, microservices need to be designed with redundancy in mind. MongoDB is well suited for this requirement, as it provides built-in redundancy through MongoDB replica sets . Replica sets not only provide greater resilience to failure, but also provide disaster recovery with multi-data center deployments and the ability to isolate operational workloads from analytical reporting in a single database cluster. Monitoring and Automation: With a small number of services, it is not difficult to manage tasks manually. As the number of services grow, productivity can stall if there is not an automated process in place to handle the growing complexity. Choosing technology that handles monitoring and automation is key to ensuring devops teams can remain productive, especially as the environment becomes more complex. MongoDB Ops Manager (also available as the hosted Cloud Manager service) features visualization, custom dashboards, and automated alerting to help manage a complex environment. Ops Manager tracks 100+ key database and systems health metrics including operations counters, CPU utilization, replication status, and any node status. The metrics are securely reported to Ops Manager where they are processed and visualized. Figure 1: Ops Manager provides real time & historic visibility into the MongoDB deployment IIntegration with existing monitoring tools is also straightforward via the Ops Manager RESTful API, and with packaged integrations to leading Application Performance Management (APM) platforms such as New Relic. This integration allows MongoDB status to be consolidated and monitored alongside the rest of your application infrastructure, all from a single pane of glass. Scalability: Scaling to meet extra demand is a requirement of any IT environment, and microservices are no exception. MongoDB provides a scalable solution that automatically partitions and distributes the database across nodes, which can easily serve IT infrastructures that require dynamic and high-performance capabilities. Additionally, MongoDB is ideally suited to scale-out on commodity hardware with auto-sharding, which, if needed, allows the service to be easily distributed across different geographic regions. This is better from the monolithic, scale up design of traditional RDBMS because scaling in MongoDB is automatic and transparent. Manage Multiple Database Instances: In a microservices architecture it is best practice to dedicate a separate database for each service. This leads to multiple database instances, which can be difficult to manage. At MongoDB World 2016 , we announced MongoDB Atlas , which is hosted MongoDB as a Service. Developers don’t need to worry about provisioning, configuration, patching, upgrades, backups, and failure recovery of the database. MongoDB Atlas offers elastic scalability, either by scaling up on a range of instance sizes or scaling out with automatic sharding, all with no application downtime. Additionally, you can view, monitor, and manage all your MongoDB clusters from a single GUI, streamlining the management of your database clusters. To capture more business benefit, many organizations are also shifting microservices to the cloud. The dynamic nature of the cloud allows enterprises to spin instances up and down, while providing continuous availability in case of any failures. Considerations Before Moving to Microservices Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project: Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project: Monitoring Challenges: One of the biggest challenges for microservices is effectively monitoring the overall system. Monitoring one or two services is relatively straightforward, but effectively monitoring many services can be very challenging. Not only are there are more servers to monitor, but there are also more log files to analyze, as well as additional opportunities for network partitions. Traditional approaches to monitoring stats, such as CPU, memory, and network latencies are important, but enterprises also need to expand ways to view metrics about the system and how it behaves over a long period of time. Automating the monitoring process can help mitigate some of these challenges and reduce operational overhead. High Developer Skillset: Microservices is implemented on distributed systems, which are necessarily more complex. Network latency, hardware failures, unreliable networks, asynchronicity, and fault tolerance need to be dealt with gracefully and appropriately. In order to handle the added complexity, developers need to have a strong operations and production background. Developers can no longer create the application and hand it off to the operations team; they need to understand the interdependencies between DevOps, testing, and release in order to properly design a service. Before implementing a microservices architecture, it is important to determine if your team has the right capabilities to handle the associated complexities. More Operations Overhead: For a given monolithic application, it may require one application server cluster with a few processes, while a microservice application may comprise 50 services and 200 processes after adding in resiliency. Operating and monitoring all these new process can be a daunting task. Additionally, services need to be tested and quickly propagated through the continuous delivery pipeline, which requires proper tooling and skills. Incorrect Service Boundaries: It is imperative to establish the proper service boundaries during the design phase. A common problem is to create services from internal components without considering the proper service boundaries. As more functionality gets added, there is a risk that the team ends up building a giant distributed monolith. Getting the service boundaries incorrect may result in higher costs, overcoupled services, and more testing complexity. MongoDB Microservice Deployments MongoDB is deployed by thousands of organizations around the world, including over half of all Fortune 100 companies. Many enterprises use MongoDB in a microservices architecture to achieve their business and deployment goals. Comparethemarket.com is a one of the UK’s leading providers for price comparison services and uses MongoDB as the operational database behind its large microservice environment. Service uptime is critical, and MongoDB’s distributed design is key to ensure that SLA’s are always met. Comparethemarket.com’s deployment consists of microservices deployed in AWS. Each microservice, or logical grouping of related microservices, is provisioned with its own MongoDB replica set running in Docker containers , and deployed across multiple AWS Availability Zones to provide resiliency and high availability. MongoDB Ops Manager is used to provide the operational automation that is essential to launch new features quickly: deploying replica sets, providing continuous backups, and performing zero downtime upgrades. fuboTV is a streaming service in North America that streams sports content from soccer leagues all over the world and uses MongoDB as the core database for its microservices architecture. The traffic profile of the fuboTV streaming service is extremely bursty with the site typically handling 100x normal traffic volumes ten minutes before a match. To keep pace with business growth and demanding software release schedule, fuboTV migrated its MongoDB database to Docker containers managed by the Kubernetes orchestration system on the Google Cloud Platform. Figure 2: fuboTV Microservices Architecture This brings high levels of flexibility, efficiency, and uptime to fuboTV. Using containers managed by Kubernetes, fuboTV can provision all of its environments – development, test, QA and production – to a single cluster of physical hosts. Kubernetes scheduler is used to precisely control resource allocation across all of its apps, enabling fuboTV to maximize utilization and reduce costs. Kubernetes replication controller automatically reschedules containers if an instance fails — enabling fault resiliency and continuous availability. Data redundancy is provided by MongoDB replication within the replica set. This enables fuboTV to have zero downtime as it deploys and upgrades its applications OTTO is top German retailer for fashion and lifestyle goods that has two million daily site visitors. The problem was that OTTO had parallel teams spanning multiple business domains (business, project management, IT) that had various business problems but all needed to deliver results quickly. Independently, all the teams chose MongoDB as the best tool to quickly and easily achieve results. With loosely coupled teams, architecture, and operations, OTTO removed the bottleneck to deploy and test. Teams could quickly and iteratively correct errors and support continuous delivery. MongoDB was the driving force to enable OTTO’s business, IT, and project management teams to deliver fast results, drive development agility, and allow teams to innovate risk-free. Summary A microservices architecture provides many advantages over a monolithic architecture, but this does not imply microservices do not come without their own challenges. Proper planning and application decoupling is required to ensure that a microservices architecture will achieve your desired results. MongoDB is well suited for a microservices architecture with its ability to provide a flexible schema, redundancy, automation, and scalability. Together, MongoDB and microservices can help organizations align teams effectively, achieve faster innovation, and meet the challenges of a demanding new age in application development and delivery. Learn more about MongoDB and microservices. Read the white paper. Microservices: The Evolution of Building Modern Applications About the Author - Jason Ma Jason is a Principal Product Marketing Manager based in Palo Alto, and has extensive experience in technology hardware and software. He previously worked for SanDisk in Corporate Strategy doing M&A and investments, and as a Product Manager on the Infiniflash All-Flash JBOF. Before SanDisk, he worked as a HW engineer at Intel and Boeing. Jason has a BSEE from UC San Diego, MSEE from the University of Southern California, and an MBA from UC Berkeley.
Choosing the Right Tool for the Job: Understanding the Analytics Spectrum
Data-driven organizations share a common desire to get more value out of the data they're generating. To maximize that value, many of them are asking the same or similar questions: How long does it take to get analytics and insights from our application data? What would be the business impact if we could make that process faster? What new experiences could we create by having analytics integrated directly within our customer-facing apps? How do our developers access the tools and APIs they need to build sophisticated analytics queries directly into their application code? How do we make sense of voluminous streams of time-series data? We believe the answer to these questions in today's digital economy is application-driven analytics. What is Application-Driven Analytics? Traditionally, there's been a separation at organizations between analytics that run the business and analytics that manage the business. They're built by different teams, they serve different audiences, and the data itself is replicated and stored in different systems. There are benefits to the traditional way of doing things and it's not going away. However, in today's digital economy, where the need to create competitive advantage and reduce costs and risk are paramount, organizations will continue to innovate upon the traditional model. Today, those needs manifest themselves in the demand for smarter applications that drive better customer experiences and surface insights to initiate intelligent actions automatically. This all happens within the flow of the application on live, operational data in real time. Alongside those applications, the business also wants faster insights so it can see what's happening, when it's happening. This is known as business visibility, and the goal of it is to increase efficiency by enabling faster decisions on fresher data. In-app analytics and real-time visibility are enabled by what we call application-driven analytics. Find out why the MongoDB Atlas developer data platform was recently named a Leader in Forrester Wave: Translytical Data Platforms, Q4 2022 You can find examples of application-driven analytics in multiple real-world industry use cases including: Hyper-personalization in retail Fraud prevention in financial services Preventative maintenance in manufacturing Single subscriber view in telecommunications Fitness tracking in healthcare A/B testing in gaming Where Application-Driven Analytics fits in the Analytics Ecosystem Application-driven analytics complements existing analytics processes where data is moved out of operational systems into centralized data warehouses and data lakes. In no way does it replace them. However, a broader spectrum of capabilities are now required to meet more demanding business requirements. Contrasting the two approaches, application-driven analytics is designed to continuously query data in your operational systems. The freshest data comes in from the application serving many concurrent users at very low latency. It involves working on much smaller subsets of data compared to centralized analytics systems. Application-driven analytics is typically working with hundreds to possibly a few thousand records at a time. And it's running less complex queries against that data. At the other end of the spectrum is centralized analytics. These systems are running much more complex queries across massive data sets — hundreds of thousands or maybe millions of records, and maybe at petabyte scale — that have been ingested from many different operational data sources across the organization. Table 1 below identifies the required capabilities across the spectrum of different classes of analytics. These are designed to help MongoDB’s customers match appropriate technologies and skill sets to each business use case they are building for. By mapping required capabilities to use cases, you can see how these different classes of analytics serve different purposes. If, for example, we're dealing with recommendations in an e-commerce platform, the centralized data warehouse or data lake will regularly analyze vast troves of first- and third-party customer data. This analysis is then blended with available inventory to create a set of potential customer offers. These offers are then loaded back into operational systems where application-driven analytics is used to decide which offers are most relevant to the customer based on a set of real-time criteria, such as actual stock availability and which items a shopper might already have in their basket. This real-time decision-making is important because you wouldn't want to serve an offer on a product that can no longer be fulfilled or on an item a customer has already decided to buy. This example demonstrates why it is essential to choose the right tool for the job. Specifically, in order to build a portfolio of potential offers, the centralized data warehouse or data lake is an ideal fit. Such technologies can process hundreds of TBs of customer records and order data in a single query. The same technologies, however, are completely inappropriate when it comes to serving those offers to customers in real time. Centralized analytics systems are not designed to serve thousands of concurrent user sessions. Nor can they access real-time inventory or basket data in order to make low latency decisions in milliseconds. Instead, for these scenarios, application-driven analytics served from an operational system is the right technology fit. As we can see, application-driven analytics is complementary to traditional centralized analytics, and in no way competitive to it. The benefits to organizations of using these complementary classes of analytics include: Maximizing competitive advantage through smarter and more intelligent applications Out-innovating and differentiating in the market Improving customer experience and loyalty Reducing cost by improving business visibility and efficiency Through its design, MongoDB Atlas unifies the essential data services needed to deliver on application-driven analytics. It gives developers the tools, tech, and skills they need to infuse analytics into their apps. At the same time, Atlas provides business analysts, data scientists, and data engineers direct access to live data using their regular tools without impacting the app. For more information about how to implement app-driven analytics and how the MongoDB developer data platform gives you the tools needed to succeed, download our white paper, Application-Driven Analytics: Defining the Next Wave of Modern Apps .