Building Modern Applications with Microservices: Part 2
In the
previous post
, I discussed the background behind microservices and their advantages. In this post, I will talk about how MongoDB enables microservices, as well as considerations to keep in mind before implementing a microservices project.
How MongoDB Enables Microservices
There are some fundamental technology principles that are required to ensure companies can reap the advantages of microservices, specifically around a flexible data model, redundancy, automation, and scalability.
Flexible Data Model:
MongoDB’s dynamic schema is ideal for handling the requirements of microservices and continuous delivery. When rolling out a new feature that changes the data model, there’s no requirement to update all of the existing records, something that can take weeks for a relational database. Developers can quickly iterate and model data against an ever changing environment, resulting in faster time to market and greater agility.
Redundancy:
Due to the distributed nature of microservices, there are more potential failure points, such as more network links, and thus, microservices need to be designed with redundancy in mind. MongoDB is well suited for this requirement, as it provides built-in redundancy through MongoDB
replica sets
. Replica sets not only provide greater resilience to failure, but also provide disaster recovery with multi-data center deployments and the ability to isolate operational workloads from analytical reporting in a single database cluster.
Monitoring and Automation:
With a small number of services, it is not difficult to manage tasks manually. As the number of services grow, productivity can stall if there is not an automated process in place to handle the growing complexity. Choosing technology that handles monitoring and automation is key to ensuring devops teams can remain productive, especially as the environment becomes more complex.
MongoDB Ops Manager
(also available as the hosted
Cloud Manager
service) features visualization, custom dashboards, and automated alerting to help manage a complex environment.
Ops Manager tracks 100+ key database and systems health metrics including operations counters, CPU utilization, replication status, and any node status. The metrics are securely reported to Ops Manager where they are processed and visualized.
Figure 1:
Ops Manager provides real time & historic visibility into the MongoDB deployment
IIntegration with existing monitoring tools is also straightforward via the Ops Manager RESTful API, and with packaged integrations to leading Application Performance Management (APM) platforms such as New Relic. This integration allows MongoDB status to be consolidated and monitored alongside the rest of your application infrastructure, all from a single pane of glass.
Scalability:
Scaling to meet extra demand is a requirement of any IT environment, and microservices are no exception. MongoDB provides a scalable solution that automatically partitions and distributes the database across nodes, which can easily serve IT infrastructures that require dynamic and high-performance capabilities. Additionally, MongoDB is ideally suited to scale-out on commodity hardware with auto-sharding, which, if needed, allows the service to be easily distributed across different geographic regions. This is better from the monolithic, scale up design of traditional RDBMS because scaling in MongoDB is automatic and transparent.
Manage Multiple Database Instances:
In a microservices architecture it is best practice to dedicate a separate database for each service. This leads to multiple database instances, which can be difficult to manage. At
MongoDB World 2016
, we announced
MongoDB Atlas
, which is hosted MongoDB as a Service. Developers don’t need to worry about provisioning, configuration, patching, upgrades, backups, and failure recovery of the database. MongoDB Atlas offers elastic scalability, either by scaling up on a range of instance sizes or scaling out with automatic sharding, all with no application downtime. Additionally, you can view, monitor, and manage all your MongoDB clusters from a single GUI, streamlining the management of your database clusters.
To capture more business benefit, many organizations are also shifting microservices to the cloud. The dynamic nature of the cloud allows enterprises to spin instances up and down, while providing continuous availability in case of any failures.
Considerations Before Moving to Microservices
Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project:
Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project:
Monitoring Challenges:
One of the biggest challenges for microservices is effectively monitoring the overall system. Monitoring one or two services is relatively straightforward, but effectively monitoring many services can be very challenging. Not only are there are more servers to monitor, but there are also more log files to analyze, as well as additional opportunities for network partitions. Traditional approaches to monitoring stats, such as CPU, memory, and network latencies are important, but enterprises also need to expand ways to view metrics about the system and how it behaves over a long period of time. Automating the monitoring process can help mitigate some of these challenges and reduce operational overhead.
High Developer Skillset:
Microservices is implemented on distributed systems, which are necessarily more complex. Network latency, hardware failures, unreliable networks, asynchronicity, and fault tolerance need to be dealt with gracefully and appropriately. In order to handle the added complexity, developers need to have a strong operations and production background. Developers can no longer create the application and hand it off to the operations team; they need to understand the interdependencies between DevOps, testing, and release in order to properly design a service. Before implementing a microservices architecture, it is important to determine if your team has the right capabilities to handle the associated complexities.
More Operations Overhead:
For a given monolithic application, it may require one application server cluster with a few processes, while a microservice application may comprise 50 services and 200 processes after adding in resiliency. Operating and monitoring all these new process can be a daunting task. Additionally, services need to be tested and quickly propagated through the continuous delivery pipeline, which requires proper tooling and skills.
Incorrect Service Boundaries:
It is imperative to establish the proper service boundaries during the design phase. A common problem is to create services from internal components without considering the proper service boundaries. As more functionality gets added, there is a risk that the team ends up building a giant distributed monolith. Getting the service boundaries incorrect may result in higher costs, overcoupled services, and more testing complexity.
MongoDB Microservice Deployments
MongoDB is deployed by thousands of organizations around the world, including over half of all Fortune 100 companies. Many enterprises use MongoDB in a microservices architecture to achieve their business and deployment goals.
Comparethemarket.com is a one of the UK’s leading providers for price comparison services and uses MongoDB as the operational database behind its large microservice environment. Service uptime is critical, and MongoDB’s distributed design is key to ensure that SLA’s are always met. Comparethemarket.com’s deployment consists of microservices deployed in AWS. Each microservice, or logical grouping of related microservices, is provisioned with its own MongoDB replica set running in
Docker containers
, and deployed across multiple AWS Availability Zones to provide resiliency and high availability. MongoDB
Ops Manager
is used to provide the operational automation that is essential to launch new features quickly: deploying replica sets, providing continuous backups, and performing zero downtime upgrades.
fuboTV
is a streaming service in North America that streams sports content from soccer leagues all over the world and uses MongoDB as the core database for its microservices architecture. The traffic profile of the fuboTV streaming service is extremely bursty with the site typically handling 100x normal traffic volumes ten minutes before a match. To keep pace with business growth and demanding software release schedule, fuboTV migrated its MongoDB database to Docker containers managed by the Kubernetes orchestration system on the Google Cloud Platform.
Figure 2:
fuboTV Microservices Architecture
This brings high levels of flexibility, efficiency, and uptime to fuboTV. Using containers managed by Kubernetes, fuboTV can provision all of its environments – development, test, QA and production – to a single cluster of physical hosts. Kubernetes scheduler is used to precisely control resource allocation across all of its apps, enabling fuboTV to maximize utilization and reduce costs. Kubernetes replication controller automatically reschedules containers if an instance fails — enabling fault resiliency and continuous availability. Data redundancy is provided by MongoDB replication within the replica set. This enables fuboTV to have zero downtime as it deploys and upgrades its applications
OTTO
is top German retailer for fashion and lifestyle goods that has two million daily site visitors. The problem was that OTTO had parallel teams spanning multiple business domains (business, project management, IT) that had various business problems but all needed to deliver results quickly. Independently, all the teams chose MongoDB as the best tool to quickly and easily achieve results. With loosely coupled teams, architecture, and operations, OTTO removed the bottleneck to deploy and test. Teams could quickly and iteratively correct errors and support continuous delivery. MongoDB was the driving force to enable OTTO’s business, IT, and project management teams to deliver fast results, drive development agility, and allow teams to innovate risk-free.
Summary
A microservices architecture provides many advantages over a monolithic architecture, but this does not imply microservices do not come without their own challenges. Proper planning and application decoupling is required to ensure that a microservices architecture will achieve your desired results. MongoDB is well suited for a microservices architecture with its ability to provide a flexible schema, redundancy, automation, and scalability. Together, MongoDB and microservices can help organizations align teams effectively, achieve faster innovation, and meet the challenges of a demanding new age in application development and delivery.
Learn more about MongoDB and microservices. Read the white paper.
Microservices: The Evolution of Building Modern Applications
About the Author - Jason Ma
Jason is a Principal Product Marketing Manager based in Palo Alto, and has extensive experience in technology hardware and software. He previously worked for SanDisk in Corporate Strategy doing M&A and investments, and as a Product Manager on the Infiniflash All-Flash JBOF. Before SanDisk, he worked as a HW engineer at Intel and Boeing. Jason has a BSEE from UC San Diego, MSEE from the University of Southern California, and an MBA from UC Berkeley.
June 7, 2016