MongoDB 3.3.8 has been released. As a reminder, 3.3.8 is a development release and is not intended for production use. The 3.3 series will evolve into 3.4, which will be for production.
New/fixed in this release:
- SERVER-1393 Support decimal numbers
- SERVER-23043 Community and Enterprise builds on Ubuntu 16.04 LTS (Xenial Xerus)
- SERVER-23115 Include the prefixes of the indexed fields that cause index to be multikey in explain output
- SERVER-23644 Add additional tests for validate()
- SERVER-23697 Release shell as separate download
- SERVER-23725 Implement $graphLookup
- SERVER-23938 Include startup warning if running without access control
As always, please let us know of any issues.
-- The MongoDB Team
Building Modern Applications with Microservices: Part 2
In the previous post , I discussed the background behind microservices and their advantages. In this post, I will talk about how MongoDB enables microservices, as well as considerations to keep in mind before implementing a microservices project. How MongoDB Enables Microservices There are some fundamental technology principles that are required to ensure companies can reap the advantages of microservices, specifically around a flexible data model, redundancy, automation, and scalability. Flexible Data Model: MongoDB’s dynamic schema is ideal for handling the requirements of microservices and continuous delivery. When rolling out a new feature that changes the data model, there’s no requirement to update all of the existing records, something that can take weeks for a relational database. Developers can quickly iterate and model data against an ever changing environment, resulting in faster time to market and greater agility. Redundancy: Due to the distributed nature of microservices, there are more potential failure points, such as more network links, and thus, microservices need to be designed with redundancy in mind. MongoDB is well suited for this requirement, as it provides built-in redundancy through MongoDB replica sets . Replica sets not only provide greater resilience to failure, but also provide disaster recovery with multi-data center deployments and the ability to isolate operational workloads from analytical reporting in a single database cluster. Monitoring and Automation: With a small number of services, it is not difficult to manage tasks manually. As the number of services grow, productivity can stall if there is not an automated process in place to handle the growing complexity. Choosing technology that handles monitoring and automation is key to ensuring devops teams can remain productive, especially as the environment becomes more complex. MongoDB Ops Manager (also available as the hosted Cloud Manager service) features visualization, custom dashboards, and automated alerting to help manage a complex environment. Ops Manager tracks 100+ key database and systems health metrics including operations counters, CPU utilization, replication status, and any node status. The metrics are securely reported to Ops Manager where they are processed and visualized. Figure 1: Ops Manager provides real time & historic visibility into the MongoDB deployment IIntegration with existing monitoring tools is also straightforward via the Ops Manager RESTful API, and with packaged integrations to leading Application Performance Management (APM) platforms such as New Relic. This integration allows MongoDB status to be consolidated and monitored alongside the rest of your application infrastructure, all from a single pane of glass. Scalability: Scaling to meet extra demand is a requirement of any IT environment, and microservices are no exception. MongoDB provides a scalable solution that automatically partitions and distributes the database across nodes, which can easily serve IT infrastructures that require dynamic and high-performance capabilities. Additionally, MongoDB is ideally suited to scale-out on commodity hardware with auto-sharding, which, if needed, allows the service to be easily distributed across different geographic regions. This is better from the monolithic, scale up design of traditional RDBMS because scaling in MongoDB is automatic and transparent. Manage Multiple Database Instances: In a microservices architecture it is best practice to dedicate a separate database for each service. This leads to multiple database instances, which can be difficult to manage. At MongoDB World 2016 , we announced MongoDB Atlas , which is hosted MongoDB as a Service. Developers don’t need to worry about provisioning, configuration, patching, upgrades, backups, and failure recovery of the database. MongoDB Atlas offers elastic scalability, either by scaling up on a range of instance sizes or scaling out with automatic sharding, all with no application downtime. Additionally, you can view, monitor, and manage all your MongoDB clusters from a single GUI, streamlining the management of your database clusters. To capture more business benefit, many organizations are also shifting microservices to the cloud. The dynamic nature of the cloud allows enterprises to spin instances up and down, while providing continuous availability in case of any failures. Considerations Before Moving to Microservices Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project: Though microservices offer many advantages, they are not appropriate for all deployments. There are several considerations to keep in mind before implementing a microservices project: Monitoring Challenges: One of the biggest challenges for microservices is effectively monitoring the overall system. Monitoring one or two services is relatively straightforward, but effectively monitoring many services can be very challenging. Not only are there are more servers to monitor, but there are also more log files to analyze, as well as additional opportunities for network partitions. Traditional approaches to monitoring stats, such as CPU, memory, and network latencies are important, but enterprises also need to expand ways to view metrics about the system and how it behaves over a long period of time. Automating the monitoring process can help mitigate some of these challenges and reduce operational overhead. High Developer Skillset: Microservices is implemented on distributed systems, which are necessarily more complex. Network latency, hardware failures, unreliable networks, asynchronicity, and fault tolerance need to be dealt with gracefully and appropriately. In order to handle the added complexity, developers need to have a strong operations and production background. Developers can no longer create the application and hand it off to the operations team; they need to understand the interdependencies between DevOps, testing, and release in order to properly design a service. Before implementing a microservices architecture, it is important to determine if your team has the right capabilities to handle the associated complexities. More Operations Overhead: For a given monolithic application, it may require one application server cluster with a few processes, while a microservice application may comprise 50 services and 200 processes after adding in resiliency. Operating and monitoring all these new process can be a daunting task. Additionally, services need to be tested and quickly propagated through the continuous delivery pipeline, which requires proper tooling and skills. Incorrect Service Boundaries: It is imperative to establish the proper service boundaries during the design phase. A common problem is to create services from internal components without considering the proper service boundaries. As more functionality gets added, there is a risk that the team ends up building a giant distributed monolith. Getting the service boundaries incorrect may result in higher costs, overcoupled services, and more testing complexity. MongoDB Microservice Deployments MongoDB is deployed by thousands of organizations around the world, including over half of all Fortune 100 companies. Many enterprises use MongoDB in a microservices architecture to achieve their business and deployment goals. Comparethemarket.com is a one of the UK’s leading providers for price comparison services and uses MongoDB as the operational database behind its large microservice environment. Service uptime is critical, and MongoDB’s distributed design is key to ensure that SLA’s are always met. Comparethemarket.com’s deployment consists of microservices deployed in AWS. Each microservice, or logical grouping of related microservices, is provisioned with its own MongoDB replica set running in Docker containers , and deployed across multiple AWS Availability Zones to provide resiliency and high availability. MongoDB Ops Manager is used to provide the operational automation that is essential to launch new features quickly: deploying replica sets, providing continuous backups, and performing zero downtime upgrades. fuboTV is a streaming service in North America that streams sports content from soccer leagues all over the world and uses MongoDB as the core database for its microservices architecture. The traffic profile of the fuboTV streaming service is extremely bursty with the site typically handling 100x normal traffic volumes ten minutes before a match. To keep pace with business growth and demanding software release schedule, fuboTV migrated its MongoDB database to Docker containers managed by the Kubernetes orchestration system on the Google Cloud Platform. Figure 2: fuboTV Microservices Architecture This brings high levels of flexibility, efficiency, and uptime to fuboTV. Using containers managed by Kubernetes, fuboTV can provision all of its environments – development, test, QA and production – to a single cluster of physical hosts. Kubernetes scheduler is used to precisely control resource allocation across all of its apps, enabling fuboTV to maximize utilization and reduce costs. Kubernetes replication controller automatically reschedules containers if an instance fails — enabling fault resiliency and continuous availability. Data redundancy is provided by MongoDB replication within the replica set. This enables fuboTV to have zero downtime as it deploys and upgrades its applications OTTO is top German retailer for fashion and lifestyle goods that has two million daily site visitors. The problem was that OTTO had parallel teams spanning multiple business domains (business, project management, IT) that had various business problems but all needed to deliver results quickly. Independently, all the teams chose MongoDB as the best tool to quickly and easily achieve results. With loosely coupled teams, architecture, and operations, OTTO removed the bottleneck to deploy and test. Teams could quickly and iteratively correct errors and support continuous delivery. MongoDB was the driving force to enable OTTO’s business, IT, and project management teams to deliver fast results, drive development agility, and allow teams to innovate risk-free. Summary A microservices architecture provides many advantages over a monolithic architecture, but this does not imply microservices do not come without their own challenges. Proper planning and application decoupling is required to ensure that a microservices architecture will achieve your desired results. MongoDB is well suited for a microservices architecture with its ability to provide a flexible schema, redundancy, automation, and scalability. Together, MongoDB and microservices can help organizations align teams effectively, achieve faster innovation, and meet the challenges of a demanding new age in application development and delivery. Learn more about MongoDB and microservices. Read the white paper. Microservices: The Evolution of Building Modern Applications About the Author - Jason Ma Jason is a Principal Product Marketing Manager based in Palo Alto, and has extensive experience in technology hardware and software. He previously worked for SanDisk in Corporate Strategy doing M&A and investments, and as a Product Manager on the Infiniflash All-Flash JBOF. Before SanDisk, he worked as a HW engineer at Intel and Boeing. Jason has a BSEE from UC San Diego, MSEE from the University of Southern California, and an MBA from UC Berkeley.
Solving Customer Challenges: Meet Consulting Engineer Paul-Emile Brotons
Our Professional Services team is growing. Hear from Paul-Emile Brotons about his Consulting Engineer (CE) role, the types of projects he works on for customers, how he continually learns, and what makes this role a great opportunity for people with technical backgrounds who enjoy solving a variety of problems. Jackie Denner: Thanks for sharing your experience as a Consulting Engineer. Can you tell me about the Consulting Engineer team within Professional Services at MongoDB? Paul-Emile Brotons: I joined MongoDB a year and a half ago. The Consulting Engineering team is responsible for assisting customers at every stage of their MongoDB journey to ensure they are successful. We assist customers with training, database design, architecture design, code reviews, preproduction audits and reviews, setup, and health checks. I’m part of the South European team and I’m based out of Paris, but the Consulting Engineering team is worldwide. Since we are solving challenging problems, the team is very close and meets daily to share ideas and discuss solutions. I always have colleagues available to help at any time of day. JD: As a junior engineer, why did you opt for a Consulting Engineer role instead of a traditional Product Engineer role? PEB: Before joining MongoDB, I was a full-stack engineer at a French startup specializing in revenue management. I learned great technical skills there, but, in the end, I felt I was missing the big picture: What other stacks exist on the market? What tools are other engineering teams at big companies or startups working with? That is exactly what the Consulting Engineer role made possible for me. Since our projects are usually short-term, a typical CE may see 50 projects in a year. In my current role, I have been working with almost every new and exciting technology. I also get to learn how people within product and engineering work in other organizations. I find this very valuable, and it’s not something you can easily find in a traditional Product Engineer role. JD: What does a day in your role look like? PEB: CEs are assigned to “missions,” which typically range from one to four days and concern a specific customer. Longer-term projects can span several months. My role generally starts the week before. Before each mission, I try to set up a short preconsult session where I meet with customers and discover the topics they want to discuss. Then, on the day of the mission, I provide training, performance evaluation, tuning, and more. I learn a lot in my role, and I try to find solutions to all the difficult problems the customer has not been able to solve alone. It’s challenging and very rewarding. In some cases, I may not be assigned to a customer and I will be working on preparation and continuous learning. I appreciate the liberty my role gives me. JD: What was your onboarding like, and what learning and growth opportunities are there on the Consulting Engineer team? PEB: To be completely honest, I was a bit scared when I joined. I was very impressed with the way people work here, and I had a feeling it would be hard for me to onboard. However, the ramp-up process is so well-done that it almost felt easy. The first weeks were dedicated only to training. First, we have to learn a lot about MongoDB. A CE is a database expert. Since almost every software needs a persistent layer, this expertise is very valuable. Second, we have to know our stuff when it comes to Linux, networking, cloud providers, architecture, coding, and more. Afterward, everything is done to gradually increase the level of difficulty; complex missions are not delivered by new hires. Management is really careful about that, which is reassuring. Once a CE is performing well in their role, they may be promoted to Senior and then Principal grades. Many of us also study to pass certifications. I will soon start studying for a Linux sysadmin certification. The management team is very supportive and encourages continuous learning. JD: How do you interact with other teams at MongoDB? PEB: The CE role requires a lot of interaction with teams such as Sales, Presales Engineering, and Product Engineering. Consulting Engineers can be leveraged to help Sales and Solution Architects before the sale happens, since we are seen as trusted advisers. We also often speak to product teams to discuss the inner workings of a product, feature, or system. I’ve had the opportunity to meet many people within MongoDB. JD: What is one of the most interesting or challenging projects you’ve worked on? PEB: It is honestly difficult to choose, but I would pick a long project I worked on with a major container transportation and shipping company. It was challenging given the scope of the project and the number of interactions and subjects I had to deal with. The project was key for the customer, and it was technically demanding. We had to review the whole application architecture; analyze the front end to infer the requests and schema design needed on the database side; work with a wide range of professionals, including developers, solution architects, Linux engineers, and project managers; and test that everything would happen as expected. It was a great learning experience, from both a personal and professional perspective. JD: What makes someone successful in a CE role? PEB: Aside from sufficient knowledge of computer science, the CE role requires good communication and problem-solving skills. You have to know how to listen to and understand the problems customers encounter before you can think of a solution. Good customer contact is often the key to a mission’s success, and it makes the difference between a satisfied customer and a happy customer. JD: What advice would you offer someone looking to move into Professional Services at MongoDB? PEB: First, prepare well for the interviews — study up on algorithms, two programming languages, and basic database and hardware concepts. The interviews can be challenging, and there are a lot of rounds. Second, I would advise candidates to look at the beginners course on the MongoDB University website. The courses are free and they’re the best I have done on the web so far. Going deeper into learning MongoDB before joining the company saved me a lot of time. Last but not least, I would encourage candidates to contact CEs at MongoDB to get a clear view of the company and the role. My colleagues and I are more than happy to answer any questions that might help someone decide if this role is the right fit for them. Interested in a Professional Services career at MongoDB? We have several open roles on our team and would love for you to transform your career with us!