MongoDB 3.4.0-rc2 is out and is ready for testing. This is the culmination of the 3.3.x development series.
Fixed in this release candidate:
- SERVER-7306 Mongod as windows service should not claim to be 'started' until it is ready to accept connections
- SERVER-18908 Secondaries unable to keep up with primary under WiredTiger
- SERVER-26420 Make internal clients identify themselves in the isMaster handshake
- SERVER-26514 Create command should take idIndex option
- SERVER-26648 Tolerate bad collection metadata produced on version 2.4 or earlier
- SERVER-26652 Invalid definitions in systemd configuration for debian
- WT-1592 Dump detailed cache information via statistics
- WT-2954 Inserting multi-megabyte values can cause large in-memory pages
As always, please let us know of any issues.
-- The MongoDB Team
Microservices Webinar Recap
Recently, we held a webinar discussing microservices, and how two companies, Hudl and UPS i-parcel, leverage MongoDB as the database powering their microservices environment. There have been a number of theoretical and vendor-led discussions about microservices over the past couple of years. We thought it would be of value to share with you real world insights from companies who have actually adopted microservices, as well as answers to questions we received from the audience during the live webinar. Jon Dukulil is the VP of Engineering from Hudl and Yursil Kidwai is the VP of Technology from UPS i-parcel. How are Microservices different from Service Oriented Architectures (SOAs) utilizing SOAP/REST with an Enterprise Service Bus (ESB)? Microservices and SOAs are related in that both approaches distribute applications into individual services. Where they differ though, is the scope of the problem they address today. SOAs aim for flexibility at the enterprise IT level. This can be a complex undertaking as SOAs only work when the underlying services do not need to be modified. Microservices represent an architecture for an individual service, and aim at facilitating continous delivery and parallel development of multiple services. The following graphic highlights some of the differences. One significant difference between SOAs and microservices revolves around the messaging system, which coordinates and synchronizes communication between different services in the application. Enterprise service buses (ESB) emerged as a solution for SOAs because of the need for service integration and a central point of coordination. As ESBs grew in popularity, enterprise vendors packaged more and more software and smarts into the middleware, making it difficult to decouple the different services that relied on the ESB for coordination. Microservices keep the messaging middleware focused on sharing data and events, and enabling more of the intelligence at the endpoints. This makes it easier to decouple and separate individual services. How big should a microservice be? There are many differing opinions about how large a microservice should be, thus it really depends on your application needs. Here is how Hudl and UPS i-parcel approach that question. Jon Dukulil (Hudl) : We determine how big our microservice should be the amount of work that can be completed by a squad. For us, a squad is a small completely autonomous team. It consists of 4 separate functions: product manager, developer, UI designer, and QA. When we are growing headcount we are not thinking of growing larger teams, we are thinking of adding more squads. !(https://webassets.mongodb.com/_com_assets/cms/Microservices_MongoDB_Blog2-a6l74owk23.png) Yursil Kidwai (UPS i-parcel) : For us, we have defined microservice as a single verb (e.g. Billing), and are constantly challenging ourselves on how that verb should be defined. We follow the “two pizza” rule, in which a team should never be larger than what you can feed with two large pizzas. Whatever our “two pizza” team can deliver in one week is what we consider to be the right size for a microservice. Why should I decouple databases in a microservices environment? Can you elaborate on this? One of the core principles behind microservices is strong cohesion (i.e. related code grouped together) and loose coupling (i.e. a change to one service should not require a change to another). With a shared database architecture both these principles are lost. Consumers are tied to a specific technology choice, as well as particular database implementation. Application logic may also be spread among multiple consumers. If a shared piece of information needs to be edited, you might need to change the behavior in multiple places, as well as deploy all those changes. Additionally, in a shared database architecture a catastrophic failure with the infrastructure has the potential to affect multiple microservices and result in a substantial outage. Thus, it is recommended to decouple any shared databases so that each microservice has its own database. Due to the distributed nature of microservices, there are more failure points. Because of all these movable parts in microservices, how do you deal with failures to ensure you meet your SLAs? Jon Dukulil (Hudl) : For us it’s an important point. By keeping services truly separate where they share as little as possible, that definitely helps. You’ll hear people working with microservices talk about “minimizing the blast radius” and that’s what I mean by the separation of services. When one service does have a failure it doesn’t take everything else down with it. Another thing is that when you are building out your microservices architecture, take care of the abstractions that you create. Things in a monolith that used to be a function call are now a network call, so there are many more things that can fail because of that: networks can timeout, network partitions, etc. Our developers are trained to think about what happens if we can’t complete the call. For us, it was also important to find a good circuit breaker framework and we actually wrote our own .NET version of a framework that Netflix built called Hystrix. That has been pretty helpful to isolate points of access between services and stop failures from cascading. Yursil Kidwai (UPS i-parcel) : One of the main approaches we took to deal with failures and dependencies was the choice to go with MongoDB. The advantage for us is MongoDB’s ability to deploy a single replica set across multiple regions. We make sure our deployment strategy always includes multiple regions to create that high availability infrastructure. Our goal is to always be up, and the ability of MongoDB’s replica sets to very quickly recover from failures is key to that. Another approach was around monitoring. We built our own monitoring framework that we are reporting on with Datadog. We have multiple 80 inch TVs displaying dashboards of the health of all our microservices. The dashboards are monitoring the throughput of the microservices on a continual basis, with alerts to our ops team configured if the throughput for a service falls below an acceptable threshold level. Finally, it’s important for the team to be accountable. Developers can’t just write code and not worry about, but they own the code from beginning to end. Thus, it is important for developers to understand the interdependencies between DevOps, testing, and release in order to properly design a service. Why did you choose MongoDB and how does it fit in with your architecture? Jon Dukulil (Hudl) : One, from a scaling perspective, we have been really happy with MongoDB’s scalability. We have many small databases and a couple of very large databases. Our smallest database today is serving up just 9MB of data. This is pretty trivial so we need these small databases to run on cost effective hardware. Our largest database is orders of magnitude larger and is spread over 8 shards. The hardware needs of those different databases are very different, but they are both running on MongoDB. Fast failovers are another big benefit for us. It’s fully automated and it’s really fast. Failovers are in the order of 1-5 seconds for us, and the more important thing is they are really reliable. We’ve never had an issue where a failover hasn’t gone well. Lastly, since MongoDB has a dynamic schema, for us that means that the code is the schema. If I’m working on a new feature and I have a property that last week was a string, but this week I want it to be an array of strings, I update my code and I’m ready to go. There isn’t much more to it than that. Yursil Kidwai (UPS i-parcel) : In many parts of the world, e-commerce rules governing cross border transaction are still changing and thus our business processes in those areas are constantly being refined. To handle the dynamic environment that our business operates in, the requirement to change the schema was paramount to us. For example, one country may require a tax identification number, while another country may suddenly decide it needs your passport, as well as some other classification number. As these changes are occurring, we really need something behind us that will adapt with us and MongoDB’s dynamic schema gave us the ability to quickly experiment and respond to our ever changing environment. We also needed the ability to scale. We have 20M tracking events across 100 vendors processed daily, as well as tens of thousands of new parcels that enter into our system every day. MongoDB’s ability to scale-out on commodity hardware and its elastic scaling features really allowed us to handle any unexpected inflows. Next Steps To understand more about the business level drivers and architectural requirements of microservices, read Microservices: Evolution of Building Modern Apps Whitepaper . For a technical deep dive into microservices and containers, read Microservices: Containers and Orchestration Whitepaper
How Thoughtful Illustration Is Setting MongoDB Apart: Meet Champa Lo
I sat down with Champa Lo, Technical Illustrator based in our New York headquarters, to learn more about her role as the first full-time illustrator at MongoDB. We talked about her passion for illustration, what she does, and how she’s shaping the future of design within the company. Ashley Perez: Welcome to the team! Can you tell me about your role? Champa Lo: Sure. I joined MongoDB right before COVID-19 hit. I came into the headquarters twice for an interview but ended up being one of the first new hires who had to start at home, on top of being the first person in a brand-new role. Technical Illustration is a first for MongoDB. The company has never had an illustrator on hand. Although we have talented designers who can illustrate within a design, that’s not their main focus: the overall design is. The difference with my role is that I work specifically on illustration. I also work to define the illustration style and help create a style guide. The most important aspect of my job is building good relationships with people throughout the company. I need to understand their goals and what they’re looking for so I can tell a purely visual story. AP: How did you get into illustration? CL: I guess you can say I fell into it (at least the illustration part). I always knew I wanted to be a graphic designer early on. I was a mentee for a graphic designer in high school and absolutely fell in love with the profession. I even have a cute clipping from my senior year high school paper where I talk about my dreams of being a designer. Interview excerpt from Champa's senior-year high school newspaper After high school, I studied graphic design at the University of Colorado Denver. When I was in the design program, I always found ways to incorporate fun illustrations in my projects. A year after I graduated, I moved to New York City because there were more jobs in design there and landed a job that allowed me to put my illustrating skills to good use. My first job was working with an incredible Creative Director at a small startup who built an amazing brand using illustrations to convey the company’s goals and messages. This was a part-time job: for four hours a day, I would concentrate on illustrating bespoke email banners for marketing prompts the team created that morning. With her guidance, I saw my illustration skills grow. It showed me the possibility of being a full-time illustrator. Here’s an example of a design I did while I was there: Email banner Champa created for ThinkEco during her first job as illustrator I love to illustrate (especially this type of illustration) because I’m a designer by trade, and the core of designing is to problem-solve. Illustration is no different. As a Technical Illustrator, I simplify and visualize complicated theories and concepts. Also, it’s fun! If I’m not having fun while illustrating, I’m very unmotivated. My creativity relies on avoiding boredom. I’m always working to improve my artistic skills. I’m a lover of learning, so I subscribe to tutorial sites such as Skillshare; follow artists on YouTube who share tutorials; and subscribe to a monthly art box that sends paints, brushes, pens, and so forth so I can try new mediums. Champa's illustration for a Google Local Guides social media post AP: How do you make your illustrations purposeful, engaging, and memorable? CL: Having thoughtful conversations about the subject matter is how you get good designs and illustration. If you don’t understand the subject to the best of your ability, how can you be successful at visualizing it? In school, I was taught to always research your subject matter and not design blindly. Putting in the extra work makes a huge difference. That’s also why 1:1 meetings are so important. It’s a time for me to learn, and it’s also a creative process for the stakeholders, because they find creative ways to help me understand. GIF Champa created for a MongoDB University Page We want to understand the goal. For example, should the illustration be futuristic or nostalgic? Recently, we had a conversation about cars and how we wanted to present them for a project. We decided to design the cars as compact or electric to show MongoDB as forward thinking and environmentally conscious, because those are the kinds of people we want to hire and work with. Or take COVID-19, for instance. The pandemic has changed the way people illustrate office environments. No longer do you have teams sitting in conference rooms. Instead, you have people working at home. So, I had to think of things to illustrate such as a sofa, home desk, and desk lamp. Maybe even a dog or a child. We thought about how we could incorporate this into the Zoom interface. Before, we didn’t have to think about it. Now, Zoom can be a way to add some personality to everyone’s digital space as we work remotely. That’s what I’m here for. To have those conversations and get deeper behind the meaning of everything we create. AP: Let’s talk a little more about your role at MongoDB. What projects do you work on? CL: I’m part of the Visual Design Team, which supports the whole company. It’s fun to meet and talk to many different people at MongoDB. It gives us a lot of diversity in the projects we work on. Along with illustrations, I also work on diagrams and small animations. Projects include campaigns, web illustrations, and events. Because I’ve joined the team, we’re able to have fuller discussions about illustration. Our designers work in a fast-paced world, but my process is slower because I make more bespoke illustrations and have to talk to people to understand the technicalities so we can go beyond generic illustrations. I have to be more thoughtful of what we’re presenting to the audience. Even though by having these conversations I slow down how quickly the designers move, I'm striving to build stronger relationships on the team through this practice. Top left: Champa’s illustration for MongoDB's new multi-cloud feature. Bottom right: An illustration for MongoDB's vendors page. I have found that by showing and explaining my illustration process and inviting them into it, people seem to trust me more. For example, I always share my sketches with stakeholders before digitizing the work. My sketches aren’t perfect, but by showing them not-so-perfect work, we can build the relationship and align on ideas. My hope is that the sketches allow people to see I’m open for collaboration and conversation. Example of a project working with MongoDB's Web Design team from initial sketch through final illustration AP: How does having these conversations help your design? CL: Great question! Working with such a diversity of people and projects helps me gain an immense amount of knowledge and insight. Past conversations and concerns help inform my design decisions. I’m almost like a liaison for all these different departments, and it's nice to transfer the information so we’re all aligned. For example, I’ve been working closely with Product Marketing on diagrams, and soon I’ll be working on diagrams with a member from the Docs team, too. Each team has taken its own paths for diagrams, but I would love to eventually create a holistic style that works for all teams beyond just these two. I believe having a good process to follow leads to meaningful and engaging illustrations. However, it’s important to find balance. You can’t overengineer it, because that can easily turn unproductive and formulaic. I always want an open dialogue and strive to show there’s room to collaborate. The process we have created has been successful so far, but it’s not set in stone. Further along we can add another step, or we may find certain things aren’t needed. AP: What’s your creative vision for MongoDB? CL: My goal for illustration is that we are inclusive, diverse, and thoughtful. What I’ve seen here is a global company full of people who are very passionate and kind. As designers, we have the power to show who and what MongoDB is. For me, that’s showing off who we are. One of our company’s values is “Own What You Do.” I think it’s such an important one for designers, because we should always add our personal experiences and perspectives to our work and translate the rest of the company’s perspectives and experiences, too. For the team, my goal is to continue streamlining a process so we’re transparent and support a collaborative spirit when it comes to working with us. Champa’s illustration for the MongoDB Atlas onboarding experience My goal is to create a unified vision between our two audiences: developer and enterprise customers. My hope is the illustrations bring joy and delight, and that our audiences see MongoDB has a personality. A really effective illustration system is memorable, and our research is starting to show that our audiences are beginning to remember our visuals. This is a huge brand lift, creating a personal experience versus the cold one people may experience with other tech brands. Interested in pursuing a career at MongoDB? We have several open roles on our teams across the globe , and would love for you to build your career with us!