We’re excited to announce analytics node tiers for MongoDB Atlas. Analytics node tiers provide greater control and flexibility by allowing you to customize the exact infrastructure you need for your analytics workloads.
Analytics node tiers provide control and flexibility
Until now, analytics nodes in MongoDB’s Atlas clusters have used the same cluster tier as all other nodes. However, operational and analytical workloads can vary greatly in terms of resource requirements. Analytics node tiers allow you to enhance the performance of your analytics workloads by choosing the best tier size for your needs. This means you can choose an analytics node tier larger or smaller than the operational nodes in your cluster. This added level of customization ensures you achieve the performance required for both transactional and analytical queries — without the need to over- or under-provision your entire cluster for the sake of the analytical workload. Analytics node tiers are available in both Atlas and Atlas for Government.
Choose a higher or lower analytics node tier based on your analytics needs
Teams with large user bases using their BI dashboards may want to increase their analytics node tiers above that of their operational nodes. Choosing a higher tier can be useful when you have many users or require more memory to serve analytics needs. Scaling up the entire cluster tier would be costly, but scaling up just your analytics node tiers helps optimize the cost.
Teams with inconsistent needs may want to decrease their analytics node tier below that of their operational nodes. The ability to set a lower tier gives you flexibility and cost savings when you have fewer users or analytics are not your top priority.
With analytics node tiers, you get more discretion and control over how you manage your analytics workloads by choosing the appropriately sized tier for your analytics needs.
Video: Canva's Lessons From Scaling MongoDB Atlas to 10 Billion Documents Across 100 Nodes
Running complex, global, and mission-critical infrastructure at scale is difficult, and anyone who has done it for any length of time usually has a few gnarly lessons to share. At MongoDB World in June 2022, we were lucky enough to feature someone who had done just that. Michael Pearson , software engineering team lead at Canva , gave a talk titled “10 Billion Documents: How Canva Scaled MongoDB to 100 Nodes.” I’ve had the pleasure of working alongside Pearson and his team for almost a year now, and his presentation focused on some of the massive challenges (and lessons) they’ve faced over the last two years as they have scaled into tens of terabytes of data and tens of billions of documents. I’m writing this blog to give a few highlights, but I’d recommend everyone check the original talk in full: A tricky problem For the uninitiated, Canva is a visual communication platform that empowers its users to design anything and publish anywhere. Or, as Pearson explained in his talk, “Canva is a really simple way to create beautiful designs and presentations.” Canva’s mission is to empower the world to design, and more than 85 million people in over 190 countries use the platform every month. As you can imagine, this presents a huge data challenge — and opportunity. Canva holds more than 10 billion designs and receives up to 30,000 document requests per second. The success of the platform comes down to providing a fantastic user experience every time, and to do that they need to present their customers with the right data at the right time. “This could be a really tricky problem for a database platform, particularly for a business based in Sydney with many users on the other side of the world,” said Pearson. MongoDB Atlas supports the Document Service, which enables opening, creating, updating, or deleting any design on Canva. The Document Service is critical for every single user — if the Document Service is down, then Canva’s users can’t design. But before we get too far into things, we should probably start with why Canva started using MongoDB in the first place. Flexibility to support rapidly changing requirements Michael Pearson, software engineering team lead at Canva. “Canva was launched to the world back in 2013, when MongoDB was very new to the scene,” explains Pearson. “I'm not sure if there were any other databases that would have been up for the challenge.” From those earliest days, MongoDB's flexible document model was the perfect fit for Canva's increasingly complex designs and document types. “The flexibility that MongoDB gave us in those early days was instrumental to our success. As the Canva platform evolved, we were throwing new schema and new features at it. MongoDB would just handle it.” Its continued innovation and problem-solving means MongoDB remains as valuable to us today as it was in 2012. Michael Pearson, software engineering team lead at Canva At the same time, it was essential that Canva’s engineering team was focused on building Canva, rather than time spent managing the data platform. With that in mind, Canva chose to run MongoDB as a service. After trying out multiple options, they went with MongoLabs and, in 2019, following MongoDB's acquisition of MongoLabs, Canva migrated onto MongoDB Atlas , running on AWS, where they remain to this day. Ten years of relative bliss “Before 2021, we had a very hands-off approach to how we used MongoDB,” said Pearson. “MongoDB just handled it. We didn't have to think about it at all." That's incredible, right? Think about it — for nearly a decade the team barely had to think about their data layer and could spend their time working on new features and making the actual product better for its millions of users around the world. It's what every developer wants. Eventually, though, Canva’s own success created certain challenges around scaling. With the stratospheric increase in growth, the load on the Document Service also continued to increase. MongoDB’s ability to scale horizontally through the use of sharding was critical to overcoming initial scale challenges, something that traditional database management systems would have struggled to achieve, said Pearson. With MongoDB, sharding is distributed or partitioned across multiple machines — useful when no single machine can handle large workloads. In due course, though, some attributes of Canva’s workload presented a new challenge. Said Pearson: “We were unique in that we have one cluster with one collection with a million chunks. Our documents are fairly large, given our product has evolved over the years and we put more and more stuff into our documents.” Or, Canva does many updates to relatively large documents, and by mid-2021 the surge in traffic was causing issues. “Our peak traffic caused three main problems: inability to run the balancer, latency issues, and a disk usage pretty much at capacity,” Pearson explained. “A really ineffective cache caused a really high write load to our cluster. This was causing downstream failures." Pearson discussed some of the tactical solutions the company took. “Disabling the balancer immediately brought us back to service, but now we knew that there was something wrong with that cluster and we couldn’t operate without the balancer,” said Pearson. “We also noticed that the number of chunks in our cluster had skyrocketed, from around 400,000 to just over a million.” Getting to the root of the problem The professional services team at MongoDB discovered that “metadata updates were causing anywhere from a one-second to five-minute stalls in the cluster.” Going from 400,000 chunks to a million chunks, at the rate of a minute of each change, was not optimal. There were three things to address with that cluster: reduce the number of chunks, reduce that disk contention, and reduce the size of documents. “With regard to reducing the number of chunks, we just took any contiguous chunks on a shard and merged them unconditionally,” said Pearson. “This was tooling built in collaboration with MongoDB.” After three months of merging chunks, Canva saw massive improvements in its cluster’s performance. A failure rate during reboot of around 4% dwindled to less than 1% during maintenance operations. Further, to address latency spikes and full-disk capacity, the team formulated a six-step plan to move from network-based storage volumes to locally attached disks. This has proved a huge success. “We were able to run the balancer. Our large spikes in latency were pretty much all gone, and our disk usage was almost at zero,” Pearson said. He continued: "The key takeaway for me is that sharding is great, but it's never a silver bullet. I don't think we would have caught these issues so quickly without such a thorough incident review process and such a close working relationship with MongoDB." What was learned? After presenting all of that information, Pearson closed out the presentation with a few key lessons. For anyone interested in running infrastructure at a massive scale, they are simple and worth taking note of: Take advantage of the flexible document model to accelerate your pace of development. Ensure chunks are distributed uniformly across the cluster in a consistent size. Maintain a thorough incident review process and include your trusted partners (such as MongoDB). Reliability is an essential part of Canva’s engineering practice, and prolonged service disruptions were particularly upsetting not only for engineers but for Canva’s global users. Pearson is glad to report that Canva has seen a turnaround in the number of incidents impacting its Document Service. This has freed the document team to shift focus back to shipping features and ensuring every user has a flawless experience using Canva. Interested in joining Canva as it pursues its mission to empower the world to design? Canva is looking for a software engineer to join its Core Data team. Want to take advantage of the flexible document model to accelerate your pace of development? Learn more about MongoDB Atlas .
Honoring Latine Heritage Month at MongoDB
Heritage and culture sits at the centerfold of human interaction. With a population of more than 650 million people, speaking over 400 different languages, and spanning a geographic area from the tip of Patagonia to the Caribbean, the people of Latin America and the culture of their 33 countries are difficult to condense into one identity. In celebration of Latine Heritage Month, we asked a few Latine MongoDB employees to reflect on their heritage and ultimately how that shapes their work. Tayrin S Riojas , Head of Government Relations and Public Policy I was born in Los Angeles and moved to Mexico City before my third birthday. In my junior year of high school, my family moved back to the United States and ended up in Dallas. I feel so incredibly fortunate to have experienced living in both countries for extended periods of time. I remember high school in the United States feeling like I was in a Hollywood movie — there were big lockers, cheerleaders, and sports teams. However, I felt my friends in Mexico City had a wider variety of social activities compared to the friends I made in the United States. As Mexicans, and in many Latino cultures, we are passionate and socially driven with our families, extended families, and friendships. This is what I personally love most about my culture. We have great traditions and share in them together, from posadas, piñatas, soccer games, and even mourning. This is something that transcends our location, and I feel honored to have been raised with these values. Throughout my career, I have worked in telecommunications, film post-production, healthcare, and the government and held roles such as lobbyist, Senate Committee Consultant, and International Relations Advisor. Tech is at the core of every single one of these opportunities. I am certainly not an engineer, nor can I code anything functional, but I do have a passion for learning about technology. After having my second “COVID baby” and being on parental leave, I decided I wanted to get back into tech. A relative recommended MongoDB, and soon after, I started as a Cloud Account Executive for the Latin America market. I loved talking to our customers, and it taught me so much about the power and versatility of our tech. It was a great role, but I had spent so much time working with the government that I honestly missed it. I truly believe that to excel at what you do, you must have your heart in it. MongoDB is growing fast, and we are encouraged to build our own careers here. When I realized we had no Government Affairs department, I decided to propose it. I wrote a paper on why Government Affairs, why now, and the incredible value and ROI this could have for us (especially with our partnerships). I sent my proposal to leadership for their consideration. From ideation to leadership approving the department and role, I had amazing mentors, guidance, and support from other women at MongoDB and employee resource groups like Sell Like a Girl and The Underrepresented People of Color. I am now the Head of Government Relations and Public Policy at MongoDB. As a Latina woman, having a company of MongoDB’s size make room for your ideas and contributions has been an incredibly fulfilling journey. There is still much work to be done to build our Government Affairs department, but I am incredibly blessed to work for people I admire and contribute to the company through a role I am passionate about. If you are looking for a great career in tech, I urge you to consider MongoDB. Adriano Fratelli , Customer Success Manager My family’s history in Brazil began with my grandparents who migrated from Calabria, Italy to São Paulo in the mid-1960s. My grandfather had received a job opportunity in the largest and most modern port in Latin America, Santos. Growing up in São Paulo, my childhood was rich with Brazilian culture. I was surrounded by family, music, dancing, great food, festivals (like Brazilian Carnival ), and sports. My journey into technology began with my father. He worked for 40 years as a technology product manager in the retail industry and inspired me to pursue a career in tech. I finished my degree in Information Systems in 2014 and started my professional career at IBM as a Field Technical Sales Specialist. I then worked at Lenovo and Oracle before looking for a new career opportunity. My decision to start a new journey at MongoDB was due to the great perspective that customers have regarding our products and services, along with MongoDB’s inclusive culture. The world of technology has opened up many opportunities in my personal life by helping me improve my English language skills and giving me exposure to different countries and cultures around the world. MongoDB is growing exponentially in the Latin American region and, as part of the Customer Success team, I enjoy that I’m able to help our customers onboard and adopt MongoDB’s services. One thing that makes working at MongoDB stand out is knowing that employee’s differences are embraced and our ideas are heard. As part of a global team, it’s great to know that I have the space and support to share my ideas and am valued for the unique perspective I bring. Read more stories from Hispanic and Latine employees at MongoDB . We’re embracing differences every day at MongoDB. Join us to make an impact and transform your career.