What We're Reading

MongoDB
October 25, 2013 | Updated: July 5, 2016
#Company

Here are some great articles about MongoDB to read this weekend:

ScaleGrid: Should You Enable MongoDB Journaling?, 10/18

Business Insider: MongoDB co-founder Dwight Merriman and CEO Max Schireson were chosen for the Silicon Alley Top 100 in New York Tech roundup, October

InfoWorld: Use MongoDB to Make Your App Location-Aware, 10/24

comSysto: Getting Started with MongoSoup, 10/25

Digital Misinformation: Collections and Embedded Documents in MongoDB, 10/22

We’re happy to announce a new partnership with Soldfire. You can read more about this on Consumer Electronics and the MongoDB blog, 10/24

← Previous

Performance Tuning MongoDB on Solidfire

This is a guest post by by Chris Merz & Garrett Clark, SolidFire We recently had a large enterprise customer implement a MongoDB sharded cluster on SolidFire as the backend for a global e-commerce system. By leveraging solid-state drive technology with features like storage virtualization, Quality of Service (guaranteed IOPS per volume), and horizontal scaling, the customer was looking to combine the benefits of dedicated storage performance with the simplicity and scalability of a MongoDB environment. During the implementation the customer reached out to us with some performance and tuning questions, requesting our assistance with the configuration. After meeting with the team and reviewing the available statistics, we discovered response times that were deemed out of range for the application’s performance requirements. Response times were ~13-20ms (with an average of 15-17 ms). While this is considered acceptable latency in many implementations, the team was targeting < 5ms average query response times. When troubleshooting any storage latency performance issue it is important to focus on two critical aspects of the end-to-end chain: potential i/o queue depth bottlenecks and the main contributors to the overall latency in the chain. A typical end-to-end sequence with attached storage can be described by: MongoDB > OS > NIC > Network > Storage > Network > NIC > OS > MongoDB First off, we looked for any i/o queue depth bottlenecks and found the first one on the operating system layer. MongoDB was periodically sending an i/o queue depth of >100 to the operating system and, by default, iSCSI could only release a 32 queue depth per iSCSI session. This drop from an i/o queue depth of >100 to 32 caused frames to be stalled on the operating system layer while they were waiting to continue down the chain. We alleviated the issue by increasing the number of iSCSI sessions to the volume from 1 to 4, which proportionally increased the queue depth exiting the operating system to 128 (32*4). This enabled all frames coming off the application layer to immediately pass through the operating system and NIC, decreased the overall latency from ~15ms to ~4ms. Despite the latency average being 4ms, performance was still rather variable. We then turned our focus to pinpointing the sources of the remaining end-to-end latency. We were able to determine the latency factors in the stack through the study of three latency loops: First, the complete chain of: MongoDB > OS > NIC > Network > Storage > Network > NIC > OS > MongoDB . This loop took an average of 3.9ms to complete. Secondly, the subset loop of: OS > NIC > Network > Storage > Network > NIC > OS . This loop took ~1.1ms to complete. We determined the latency of this loop by the output of “iostat –xk 1” then greping for the corresponding volume. The last loop segment, latency on the storage subsystem, was 0.7ms and was obtained through a polling API command issued to the SolidFire unit. Our analysis pointed to the first layers of the stack contributing the most significant percent (>70%) of the end-to-end latency, so we decided to start there and continue downstream. We reviewed the OS configuration and tuning, with an eye towards both SolidFire/iSCSI best practices and MongoDB performance. Several OS-level tunables were found that could be tweaked to ensure optimal throughput for this type of deployment. Unfortunately, none of these resulted in any major reduction in the end-to-end latency for mongo. Having eliminated the obvious, we were left with what remained: MongoDB itself. A phrase oft-quoted by the famous fictional detective, Sherlock Holmes came to mind: “when you have eliminated the impossible, whatever remains, however improbable , must be the truth.” Upon going over the collected statistics runs with a fine-toothed comb, we noticed that the latency spikes had intervals of almost exactly 60 seconds. That’s when the light bulb went off… The MongoDB flush interval. The architecture of MongoDB was developed in the context of spinning disk, a vastly slower storage technology requiring batched file syncs to minimize query latency. The syncdelay setting defaults to 60 seconds for this very reason. In the documentation , it is clearly stated “In almost every situation you should not set this value and use the default setting”. ‘Almost’ was the key to our solution, in this particular case. It should be noted that changing syncdelay is an advanced tuning, and should be carefully evaluated and tested on a per-deployment basis. Little’s Law (IOPS = Queue Depth / Latency) indicated that lowering the flush interval would reduce the variance in queue depth thereby smoothing the overall latency. In lab testing, we had found that, under maximum load, decreasing the syncdelay to 1 second would force a ‘continuous flush’ behavior usually repeating every 6-7 seconds, reducing i/o spikes in the storage path. We had seen this as a useful technique for controlling IOPS throughput variability, but had not typically viewed it as a latency reduction technique. It worked! After implementing the change, the customer excitedly reported that they were seeing average end-to-end MongoDB response times of 1.2ms, with a throughput of ~4-5k IOPS per mongod (normal for this application), and NO obvious increase in extraneous i/o. By increasing the number of iSCSI sessions, normalizing the flush rate and removing the artificial 60s buffer, we reduced average latency more than an order of magnitude, proving out the architecture at scale in a global production environment. Increasing the iSCSI sessions increased parallelism, and decreased the latency by 3.5-4x. The reduction in syncdelay had the effect of smoothing the average queue depth being sent to the storage system, decreasing latency by slightly more than 3x. This customer’s experience is a good example of how engaging the MongoDB team early on can ensure a smooth product launch. As of today, we’re excited to announce that SolidFire is a MongoDB partner. Learn more about the SolidFire and MongoDB integration on our Database Solutions page . To learn more about performance tuning MongoDB on SolidFire, register for our upcoming webinar on November 6 with MongoDB. For more information on MongoDB performance, check out Performance Considerations for MongoDB , which covers other topics such as indexes and application patterns and offers insight into achieving MongoDB performance at scale.

October 24, 2013
Next →

Building a Culture of Growth: SVP Simon Eid on MongoDB's Massive Opportunity in APAC

Simon Eid is Senior Vice President Asia-Pacific (APAC) at MongoDB and leads the sales teams across Australia and New Zealand, India, ASEAN, and Japan. Simon's go-to-market organisation in APAC is growing rapidly and has nearly tripled in size in the past three years. They are hiring in all regions . In this article, Simon discusses MongoDB’s opportunity in APAC and how he builds a culture of growth and accountability. Simon Eid, SVP APAC, MongoDB (left) and Anoop Dhankar, RVP ANZ, MongoDB (right) MongoDB's opportunity in Asia-Pacific Out of the top 13 economies by GDP in the world , five of them are located in APAC: China, Japan, Australia, India, and South Korea. And that's to say nothing of the ASEAN countries which alone have more than 650 million inhabitants. Combine this with the worldwide database market, one of the largest markets in the software industry. IDC estimates that it will grow to $137B in 2027, and MongoDB has just reached $1B in ARR. This gives you a sense of the massive market opportunity we have globally. Regardless of industry, product, or service, almost every company is becoming a technology company, which means that every company is becoming a data company. We believe MongoDB is the Developer Data Platform that is best placed to support and accelerate that trend. We’ve already captured thousands of customers around the globe, but it’s important to keep in mind that our world is still in the early stages of shifting to the cloud and changing how applications are built and run. Compared to other software, what's special about the market we play in is that the database is not a “nice-to-have”; it’s mission-critical for organisations. As our world continues to undergo this digital transformation, we have the opportunity to transform how our customers use software and data to innovate, create, and disrupt industries. For example, look at Cathay Pacific , Hong Kong's home airline carrier operating in more than 60 destinations worldwide. The company's digital team turned to MongoDB on their journey to become one of the first airlines to create a truly paperless flight deck. Flight Folder, their application built on MongoDB, consolidates dozens of different information sources into one place. Since the Flight Folder launch, Cathay Pacific has completed more than 340,000 flights with full digital integration in the flight deck. Their innovation is enabled by MongoDB. Building a team across regions and cultures Our team in APAC is unique because of the different markets and cultures within the region. What this means is that we go to market differently in India than we do in Australia, in Singapore than we do in South Korea, and so on. Each market is completely different, but within all of them, there is a huge opportunity. Different from many of our peers, in APAC we've established business leaders who run regionalized teams in India, ASEAN, and ANZ with all functions reporting to them. These teams essentially operate as their own business and implement local best practices into their strategy. But, it doesn’t mean they’re operating in a silo. At the leadership level, there is an immense amount of collaboration and sharing of experiences to identify what’s working and what isn’t within each region. We also have a fantastic global sales organisation that rolls out extensive training and best practices to help enable our local teams to best help our customers and grow the business. Members of our APAC team at a recent offsite in Phuket Culture The most important thing is culture. We have a very high standard around everything we do and how we interact with each other. We don’t entertain politics. You can teach someone new skills and coach them on how to be successful in a new role, but if they’re not aligned with the culture, they will not be a fit. It’s a non-negotiable for me and why the most important aspect of the hiring process is the cultural aspect. If you get the culture right, everything else starts to fall into place. What I hear at MongoDB and from the teams I've built at other companies is that this is the kind of culture they can really thrive and grow. At MongoDB, our culture is defined and shaped by six core values . One of the values that’s most important to my team is “Embrace the Power of Differences”. Within APAC, there are a variety of cultural identities and nuances that can often be difficult to navigate, whether it is cultural values, beliefs, or go-to-market strategy. It’s important that everyone who joins my team is respectful of each other’s regional culture. What we’ve done within the APAC region, and with teams across the globe, is take everyone on a journey to understand and embrace these cultural differences. Our role as leaders is to develop our teams, from the bottom all the way up, which is part of MongoDB’s BDR to CRO career development initiative. We need to develop the next wave of leaders so that they’re prepared to step up when the time comes. For APAC, this means that regardless of where someone is from, each team member has been coached and developed on the cultural nuances so that they can lead people and go to market in each of the different regions. It’s also important that each team member contributes to a culture of psychological safety. Being part of a high-growth tech company requires taking risks and making mistakes. We have a high standard and we hold each other accountable, but it never comes at the cost of creating an environment where people are afraid to fail. When someone faces setbacks, I encourage them to share those experiences so that we can collectively learn. Through mutual support, we foster a stronger team capable of delivering exceptional results. The future of MongoDB in Asia-Pacific For any organisation to be successful, I believe it’s critically important for the entire ecosystem to act as one. As I mentioned earlier, at MongoDB the whole country ecosystem is aligned around one set of goals, so it's not a case of different teams running off in different directions. The teams are willing to lean in and do what's required to help each other build a great business. I can confidently say that in APAC, we are one team. This means sales, marketing, customer success, solutions consulting, and professional services all working together to focus on three things: making customers successful, building technical champions, and driving new workloads. As we continue to grow our team and MongoDB’s footprint in the region, these are the three things that will drive our success. As I mentioned earlier, there's a huge opportunity for MongoDB in APAC. Despite hiring slowing down or stopping completely at many other organisations, we're continuing to invest heavily in the region. To give you a sense of that - we've nearly tripled the size of our APAC go-to-market team in the past three years, and we've got more open roles across the different functions and regions. If you want to be part of this journey, there are three things I want to reiterate: First, we are extremely passionate about our culture, from the field level up to the leadership level. As a team, this is the brand we bring to the market. Second, the opportunity here is massive based on the total addressable market and our current share. And third, we place critical importance on development. By joining this team, I can promise that you’ll be provided with countless opportunities to develop your career and make an impact. I’m confident in my team and the leadership we have in place who are ready to take MongoDB APAC to the next level. Join us !

September 28, 2023