How Bench Uncovered Hidden Value with MongoDB Atlas

Bench is an independent marketing technology company based in Sydney, Australia. Thanks to MongoDB Atlas, the team was able to improve developer productivity, discover hidden inefficiencies in their core application, and support future growth. We spoke with David Cameron, Engineering Director at Bench, to learn more about why they chose to migrate to Atlas and how it’s impacted the company.

The Bench Platform & MongoDB

The Bench Platform is a proprietary technology solution that brings together the entire digital ecosystem (every DSP, ad exchange, ad network, ad platform, DMP, SSP & publisher) into a single meta-platform, streamlining the multi-channel campaign lifecycle for marketers.

When Bench first launched the platform in 2013, it recognized that it was a strong use case for a document database. Marketing campaign data is naturally complex, with information from multiple sources and channels that each have their own unique attributes. “It was a natural fit,” says David. “MongoDB gives you a lot of flexibility. Once we knew the app wasn’t a fit for a relational database, MongoDB was the best choice.”

David also had previous experience with MongoDB and knew that it was great for more than handling complex data. “The absolute killer feature is sharding and replication. Scaling out is a big problem in the relational world. To be able to seamlessly shard your database across multiple servers is just great. You know you can effortlessly scale as you get more load and change sharding strategies in future. Not only is MongoDB a great platform to use for storing documents, but as we grow, it means we don’t need to buy bigger and bigger boxes.”

Migrating to a Platform-as-a-Service Model

When David joined Bench in early 2018, the team was hosting and maintaining a large number of VMs on Digital Ocean. He quickly set out to change that: “I wanted to move to a platform-as-a-service model to let the developers concentrate on writing code, rather than managing infrastructure.” He knew that in order to maximize the productivity of the team, he needed to make sure they spent as little time as possible patching infrastructure, ensuring uptime, and dealing with other issues. “We don’t have an Operations teams––we’re ’NoOps’, rather than DevOps,” notes David.

The team started the migration by moving all of their applications to AWS. Early in the planning one of the engineering team recommended Atlas as a fully managed version of MongoDB, so he decided to check it out. “Atlas was really attractive because I recognized that we’re not the experts in hosting databases,” recalls David. “We also liked it because since it is managed by MongoDB, it would get updates faster and manage the hosting better.”

Speed: A Cause for Concern

In evaluating Atlas, the team had one particular concern: speed. At the time, Bench was self-hosting MongoDB on the same VM as their application. David knew that Meteor, the framework used by the application, tended to be very latency sensitive with MongoDB. With Atlas, they would have to host the database outside of their VPC in AWS and use peering to connect to it. “We were worried that if we took it off the same machine, it would be significantly slower.”

The team put Atlas to the test by setting up a cluster in the AWS Sydney region and loading a backup of MongoDB data from the application. They ran performance testing, watching for network latency issues. “The performance was really good, which was surprising since it was hosted out of a different VPC,” remembers David. Once the team was satisfied with the performance, the migration itself was painless. “It was basically a backup and restore,” says David. “Once we proved the connectivity, it was all very smooth and straightforward.”

Unexpected Discoveries and Learnings

Soon after the team first went live with Atlas, they ran into an unexpected problem: they were running out of connections. “We hadn’t seen it on the dev side or staging environment,” notes David. “As we brought more users on board, we suddenly saw the number of connections just spike.” In fact, it took a while for the team to realize what was going on. After receiving the initial built-in alerts from Atlas that connections exceeded 70% of the total allotted, the team increased the threshold to 90% to stop getting more emails. When connections finally breached 100%, the application hit the wall and everyone scrambled to react.

As the team worked to find and resolve the underlying issue, David discovered that he could simply move their Atlas cluster to a higher tier to increase the connection limit so that it wasn’t timing out––all without any negative impact to the application. Eventually, they tracked the issue down to a bug in the platform and were able to fix it, then scale Atlas back down to the appropriate cluster size. “I knew there were tiers of MongoDB hosting in Atlas, but I didn’t know that we could just scale up a live production system and with no downtime,” says David. “It means we can compensate for a problem at the infrastructure level rather than trying to hurry out a bug fix.”

Atlas: A Window Into MongoDB

Later on, Bench ran into a different problem where the application was consuming an unusually large amount of database resources. This time, David knew that he could quickly scale up to a higher tier as the team diagnosed and resolved the issue, then back down when the load disappeared. He also credits the historical and real-time monitoring with making it easy to identify problems. “I didn’t expect the tools to be so good,” admits David. “The top three things I like most about Atlas are the alerts, being able to scale up and down seamlessly with no impact, and getting a really good window into what’s happening in MongoDB. In some cases, I didn’t know they existed until we started having problems.”

He advises other companies to put in the time to figure out which indicators represent their load, whether that’s CPU, memory, connections, or something else. David also recommends building a general purpose toolbox around alerting so that if a problem does occur, you can quickly identify the right course of action.

Realizing Economies of Scale

In addition to the Bench platform, the engineering team is responsible for around 18 different microservices, many of which use MongoDB and Atlas as a caching layer or as the system of record. “We’ve basically made a bet on MongoDB as our database of choice,” David states. “There are a lot of benefits in economies of scale and choosing a platform. As a team, using a technology, becoming an expert in it, and understanding it really well is important. We’ve been really, really happy with MongoDB.”

At the end of the day, one of the most important values of Atlas is that it keeps working and enables the team to be more productive. “It’s been great,” David says. “We’ve had hiccups in other areas of hosting infrastructure, but nothing in this space. Atlas has been rock solid. The documentation is very good, the capabilities are there. We find Atlas to be a very reliable part of our infrastructure.”

The Future of Bench

Today, the majority of Bench’s customers are based in Australia. As the company grows, it hopes to expand beyond its borders and reach customers worldwide. David is confident that scaling on Atlas won’t be a problem. In addition, Atlas’s numerous compliance certifications, such as PCI, are a huge benefit for an industry that’s still fairly new. “I see it as a strategic advantage for the future,” says David. “I’ve been trying to make sure that we have all the infrastructure and processes in place to support compliance needs because I suspect they’ll come in the future. It’s also so hard to retrofit later - it’s a lot easier to build it in from the beginning.”

Not only is David looking forward to growing with Atlas, he’s also genuinely interested in the continuous innovation of the core database: “I’m excited about MongoDB 4.2––in particular the distributed transaction and materialised views, we’re going to get some immediate benefits from those.” While the core application is currently running MongoDB 3.6, these new capabilities will be key drivers in making the jump to upgrade.