Every day at noon EST, Gilt launches new, limited-time sales of the most sought-after products and most desirable brands in fashion, home décor and leisure. Launched in December 2007, the company is the leader in the flash-sale space and has attracted millions of loyalists by offering them special access to coveted merchandise and experiences at insider prices. Consistently high performance, even in the face of extremely spiky server loads, is paramount.
Initially built as a monolithic application on Postgres, Gilt moved to a service-oriented architecture in 2010 to enable rapid development of new features while maintaining a stable codebase for core services. Because it's schemaless, MongoDB makes it easy for Gilt developers to build and quickly improve new services, and is well suited for high performance applications.
In the early stages of the company, with only a handful of developers and a “contained” model, deep end-to-end changes could be made quickly on a monolithic Postgres architecture. As Gilt matured and the business models and processes became more stable and well understood, the monolithic approach became expensive to scale and limited developers’ ability to add new features that fit the different business silos (e.g. retail vs. travel vs. food and wine.)
“Changes start cheap, but end up expensive because we have to do end-to-end changes over and over again,” said Eric Bowman, Gilt’s VP of Architecture. A service-oriented architecture leveraging a NoSQL database would help Gilt to scale horizontally in a cost-effective way.
Gilt is using MongoDB to transition user profile data out of Postgres. Gilt’s customer-facing applications now retrieve data from MongoDB, which offers a flexible, efficient data store from which multiple systems can access user data for various business processes.
Schemaless design: Innovation without risk
Gilt divides its architecture into a stable core and a less stable satellite, where innovation is encouraged. Instead of resource-intensive schema migrations, MongoDB makes it easy for developers to add new fields for specific users, for example, without having to change the entire database. This schema flexibility promotes fast, frequent innovation, where developers are able to isolate applications without modifying the core architecture.
For example, managing Gilt’s email is a massive engineering task. They send millions of personalized emails every day, generated by cutting-edge algorithms. On top of that, emails have to be dynamically re-queued – e.g., password change emails need to be delivered immediately. In two months, two developers built a reliable, fast and clean system on MongoDB, which they could bring up in isolation before deploying. “It was easy for them to build something really powerful and useful to customers,” said Bowman.
Another example is Gilt’s user preference service, which allows customers to see their favorite brands when they log in. MongoDB enables Gilt to tailor the homepage layout for millions of users – all at once, and while delivering stable, consistent performance.
Flexibility to choose level of investment
One of the major advantages MongoDB offers Gilt is the ability to scale hardware investment as needed. For many applications, MongoDB’s performance on commodity hardware or virtualized servers meets requirements. But in some cases, sub-millisecond response times are a requirement, and worth additional investment; this can be easily accomplished by providing MongoDB additional hardware resources. “That level of flexibility is difficulty to achieve with Postgres,” said Bowman.
One of Gilt’s biggest challenges is how to build data models for complex and constantly changing workflows. A single sale goes through hundreds of changes, and may need approval from dozens of different people. In addition, with changes being made to this workflow at any time, sales are dependent on being able to quickly reflect these data changes and make them available to the consumer-facing site.
MongoDB enables developers to decompose a complicated object like a sale (with hundreds of different parts and approvals) into a hierarchical tree. Using time-stamped objects, MongoDB stores the delta between each accepted state, making it straightforward to make changes to both the data model and the approval process itself, while quickly providing information on the latest state of an object.
The change in perspective MongoDB offers developers has enabled an entirely new approach to managing this complex workflow and approval process, which would be virtually impossible using a relational database. As a result, Gilt can safely make changes to critical changes up to the last seconds before a sale.
As the system started to scale both vertically and horizontally, it quickly became complicated to scale Postgres to “web scale.” While Gilt still needs professional DBAs to keep Postgres servers running really well, Bowman says it’s a “different mindset” with MongoDB; MongoDB does not require the DBA investment.
Server hardware configuration: 96GB of RAM per server with SSDs
Replica Sets: 1 replica set with 3 nodes in largest cluster
Application Language: Java / Scala (with Play! framework), using Casbah and Morphia to interface with MongoDB
Other database technologies: Postgres, Voldemort
Monitoring systems: MongoDB Management Service, TSDB
Due to its ease of use, ease of maintenance and ability to deliver high performance on commodity hardware, MongoDB delivers infrastructure cost savings.
MongoDB also enables Gilt developers to get new elements live on the site more quickly, thereby delivering a better user experience. “MongoDB simplifies certain hard problems [like the delta model],” said Bowman. “It reduces the complexity of implementation and is more efficient top to bottom – there’s monetary value to that.”
The Gilt development team is currently focused on using MongoDB for user data. In the future, they plan to experiment with new ways to work with and store product data in MongoDB.
Location: New York, NY