Shutterfly

Shutterfly, a leading Internet-based social expression and personal publishing service, safeguards more than six billion images for millions of customers. The company’s premium online photo service and products enable consumers to share memories, in the form of photos and videos, and turn everyday pictures into creative keepsakes, such as photo books and personalized websites.

As the only photo sharing site that does not down-sample, compress, or force delete photos, Shutterfly faced massive data growth that pushed the performance limits of its existing Oracle database. After an extensive analysis of open source relational and non-relational alternatives, Shutterfly chose MongoDB as its persistent data store. MongoDB delivers high performance, scalability and fast time to market – a trifecta that helps Shutterfly deliver unrivaled service to customers.

The Problem

To stay ahead of the competition and provide the highest quality customer experience, Shutterfly continually implements new features to improve the various ways in which users interact with their products and the web site. After a decade of developing on Oracle, Shutterfly’s code base was brittle, bringing many engineering use cases to a painful crawl. Applications took a long time to build properly; or, once built, they wouldn’t perform quickly enough. Oracle proved too costly an expense (between software licenses and hardware) and offered sub-par performance. In 2009, Shutterfly began searching for a replacement database that could grow with the business and sought an open source solution that would enable them to scale horizontally across commodity hardware, speed up application performance and improve time to market. MongoDB fit the bill.

Why MongoDB?

MongoDB provided Shutterfly with an agile, high performance, scalable solution at a low cost. MongoDB works seamlessly with Shutterfly’s services-based architecture, providing a clean and fast way to build applications, as well as the flexibility to match multiple use cases. To date, Shutterfly has developed nearly a dozen projects on MongoDB, which now stores more than 20TB of data.

ACCELERATED TIME TO MARKET

The “really killer reason” for using MongoDB is its rich JSON-based data structure, which offers Shutterfly an agile approach to develop software, explained Kenny Gorman, director of data services. With MongoDB, the Shutterfly team can quickly develop and deploy new applications, especially Web 2.0 and social features (e.g. tagging, likes, comments), which were difficult and costly to implement on their legacy system. MongoDB’s document model enriched Shutterfly’s data modeling abilities, allowing for new query patterns previously unavailable, while the flexible schema cut development cycles and accelerated time to market.

IMPROVED PERFORMANCE

Leaving behind complex legacy code simplified the Shutterfly architecture, allowing the company to inexpensively improve individual page and site performance. Average latency for inserts dropped to 2ms from 400ms, and Shutterfly customers now get the speed they expect when performing tasks like sharing and commenting on photos.

RELIABLE SERVER SUPPORT

Shutterfly satisfied its extremely high uptime requirements (more than 10,000 ex/sec) with MongoDB’s replica sets, which enable high availability through automatic failover of nodes and recovery of member nodes. Their unique architecture comprises four members for every replica set, and the ability to write data to more than one member has been critical for data quality and safety.

AT A GLANCE MONITORING

Shutterfly uses the MongoDB Management Service (MMS) as the primary tool for gaining system visibility and insights. Not only has it saved on DBA resources, but MMS has provided them with the confidence to move forward on projects more quickly. Plus, “it’s enabled us to have a tight link with MongoDB support,” said Gorman.

OPEN-SOURCE COMMUNITY

Shutterfly predicted early on that MongoDB had the passion, vision and know-how to build both a top-notch product and a large community. And they were right: the MongoDB community has been a great asset to their projects. “With so much energy behind MongoDB, we’ve been very pleased with the overall direction and velocity of development,” said Gorman.

Results

Shutterfly’s switch from Oracle to MongoDB resulted in a 9x performance improvement and substantial cost reduction. Additionally, Shutterfly can now develop applications in a couple of sprints – a timeframe of weeks rather than tens of months.

Lessons Learned

Gorman’s advice is to pick the right tool for the job. The recent NoSQL movement has borne new tools that thrive at specific uses; the key is to use each tool appropriately. MongoDB performs its core capabilities extremely well, but don’t ask it to do something it doesn’t do, he advised. “Your data has a format it wants to live in, so listen carefully and it will tell you.”

What’s Next?

MongoDB is the cornerstone of Shutterfly’s persistence architecture, an important capability that developers have at their fingertips. “We’ve seen huge gains so far and we’re very excited about the future,” said Gorman. “We continually evaluate all new projects against MongoDB, and if it’s a fit, we use it. I expect it will be a big part of fantastic new features for our customers.”