Shopcade

Launched in December 2011, Shopcade is a new way to shop and discover great products online. The social e-commerce application enables users to create a personalized page of all their favorite products and to share them with friends and followers in a fun and innovative way. The modern shopping application brings trust and relevancy to e-commerce, helping consumers find relevant trending products by leveraging their social network.

Shopcade initially built its prototype on MySQL, which limited its ability to scale as the company expanded into new markets and retail categories. Shopcade quickly made the switch to NoSQL, choosing MongoDB for its ease of development in an agile environment, flexible data model and scalability.

The Problem

Shopcade initially planned a UK-specific launch, preparing to support 2-3 million products from the outset. When Shopcade saw that the social e-commerce industry was heating up, it expanded the launch to the U.S. as well – resulting in the rapid development of a much larger product database.

Built on a standard LAMP stack, the prototype reached 5 million products. However, MySQL’s relational model couldn’t accommodate growth. Shopcade needed to double the size of its products database “without even thinking about it,” said CTO Evan Adelman. “We could squeeze MySQL and spend a lot of time on optimizations to try to make it more performant, or very early on we could make the transition to a NoSQL database.”

After evaluating multiple NoSQL options, they selected MongoDB.

Why MongoDB?

Today, Shopcade’s social e-commerce application enabled by MongoDB features more than 70 million products and over 30,000 brands, such as Target, Ray-Ban and Barnes & Noble. When the application is released to broader beta by fall 2012, the application will provide consumers with access to more than 100 million products.  

EASE OF MAINTENANCE IN AGILE SPRINTS

“Having worked with MongoDB for a year now, I forget all the nightmares that little things like importing/exporting test data can bring up in MySQL,” said Adelman. “The database maintenance in an agile environment is a million times less with MongoDB.”

If developers needed to effect a change in MySQL, they’d have to talk to a DBA. The resulting code transition and migration of existing data into a new schema (or coming up with a new way to migrate between releases) was agonizing within two-week sprints. With MongoDB, all that happens on the application side and developers aren’t handcuffed by the database.

FLEXIBILITY TO HANDLE VARYING PRODUCT ATTRIBUTES

MongoDB’s document structure is quite powerful once you start playing with it, said Adelman. Shopcade can easily import products with varying attributes (e.g. author vs. musician vs. album title) and store that data right in the product object. Since other e-commerce sites built on a relational database aren’t able to do that as easily, MongoDB gives Shopcade a competitive edge; only by aggregating product information from the different merchants on a daily basis can Shopcade ensure that consumers have access to the most relevant products at the best price.

SCALABILITY FOR GLOBAL REACH

Thanks to MongoDB, Shopcade can readily expand its product catalogue. The transition from 5 million to 40 million products required a very limited amount of work. “We would’ve had to jump through hoops to get to 40 million in MySQL,” said Adelman. Shopcade’s current status of 65 million products and presence in the UK and U.S. is only the tip of the iceberg. As the company expands globally, they’ll be able to add 30 million products per country without even thinking about it. “We can add shards at will and grow the database significantly without spending loads of engineering time on optimizing the database,” said Adelman.

IMPORT TIME SLASHED BY NEARLY 90%

An optimal product import to match new data against MongoDB, remove out of stock products and change products/pricing is comprised of 40-50GB of data per day. Such a large product import that previously took four days now takes only 12 hours in the MongoDB sharded environment.

CONSISTENT PERFORMANCE

Adding more data to a page typically causes latency issues. With MongoDB, Shopcade can add data (e.g. points earned, stored descriptions, list names) or new features (e.g. trendsetter score) directly to the user object without worrying about how it will impact performance.

Results

“The amount of progress we’ve made in a year with a fairly small development team is quite astounding,” said Adelman. Shopcade came up with a vision, executed on it, created a social network with shopping and gaming features, and has had rapid user adoption – early success that wouldn’t have been possible without MongoDB.

With MongoDB, Shopcade does not have to worry about the database – Will this new functionality fit the data model? Instead, they can focus on meeting customer needs – Is this the right functionality for users?

Next Steps?

Due to MongoDB’s flexibility and scalability, Shopcade will ultimately allow users to shop and share anything in the world.

Shopcade is also moving towards a gaming model where users will be rewarded for sharing products and being engaged in the community. By completing missions, users will earn points that will be converted into monetary rewards, such as a vouchers and even exclusive offers. MongoDB supports this gamification by making it easy to liberally log and store massive collections of user activity. As a result, Shopcade has the assurance that they can easily access the information, regardless of how they decide to use it for future gaming and social shopping features.