Intuit empowers small businesses, accountants, and individuals with tax preparation and financial software. Intuit Websites, which enables small businesses to build websites without any technical expertise, relies on a MongoDB-powered real-time analytics tool to derive interesting and actionable patterns from their customers’ website traffic.
In addition to helping customers create and host their websites, Intuit wanted to collect and analyze data from these sites. Intuit could then use the data to recommend conversion and lead generation improvements to their customers. By providing these analytics, Intuit hoped to enable their customers to grow their businesses.
Intuit required a datastore that could serve as a centralized repository for complex data and maintain high performance for aggregation and analytics. Intuit initially tried using off the shelf products like Google Analytics and Omniture, but realized that they needed a custom tool to meet their requirements. But building the analytics tool in house immediately posed a few challenges.
Intuit hosts more than 500,000 websites. Though they had some information about customers’ websites, they didn’t have it in the detail that they wanted.
Data Aggregation and Performance
With 10 years worth of user data, it took several days to process the information using a relational database.
Prior to the implementation of MongoDB, analytics data was scattered over multiple sources. This posed technical and logistical problems, and prevented analyzing this data in a timely fashion.
Intuit initially planned to build their analytics service using a relational database. At the time, MySQL or Oracle seemed like an obvious choice due to existing infrastructure. Yet, the team quickly encountered performance issues using a relational system. While there was also an existing Hadoop infrastructure in house, the team quickly realized that MongoDB's querying and map reduce functionality could serve as a simpler, higher-performance solution than a complex Hadoop deployment.
In their benchmarks, Intuit found MongoDB to be 2.5 times faster for writes than MySQL. Performance for both reads and writes were better than the other relational and non-relational choices evaluated. Hadoop might have provided the analytics that they were looking for, but it would have added additional systems to their setup, increased complexity, and the batch processing design of Hadoop would have delayed the delivery of reports to end users.
Intuit spent one week developing a prototype analytics solution based on MongoDB. In that time they were able to become proficient in MongoDB development, and found that they could develop application features more quickly for MongoDB than for relational databases.
"We did a prototype for one week and within one week we had made big progress. Very big progress. It was so amazing that we decided let's go with this,” said Nirmala Ranganathan. “The cool thing about MongoDB is that you change – boom... it works in QA, it works in production… that is why we can deploy once a day or sometimes twice a week based on what the changes are. It gives so much flexibility and confidence to us as well as the build team that actually does the job of deployment.”
The strength of the MongoDB community confirmed Intuit's confidence in their choice to use MongoDB. During the prototyping period, Intuit engineers would frequently post question on the user forums, where they would get quick responses from more experienced community users. Around this time, Intuit engineers registered for the first MongoSF conference, where they saw evidence of the vibrant community around MongoDB. Meeting other users overcoming similar challenges with analytics applications convinced them that using MongoDB was the right decision. Not long after deploying their app, the team at Intuit was contributing back to the MongoDB community with a presentation of their own at the next MongoDB event in the Bay Area.