Ten years ago, India took its love affair with cricket to a new level with the launch of the Indian Premier League (IPL). Based on a pared-down, Twenty20 format of the game, the idea was to capture some of the excitement of faster-paced sports.
The idea worked. The IPL’s Season 11 just finished with 700 million viewers on the scoreboard, not far off the estimated 900 million global audience for the FIFA 2018 World Cup final.
With nearly 30 percent of the IPL audience watching online, the market for fan-focused tech services is irresistible – if you can keep up with demand.
FanFight is a fantasy cricket league where fans pick a pool of cricketers and get rewarded according to real-life success on the field. “There were around 50 new companies launching alongside us in that market this season,” says Akhil Suhag, CEO and co-founder.
And FanFight is bit of a fan itself, having bet on MongoDB.
“Obviously, we needed a competitive advantage. Most of those 50 fell over on the first day. We didn’t,” says Akhil. The platform has over a million registered users, some 50,000 of whom are online concurrently at peak times; not bad going for a product launched this year. “We started developing in April 2017, took six months to build the system, and had 70k users by March 2018. The IPL season started in April, and by the time it finished at the end of May we had 1.1 million users.”
Challenges… hit for six
Part of the platform’s popularity is that while gambling is illegal in India, the fantasy sports approach of paying to enter a game where you choose your players and get rewarded by their performance is not. (See screenshot of FanFight’s user interface below.)
Screenshot showing the recently redesigned FanFight app.
It wasn’t plain sailing getting match fit, Akhil reveals. “We had experience with MongoDB from previous projects, so we did everything around that and node.js, now all deployed on AWS leveraging serverless Lambda functions for business logic.” At first, things went well. “Our first 20k, 30k users were no problem, response times were good and back-end traffic was low. At 50k, we started to see response times go up to 300ms and CPUs started to max out. The read and query operations were getting slower and slower.”
With FanFight promising fantasy league users the “best and fastest platform for fans,” they needed to overhaul the original monolithic app into easier microservices on auto-scaling EC2 instances. The user experience relies on complex aggregations returning nested JSON data of total winnings, total contests and series joined, total teams created, contests won, past performance, and friends referred to the app. “People want to see how their pool is performing in reaction to the player performances from the actual matches. Everything is updated live. With 50,000 concurrent users and their unique player pools, that means very heavy read and write loads,” says Akhil.
Furthermore, as a lean team of 22 people trying to launch a new product in a time-sensitive and competitor-heavy space, FanFight was spending more than 50% of its time trying to maintain operations and build out monitoring dashboards instead of developing new and enhanced app features, and implementing the microservices architecture. The team was burning through nights and weekends, but still performance was getting worse with only a few hundred concurrent connections. FanFight needed to find another solution, and – with thousands of new users coming onto the platform monthly, and the IPL season approaching – they needed to find it fast.
The FanFight team looked for an easy managed service offering that provided out-of-the-box analytics to alleviate the operational overhead. “At 100k users, we switched to MongoDB Atlas. It would have been a disaster if we hadn’t.” Of the 22 FanFight employees, only 8 developers are creating, testing and deploying all back-end and front-end services plus the mobile apps.
After seamlessly migrating to Atlas on AWS, the FanFight team immediately leveraged Atlas’ real-time monitoring metrics to right-size their deployment by increasing instance size from M50 to M100, which brought end-to-end app-to-user response times down to 100ms. Hosted in the AWS Mumbai region, the deployment is now all handled by Atlas, and the FanFight team scored:
0 downtime scaling or during upgrades and <50ms auto-scaling response times
No overhead when moving data between development, staging and production environments
Easy index analysis and performance tuning utilizing MongoDB Compass
With the database now on Atlas distributed across server nodes for maximum fault tolerance and high availability, the FanFight team was able to focus on integrating Amazon platform capabilities.
75% (soon to be 100%) of back-end business logic running on serverless Lambda functions
Amazon EMR with Spark for distributed computing of live match data based on real match events
Amazon SQS for inter-server communication – triggering and scheduling various internal functions
Amazon Cloudwatch for scheduling business-critical events (e.g., match start, contest suspension, payment triggers, etc.)
Amazon Elastic Beanstalk integration with Github for CI/CD
“We have a roadmap now for more efficient coding,” says Akhil, “and we know where to research.” In conjunction with its experience of AWS node.js server auto-scaling – “20-30 app server instances on average, 200 at peak” – the company is confident that it can plan for aggressive expansion next season.
“We’re aiming at minimum 15 times growth, to 15 or maybe 20 million users, but that won’t be much good if we’re not the fastest and best experience for the fans,” Akhil says. (See table below for FanFight’s growth goals.)
|2018 (actual)||2019 (goal)|
|1.2 million users||20 million users|
|1200 writes per second||10800–12000 writes per second (9-10x more)|
|50ms average read query performance||50ms average read query performance|
|5000 reads per second||25000 reads per second (5x more)|
|25GB of data||500GB of data (20x more)|
So, the game plan for IPL 12 isn’t just bigger numbers. “We think multi-document ACID transactions in MongoDB 4.0 may be very helpful. We’re also looking at user behaviour connected to payment data, building a fully-fledged analytics package, engagement-focused marketing campaigns. It’s all based on business requirements, and those are leading our move away from the monolithic platform we have now to microservices.”
And there are two other clear goals: doubling the size of the developer team and, by the end of the next season, making an operating profit. “Revenue and user numbers are very closely tied,” says Akhil, “so that gives us a solid basis for planning growth. When we look at what we want to do, how overworked our developers are, and our timescale for the next revision of three months, recruiting another 8 to 10 people is an absolute must."