Leaf in the Wild: How Loopd uses MongoDB to Power its Advanced Location-Tracking Platform for Conferences
Conferences can be incredibly hectic experiences for everyone involved. You have attendees wanting to meet and exchange information, sponsors and exhibitors looking to maximize foot traffic to their booths, and the conference hosts trying to get a sense of how they can optimize their event and if it was all worth it in the end.
While sponsors usually do get a lead list immediately after an event for their troubles, attendees often struggle to remember who they actually spoke to and event hosts are often left in the dark about what they can do to maximize the returns on their investments. Enter Loopd, an advanced events engagement platform.
I sat down with their CEO, Brian Friedman, to understand how they’re using MongoDB to help conference attendees and event hosts separate the signal from the noise.
Tell us about Loopd.
Loopd provides physical intelligence for corporate events. We help corporate marketers learn how people interact with each other, with their company, and with their company's products. The Loopd event engagement system is the industry's only bi-directional CRM solution that enables the exchange of content and contact information passively and automatically. We equip conference attendees with Loopd wearable badges, which can be used to easily exchange contact information or gain entry into sessions. Through our enterprise IoT analytics and sensors, we then gather and interpret rich data so that marketers have a more sophisticated understanding of business relationships and interactions at conferences, exhibits and product activation events.
Some of our clients include Intel, Box, Twilio, and MongoDB.
How are you using MongoDB?
We use MongoDB to store millions of datapoints from connected advertising and Bluetooth LE Loopd Badges on the conference floor. All of the attendee movement data captured by the Loopd Badge at an event can be thought of as time series data associated with location information. We track each Loopd Badge’s location and movement path in real time during the event. As a result, we handle heavy write operations during an event to make sure any and all calculations are consistent, timely, and accurate.
We also use the database for real-time analysis. For example, we calculate the number of attendee visits & returns, and average time durations in near real time. We use the aggregation framework in MongoDB to make this happen.
What did you use before MongoDB?
Before MongoDB, we used PostgreSQL as our main data store. We used Redis as a temporary data buffer queue for storing new movement data. The data was dumped, inserted, and updated into rows in the SQL database once per second. The raw location data was read and parsed from the SQL database into a user-readable format. We needed a temporary buffer because the high volume of insert and update requests drained available resources.
What challenges did you face with PostgreSQL?
With PostgreSQL, we needed a separate Redis caching server to buffer write and update operations before storing them in the database, which added architectural and operational complexity. It also wasn’t easy to scale as it’s not designed to be deployed across multiple instances.
How did MongoDB help you resolve those challenges?
When we switched to MongoDB from PostgreSQL, our write throughput significantly increased, removing the need for a separate caching server in between the client and the database. We were able to halve our VM resource consumption (CPU power and memory), which translated to significant cost savings. As a bonus, our simplified underlying architecture is now much easier to manage.
Finally, one of the great things about MongoDB is its data model flexibility, which allows us to rapidly adapt our schema to support new application demands, without the need to incur downtime or manage complex schema migrations.
Please describe your MongoDB deployment.
We typically run one replica set per event. The database size depends on the event — for MongoDB World 2016, we generated about 2 million documents over the course of a couple of days. We don’t shard our MongoDB deployments yet but having that ability in our back pocket will be very important for us going forward.
At the moment, all of our read queries are executed on the secondaries in the replica set, which means write throughput isn’t impacted by read operations. The smallest analytics window in our application is a minute, which means we can tolerate any eventual consistency from secondary reads.
Our MongoDB deployments are hosted in Google Cloud VM instances. We’re exploring using containers but they’re currently not in use for any production environments. We’re also evaluating Spark and Hadoop for doing some more interesting things with the data we have in MongoDB.
What version of MongoDB are you running?
We use MongoDB 3.2. We find the added document validation feature very valuable for checking data types. While we will still perform application-level error validation, we appreciate this added level of security.
What advice do you have for other companies looking to start with MongoDB?
MongoDB is flexible, scalable, and quite developer and DBA friendly, even if you’re used to RDBMS.
We would recommend familiarizing yourself with the basic concepts of MongoDB first, heavily leaning on the community during learning. I’d also recommend reading the production notes to optimize system configuration operational parameters.
Brian, thanks for taking the time to share your story with the MongoDB community.
Thinking of migrating to MongoDB from a relational database? Learn more from our guide: