Intelligent Stream-Filtering Using MongoDB

Mihnea Giurgea

September 18, 2011

We are one of the top social media monitoring platforms today and the volume of data is one of our biggest challenges on a daily basis. Let’s say... we processes a large volume of data every minute. But making sense of all the social data available on Twitter, Facebook or blogs is a different challenge all-together. So we introduced filters. In real-time. We needed an intelligent way to index this stream, so that we’re able to perform fast searches over it using multiple filters, like language, location, time, etc. This session is about how we managed to do just that using MongoDB on AWS.