Lessons Learned from Migrating 2+ Billion Documents at Craigslist

Jeremy Zawodny

May 24, 2011

Last fall I spoke at the MongoSV conference about using MongoDB for the historical archive at Craigslist. Since then we've successfully migrated roughly 2.2 billion documents into a MongoDB deployment using sharding and replica sets. This session will briefly review the motivation for this migration and focus on lessons learned along the way, including MongoDB features we wished we had and how to think about undertaking a data migration project of this size.