Eliot Horowitz

Eliot Horowitz: Making MongoDB Dramatically Faster, Much More Pluggable And Easier To Scale

As MongoDB has evolved over the past seven years, three design principles have guided its development: Developer Productivity - Never do anything that slows developer productivity and always seek to improve it; Horizontal Scalability - MongoDB development should always maximize scalability; and Operational Scalability - It should be as easy to manage 1,000 MongoDB servers as it is to manage one While much of the market focuses on #2, the reality is that horizontal scalability is mere table stakes for a modern database, and MongoDB has it in spades . No other database can boast as many 50+-node deployments as MongoDB, or even come close. With hundreds of thousands of deployments, ranging from a single node on a laptop to 1,000+ node clusters, MongoDB scales better than any other database. But horizontal scale, again, isn't the most interesting problem. Maximizing developer and operations productivity is, and this week at MongoDB World Eliot Horowitz, MongoDB co-founder and CTO, took to the stage to reveal big improvements to MongoDB that will excite developers and operations equally. (You can watch the full video of the keynote here .) While Eliot has blogged about the imminent arrival of document-level locking, pluggable storage and automated management of MongoDB, this was the first time the company has offered a live demo of each. Concurrency Over the last seven years we’ve made a lot of improvements, but we’re still improving it. Despite these improvements, Eliot humorously pointed out a definition of concurrency that says "Cooperative time sharing at database level using knowledge about file system cache to avoid locking around disk access" but reads like "database-level locking" to many in the MongoDB community. No more. On stage Eliot demoed document-level locking and indicated it will be ready in 2014. By making MongoDB's locking more granular, it's possible to allow more simultaneous writes to occur without locking up resources or hurting performance. In fact, Eliot commented that "we've easily seen a 10X speed increase” of MongoDB with the improvement, and that’s just on a pre-release beta build running on a standard desktop machine. On better hardware the improvements should be even more significant. Even better, Eliot noted that MongoDB will it will be a "textbook implementation that will be familiar to relational database engineers." Pluggable Storage MongoDB's storage engine has served the community well for seven years. But with hundreds of thousands of MongoDB deployments come a myriad of different use cases and work loads, and no single storage engine that perfectly fits all of them. As such Eliot announced a new pluggable storage API that allows developers to choose the storage engine that best suits their workloads. And, importantly, the API will be detailed enough to support all MongoDB features, so that in developers' chase for performance their operations counterparts don't have to give up operational scalability. "When MongoDB 2.8 comes out," declared Eliot, "We think it’s important that you be able to add a new node to a replica set and try a new storage engine out that may be focused on security, performance or whatever you want. You could actually have a replica set with multiple storage engines: one optimized for BI workloads, one focused on compression, etc." With a range of different storage engines to choose from, Eliot pointed to a few likely places to start In-memory (MongoDB has already done this one) RocksDB (MongoDB has built a preliminary version of this one, too) InnoDB TokuKV FusionIO Automation It has always been possible to do rolling upgrades of MongoDB. It has also always been a bit harder than necessary. To make it easier, Eliot announced (and demoed) MMS Automation, a core component of the MongoDB Management Service (MMS) going forward. As the audience watched, Eliot provisioned and upgraded a 10-node cluster in a matter of minutes. The crowd cheered, and for good reason: suddenly MongoDB is as easy to administer as it is to develop. In other words, MongoDB just lowered the bar for greater adoption of its wildly popular database, even as it raised the bar for every other database that hopes to compete for today's workloads. Good luck with that. To see all MongoDB World presentations, visit the [MongoDB World Presentations](https://www.mongodb.com/mongodb-world/presentations) page.