Eliot Horowitz

12 results

MongoDB now released under the Server Side Public License

MongoDB has created a new software license called the Server Side Public License , or SSPL. The SSPL clarifies the conditions for making MongoDB publicly available as a service, to ensure we can continue to invest in building MongoDB for our users rather than in costly litigation over enforcing the AGPL. All subsequent versions and patch releases to prior versions of MongoDB made after October 16th, 2018 will be issued under the new SSPL. Software as a service is one of the most significant and far-reaching paradigm shifts the technology industry has ever seen, and the market is moving to adopt it rapidly. This change is motivated by one simple truth: it is inefficient to operate software at scale when you just want to make use of its capabilities. This should be a time of incredible opportunity for open source. The revenue generated by a service can be a great source of funding for open source projects, far greater than what has historically been available. The reality, however, is that once an open source project becomes interesting, it is too easy for large cloud vendors to capture most of the value while contributing little or nothing back to the community. As a result, smaller companies are understandably unwilling to wager their existence against the strategic interests of the large cloud vendors, and most new software is being written as closed source. The best current solution for an open source company is to license their software under the AGPL, which requires a management stack used to operate that software as a service to be made available under the terms of the AGPL. This approach was believed to be good enough, as most people understood their obligations to comply with AGPL. However, as AGPL-licensed software like MongoDB has become more popular, organizations like the international cloud providers have begun to test the boundaries of this license. We would prefer to avoid litigation to defend the AGPL but instead devote our time to build great products for the community and our customers. Since we own 100% of the copyright of MongoDB, one option available to us was to convert our open source license to a closed source license. We chose not to because we fundamentally believe in open source software. We believe an open source approach leads to more valuable, robust, and secure software, and it directly enables a stronger community and better products. We also could have licensed most of the code for the MongoDB server as AGPL while applying a closed license to some critical files. We chose not to do that because we believe that a uniform licensing approach, where all the code in a repository shares a single license, makes it easier to understand the obligations of using that code, leading to a stronger development community. The community needs an updated open source license that builds on the spirit of the AGPL, but makes explicit the conditions for providing the licensed software as a service. The SSPL is designed to make sure that companies who do run a publicly available MongoDB (or any software subject to the SSPL) as a service are giving back to the community. It should be noted that the new license maintains all of the same freedoms the community has always had with MongoDB under the AGPL - they are free to use, review, modify, and redistribute the source code. We based the SSPL on GPL v3, with one new section that sets out an explicit condition for offering the licensed software as a service. We believe that the SSPL meets the standards for an open source license and are working to have it approved by the OSI. Obviously, this new license helps our business, and that is one of the primary reasons we made this decision, but it is also critical for the MongoDB community. Over the past decade MongoDB has invested approximately $300M in research and development to offer an open source database to everyone, and this change protects our ability to invest aggressively in R&D to drive further innovation and value for the community. I am convinced that the SSPL will have a positive effect on the open source community by encouraging increased investment into open source server-side software. As always, we are listening closely for your feedback , and we are eager to work with the community to drive towards a more open future. Resources The Server Side Public License SSPL FAQ Press Release

October 16, 2018

Introducing the Best Database for Modern Applications

The announcements we made today at MongoDB World 2018 represent a significant milestone in the evolution of MongoDB, making it the database of choice for all modern applications. Broadly speaking, there are three reasons for this: The document data model – presenting you the best way to work with data . It’s distributed by design – allowing you to intelligently put data where you want it . A unified experience that gives you the freedom to run anywhere – allowing you to future-proof your work and eliminate vendor lock-in. There is a ton of new stuff, and so I wanted to give you a summary of what I covered during my keynote, with links to key resources so you can learn more. Best Way to Work with Data Today we released MongoDB Server 4.0 for General Availability. The highlight of the release is multi-document ACID transactions , which we previewed back in February with a beta program that attracted thousands of members of the community, putting transactions through their paces and providing invaluable feedback to the engineering team. We’ve implemented transactions so they feel just like the transactions you are familiar with from relational databases. They enforce snapshot isolation to provide a consistent view of data, and all-or-nothing execution to maintain data integrity. And while the document model means multi-document transactions aren’t necessary for most operations, with them it’s even easier for you to address a complete range of use cases with MongoDB. It’s no secret how much I love MongoDB’s aggregation framework. Building queries stage-by-stage, checking your output as you go is by far a better way to write your most complicated queries than dealing with a monolithic snarl of SQL. To make that workflow even better, we’ve enhanced MongoDB Compass with the aggregation pipeline builder , which provides stage-by-stage, real-time feedback on the documents flowing through your pipelines. It’s easier than ever to deploy sophisticated processing pipelines that transform, aggregate, and analyze your data, all from the simple and intuitive MongoDB Compass GUI. You can then export the pipelines, and any other queries you create in Compass, to the native code of your preferred programming language. Server 4.0 also adds type conversions to the aggregation pipeline. With the new $convert operator you can transform mixed data types into standardized, cleansed formats natively within the database , preparing it for BI and machine learning, while eliminating costly, slow, and fragile ETL processes. Extending the tools you can use to work with data managed by the server, we announced the public beta of MongoDB Charts , which provides the fastest and easiest way to get insights into your operational data, in real time. With Charts, you can create and share visualisations of your MongoDB data, using a document-native interface, without needing to move it into other systems or leverage third-party tools. Documents and MongoDB’s query language are the best way to work with data, and to bring that power out of the datacenter and into the hands of app developers, MongoDB Stitch , which is GA as of today, provides two of its four services: QueryAnywhere and Functions. Using the authentication and declarative access control rules of Stitch QueryAnywhere , we can end the horrid practice of implementing shadow query languages in REST on top of application servers that just turn those REST calls into real query languages. With a native SDK, developers can make use of the full power of MongoDB from mobile and JavaScript applications, while Stitch makes sure that the right permissions are observed. Stitch Functions , JavaScript functions that execute with full access to application context, let developers compose their business logic with access to Atlas and calls to external services. With these two services, it’s easy to build complete applications without standing up a single application server. Intelligently Put Data Where you Want It As a distributed system, MongoDB enables you to spread data out across a cluster of nodes for resilience, scalability, and workload isolation. Unlike other distributed databases that randomly spray data around a cluster, MongoDB allows you to define controls that place data on specific nodes, for example in a specific region for low latency reads and writes, and for compliance with new privacy regulations. The new Global Clusters introduced to MongoDB Atlas allow you to deploy a geographically distributed, fully managed database that provides low latency writes and reads to users anywhere, with data placement controls for regulatory compliance. We also announced Atlas Enterprise, offering new security controls including LDAP integration, the encrypted storage engine with bring-your-own key management, and database-level auditing. Organizations can now also use databases managed by MongoDB Atlas to build HIPAA-compliant applications under an executed Business Associate Agreement (BAA) with MongoDB, Inc. With these now announcements, MongoDB Atlas is the most secure cloud database service available anywhere. Coding in a distributed world also means that the traditional means of responding to events in a database are no longer viable. So Stitch Triggers , also GA today, makes it possible by building on the Change Streams introduced in MongoDB 3.6. When you create a trigger, Stitch manages a change stream on your behalf, providing real-time notifications to Stitch Functions, which can react in all the ways functions can, from updating analytics rollup collections, to sending email or text messages, or kicking off other external services like Kafka or Kinesis. Freedom to Run Anywhere Whether you want to consume your database as a service in MongoDB Atlas , or manage it yourself on your own infrastructure, the announcements today make that even easier. We deliver a data platform that runs the same everywhere, that leverages the benefits of multi-cloud strategy with no lock-in, and is available in 50+ regions across the major cloud providers. If you want to run MongoDB yourself, then we have released our new free MongoDB monitoring cloud service . The service is available to all MongoDB users, without needing to install an agent, navigate a paywall, or complete a registration form. You will be able to see the metrics and topology about your environment from the moment free monitoring is enabled. You can enable free monitoring easily using the MongoDB shell, MongoDB Compass, or by starting the mongod process with the new db.enableFreeMonitoring() command line option, and you can opt out at any time. We’re seeing more DevOps teams leveraging the power of containerization and technologies like Kubernetes and Red Hat OpenShift to manage containerized clusters. Today we announced beta of the new MongoDB Enterprise Operator for Kubernetes, enabling you to deploy and manage MongoDB clusters from within the Kubernetes API, without having to connect separately to Ops Manager . You can learn more by reading our Red Hat OpenShift and MongoDB blog , and checking out the repository on GitHub . What’s Next? So as you can see, that’s a ton of stuff. The announcements today represent our biggest set of releases yet, and we’re incredibly excited to get it into your hands and see what amazing things you do with them. Head over to our MongoDB World 2018 announcements page for more resources on each of these new products and services. Please note : This article previously discussed MongoDB Mobile/Sync. Those products are currently being deprecated as we work towards a public beta of MongoDB Realm. To learn more about this, see the MongoDB Realm site .

June 27, 2018

MongoDB named a leader in The Forrester Wave™: Document Stores, Q3 2016

Today, Forrester released The Forrester Wave™: Document Stores, Q3 2016, recognizing MongoDB as a Leader based on our current offering, strategy, and market presence. The report said that "MongoDB is one of the most popular document stores." As you may recall, a few weeks ago Forrester published another important piece of research on databases, The Forrester Wave™: Big Data NoSQL, Q3 2016. In that report we were also acknowledged as a leader, with a “5 out of 5” score in 19 of the 26 criteria. In this latest report, Forrester evaluates the document database capabilities of a range of database technologies, from MongoDB to traditional relational databases. The very existence of such a report is remarkable, but beyond our position as a Leader, I see in this report evidence that consensus has come to endorse our vision of what the world needs in a datastore. The first release of MongoDB was just over 7 years ago. One of our underlying beliefs was that the document data model is the right way to model data, for a number of reasons. Documents are more flexible and inherently more agile than the relational model; they map to the objects of modern programming models; they are easier and more natural for developers to reason about; and, it turns out, large volumes of documents are much easier to scale to meet the needs of cloud infrastructure and modern workloads. Back in 2009, traditional relational vendors did not hold the same convictions of the importance of the document model. But now, just a few years later, virtually every mainstream database supports the document data model. The reason is clear - the market has embraced the document model, and vendors have either joined the document revolution, or they’re getting left behind. The world is ready for a document database to be its default. 61% of the enterprises surveyed by Forrester for the Big Data NoSQL Wave are using, planning to use, expanding or upgrading to NoSQL over the next 12 months, and we are confident that MongoDB will continue to be the most popular choice. I believe that Forrester’s research makes a critical point - not all document databases are created equal. We developed MongoDB with a broad range of use cases in mind, which is why it excels at so many workloads. Our document model is a superset of other data models, including key-value, graph, object, and relational, and we natively support complex manipulations on these data with operators like $lookup and our new graph operators in 3.4. But it’s not just the data model that makes MongoDB unique. Modern applications require flexible approaches to “always on” global deployments, and easy ways to meet demanding SLAs. Our replication and sharding architecture, pluggable storage engine framework, and tunable consistency mean that an entire spectrum of data semantics can be achieved through configuration, rather than by mixing and matching from a grab-bag of different database products. Another central aspect of our vision is that embracing the flexibility of the document data model does not require sacrificing the ability to safeguard data integrity. While this may be true with most document stores, including relational databases, with MongoDB this is not the case at all. MongoDB’s document validation features allow you to be incredibly strict in how you enforce your schema, from just a few fields, to every field in your model, to no validation at all. Best of all, we don’t require you learn a new language to express schema; instead, we rely on the find() syntax that every MongoDB developer and DBA knows today, which also means we can take advantage of Boolean, geospatial, data typing, wildcard expressions and more - it’s incredibly powerful. Our tools and integrations for MongoDB meet the needs of a broad range of enterprise users. From our beautiful GUI for the database, MongoDB Compass , to our powerful Connector for BI which provides SQL access for analysis, to our management tools like Ops Manager and Cloud Manager , which provide a comprehensive suite of monitoring, automation, and backup and point-in-time recovery capabilities - we’ve got you covered. We're also innovating in the next generation of analytics, machine learning, and streaming with our new MongoDB Connector for Apache Spark . To summarize, our vision for the modern datastore incorporates the flexibility and power of the document model, handles high availability and scale out as core features, retains the ability to safeguard data integrity, and affords enterprises the ability to leverage an ecosystem of analytical tools, and one last thing... it is a first-class citizen of the cloud. This is why we created our database as a service, MongoDB Atlas: the simplest, most robust, and most cost effective way to run MongoDB in the cloud. Using MongoDB Atlas , enterprises can spin up a fully managed, monitored, and backed up cluster with the click of a button, in just a few minutes. Now, regardless of what type of infrastructure an enterprise wants to run, they have the flexibility to deploy and manage MongoDB with ease. Learn how a database can make your organization faster, better, leaner About the Author, Eliot Horowitz Eliot is CTO and Co-Founder of MongoDB. He is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

September 8, 2016

MongoDB named a leader in The Forrester Wave™: Big Data NoSQL, Q3 2016

Today, Forrester released The Forrester Wave™: Big Data NoSQL, Q3 2016, recognizing MongoDB as a Leader based on our performance in the current offering, strategy, and market presence categories. The report said that "MongoDB remains the most popular NoSQL database." It’s always gratifying to see our efforts acknowledged, but beyond our current position as the most popular non-relational database, it is my view that this Forrester Wave report endorses our long-term strategy as clear and on-target. A little over a year ago, I concluded MongoDB World 2015 with a claim that we had entered a new era in which it was reasonable for MongoDB to be an organization's default database; I believe that this recognition shows that we’re getting there. The world is ready for a document database to be its default. 61% of the enterprises surveyed by Forrester are using, planning to use, expanding or upgrading to NoSQL over the next 12 months, and we are confident that MongoDB will continue to be the most popular choice. These enterprises have strategic needs that can only be met by a non-relational database, but they must be prudent about where they invest their fiscal and intellectual capital. They don’t want to stitch together a host of new and disparate technologies, each with its own API and narrow band of appropriate use cases, and take on work to re-implement solutions that were working fine in their relational ecosystem. We developed MongoDB with this in mind, which is why it excels at so many workloads. Our document model is a superset of other data models, including key-value, graph, object, and relational, and we natively support complex manipulations on these data with operators like $lookup and our new graph operators in 3.4. Our replication and sharding architecture, pluggable storage engine framework, and configurable read and write behavior mean that an entire spectrum of data semantics can be achieved through configuration, rather than by mixing and matching from a grab-bag of technologies. And because an unconstrained dynamic schema can sometimes be too flexible, features like document validation and tools like MongoDB Compass provide the integrity checking, schema visualization, query development and performance optimization that DBAs often miss in non-relational solutions. We are also mindful of the investment that enterprises have made in the business intelligence ecosystem that surrounds their databases. Our BI Connector allows enterprises to leverage tools like Tableau to derive insights from their data. Protecting investments in existing tools, though, doesn’t mean relying on them exclusively. We're also innovating in the next generation of analytics, machine learning, and streaming with our new MongoDB Connector for Apache Spark . Enterprises also require industrial-grade management solutions for their databases, and MongoDB has met this need with Ops Manager for on-premises management and Cloud Manager for hybrid deployments. Both of these offer monitoring, backup, and management of MongoDB clusters, making it easy to spin up a single instance to experiment with or run a massive cluster with shards spread across the globe. But these days even enterprises are starting to run their infrastructure entirely in the cloud, and we think this operational model suits a large number of teams. That is why we created our database as a service, MongoDB Atlas : the simplest, most robust, and most cost effective way to run MongoDB in the Cloud. Using Atlas, enterprises can spin up a fully managed, monitored, and backed up cluster in under five minutes. Atlas is available today on AWS, with support for Azure and GCP coming soon. Now, regardless of what type of infrastructure an enterprise wants to run, they have the flexibility to deploy and manage MongoDB with ease. After all, your data should serve you, not the other way around. We continually build and evolve MongoDB to deliver that vision, which is why MongoDB is already in use by more than half of all Fortune 100 companies. So thanks to all of our customers, users, and community contributors, for investing in us, for supporting us, for demanding more and more of MongoDB, for pushing it further, into every crazy new use case. We’re right behind you. About the Author, Eliot Horowitz Eliot is CTO and Co-Founder of MongoDB. He is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

August 17, 2016

Announcing MongoDB 3.2

Today we announced the release of MongoDB 3.2, the first since I declared a new era of maturity for our product with 3.0. This release continues to make the case for MongoDB as the default database for an organization, by meeting the needs of more mission critical use cases, and by meeting the needs of more roles within an organization. MongoDB 3.2 is a giant release, and you can read the release notes to see the full list of new features. I’m proud of each feature individually, but moreover, I’m proud of how MongoDB’s engineering team continues to increase its bandwidth. When we were starting out, we had to round-robin between development domains, focusing in turn on whatever was the highest priority, whether it was improving performance, extending the query language, building out our scaling features, or enhancing operational ease. This release demonstrates how our team is firing on all cylinders, able to make progress on many fronts at once. Let’s take a tour... We continued to push the boundaries of the document database model with features like document validation and partial indexes . We introduced huge new components, like the in-memory and encrypted storage engines, and the BI connector. We made infrastructure improvements, such as config servers as replica sets; under-the-hood improvements, such as replacing the JavaScript engine (more on that in another post); and we continued to refine our tools, giving mongodump and mongorestore the ability to compress, pipe, and stream data over a network. The work we’ve done on our core feature set, including an enormous expansion of our aggregation framework and improvements to geo and text indexing, would alone have been a significant release in previous years, but this year we did them alongside the above. We also put significant effort into housekeeping, bringing consistency in CRUD semantics between the new API and the shell, and we added fit and finish, such as optional case and diacritic sensitivity in text search. In Ops Manager land, we added file system backups as an option, a new query profiler, index suggestions, and the ability to do a rolling index build across a replica set.. Huge new features, brand new tools, performance tuning, edge cases filed smooth, and some tidying up -- we did it all this year. And we’re still just getting started. Learn more about MongoDB 3.2, register for our upcoming webinar: Find out what's new Additional Resources Visit the 3.2 information page Download the ‘What’s New in MongoDB 3.2’ white paper Participate in the MongoDB 3.2 Bug Hunt Hear more from Beta users on MongoDB 3.2 Read the release notes for MongoDB 3.2 About the Author - Eliot Horowitz Eliot Horowitz is CTO and Co-Founder of MongoDB. Eliot is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

November 3, 2015

Revisiting $lookup

Two weeks ago I announced that the new aggregation pipeline stage $lookup (a limited left-outer join operator) would be a feature available only in MongoDB Enterprise Advanced Server. Our rationale was that we viewed $lookup as an enabler of other Enterprise features and we were concerned that widespread availability of $lookup in the Community Edition would lead to users treating MongoDB like a relational database. Since that announcement, we've heard from users who disagree with our decision. They believe that $lookup should be core to a document database, and available to anyone developing against it. Dealing with that feedback has been tricky. No serious complaints about the way we sort features into the Enterprise versus Community edition have ever come up before. It required us to have long, careful discussions, in which we had to examine the criteria we use to decide what features belong in a community edition as opposed to a paid version. No simple, clear-cut principles are available -- we have tried very hard to distill them, but none of our attempts stand up to scrutiny. Nonetheless, one thing is clear: this surprised our users unpleasantly, which is something we never want to do. We hear you. $lookup is going to be a community feature. Finding the principle that makes sense of this decision (and which can guide and explain future choices) is important to us, but not as important as the confidence of our community. We’re still concerned that $lookup can be misused to treat MongoDB like a relational database. But instead of limiting its availability, we’re going to help developers know when its use is appropriate, and when it’s an anti-pattern. In the coming months, we will go beyond the existing documentation to provide clear, strong guidance in this area. MongoDB is a commercial enterprise, but it’s also a community effort, and we will never forget that we owe our success to our users. Your passion and feedback help us make MongoDB a better product with every release. Thank you to everyone who made us look twice at $lookup. Learn more about the new features coming in MongoDB 3.2, register for our webinar: What's New in MongoDB 3.2 About the Author - Eliot Horowitz Eliot Horowitz is CTO and Co-Founder of MongoDB. Eliot is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

October 29, 2015

MongoDB on Gartner’s Magic Quadrant for Operational Database Management Systems 2015

This week Gartner released its 2015 Magic Quadrant for Operational Database Management Systems . In this report MongoDB is recognized as a Leader, putting us in a coveted category of companies with both the ability to execute and completeness of vision. The recognition of our current position is exciting, but what I find most encouraging is what history now says about our progress. MongoDB has been recognized in this Magic Quadrant since Gartner released the first Operational DBMS report in 2013, when we appeared as a Niche Player: significant, but wanting improvement in both execution and completeness of vision. In 2014 we again appeared in the Magic Quadrant, but this time positioned as a “Challenger”: strong on execution, but we still had work to do in terms of making our vision clear. While I might not agree with Gartner that we didn’t have a strong vision, their analysis of our trajectory matches mine. When we started MongoDB in 2007, it was to prove that a document-oriented database could alleviate a host of problems that plague application developers. While there was a lot of early validation of that initial concept, we still needed years of building and evolving our product to fully realize it. One way we know we delivered was to see Gartner acknowledge that improvement in execution. Today we once again find our hard work validated with an acknowledgment that our vision is now far more complete and well articulated. This is exactly the trend I see for MongoDB as well. MongoDB has grown to become a mature, organized, and multi-faceted team, full of talented engineers, able to focus on delivering near-term improvements to MongoDB while at the same time steering our product towards a long-term vision. This versatility is timely, because over the past few years, conversations with our users have taught us far more than we could have known at the outset, about the nature of data itself, and its place in application development. Out of this, we have refined a truly far-reaching vision: The database is at the heart of modern applications Databases should provide both high performance and high-level functionality -- you don’t have to choose one or the other Documents are the right foundation, as they serve the needs of a wide range of applications, and can also function as the basis for other models, like key value. A unified and tunable database framework is far better than a portfolio of specialized products, for both development and operations Distributed, scaled, cloud infrastructures will be the default, so managing them must be simple Furthermore, the choices we made, that were radical and untested when we started out, are becoming standard across databases, and the technologies based on them. Our vision drives more than just our product, it drives the industry as a whole. It’s a great time for MongoDB; we’ve got so many great clients, a great roadmap, and a great team capable of delivering it. Read the press release About the Author - Eliot Horowitz Eliot Horowitz is CTO and Co-Founder of MongoDB. Eliot is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

October 13, 2015

Thoughts on our new feature, $lookup

Editors note: the content in this blog was revised in a follow-up post, Revisiting $lookup . One MongoDB 3.2 feature we demoed at MongoDB World in June was $lookup. $lookup is an aggregation pipeline stage that lets you insert data from a different collection into your pipeline. This is effectively a left outer join. $lookup will only be included in MongoDB Enterprise Server, and we’d like to give some background as to why and how we have decided this. When we decided that MongoDB would be a document-based database, we did so for pragmatic technical reasons. The relational database model has several theoretical merits, but also many real-world limitations, chiefly the need to join data together across tables. Joins impede performance, inhibit scaling, and introduce substantial technical and cognitive overhead into all but textbook examples. In the real world, people adopt complicated workarounds in order to get good performance for modern applications: any large, performant, RDBMS-backed application will incorporate ad hoc denormalization, materialized views, external caching layers, and so on. It was these limitations that motivated us to look for a data model that would both benefit developers building applications and also perform well at scale. We believe that documents are a natural, general-purpose data model that suits most applications people want to create today, and that, with careful schema design, allows for applications that are simultaneously more performant, more easily understood, and more scalable than applications that must rely on frequent joins. Using a document model does not mean that joins aren't useful. There's no question that when data is being used for reporting, business intelligence, and analytics, joins are effective. Many of our users, having built fast, flexible, and feature-rich applications with MongoDB, tell us they now have so much application data in MongoDB that they'd like to use their existing BI tools to derive more business-value from it. The full integration with these tools is not limited to $lookup, but also includes MongoDB Connector for BI, which translates SQL queries into Mongo Query Language queries. Our integration with BI and visualization tools can add tremendous value to companies looking for insights into their data, but it requires an outsized amount of time and testing to ensure smooth interoperation with these other costly products. As such, it is offered as part of our MongoDB Enterprise Advanced subscription . Our philosophy continues to be that documents are the right model for application development. If you'd like to try out this feature, it's available in the 3.1.8 Enterprise Server build . About the Author - Eliot Horowitz Eliot Horowitz is CTO and Co-Founder of MongoDB. Eliot is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

September 30, 2015

MongoDB Security Best Practices

Editor's note: This post was edited on June 23, 2015 to reflect the change from MongoDB Management Service to MongoDB Cloud Manager. Learn more here . MongoDB takes security very seriously. Recently a team of German researchers discovered unsecured instances of MongoDB running openly on the internet. Readers who are concerned about access to their systems are reminded of the following resources: The most popular installer for MongoDB (RPM) limits network access to localhost by default. Security is addressed in detail in our Security Manual . The Security Checklist discusses limiting network exposure. Note that the method to do this will vary significantly depending on where the service is hosted (AWS, Azure, locally, etc). Additionally, users of MongoDB Cloud Manager can enable alerts to detect if their deployment is internet exposed (see figure below). A discussion on security is provided in two parts. Part 1 covers Design and Configuration. Part II covers 10 mistakes that can compromise your database. We encourage users who have experienced a security incident for MongoDB to create a vulnerability report . Create a new alert to notify of host being exposed to the public internet. If you are interested in learning more about security best practices watch our on demand webinar. Securing your MongoDB deployment About the Author - Eliot Horowitz Eliot Horowitz is CTO and Co-Founder of MongoDB. Eliot is one of the core MongoDB kernel committers. Previously, he was Co-Founder and CTO of ShopWiki. Eliot developed the crawling and data extraction algorithm that is the core of its innovative technology. He has quickly become one of Silicon Alley's up and coming entrepreneurs and was selected as one of BusinessWeek's Top 25 Entrepreneurs Under Age 25 nationwide in 2006. Earlier, Eliot was a software developer in the R&D group at DoubleClick (acquired by Google for $3.1 billion). Eliot received a BS in Computer Science from Brown University.

February 10, 2015

Announcing MongoDB 3.0

Today we are announcing MongoDB 3.0. This release marks the beginning of a new phase in which we build on an increasingly mature foundation to deliver a database so powerful, flexible, and easy to manage that it can be the new DBMS standard for any team, in any industry. MongoDB 3.0 brings with it massive improvements to performance and scalability, enabled by comprehensive improvements in the storage layer. We have built in the WiredTiger storage engine, an incredible technology with a distinguished pedigree. WiredTiger was engineered with latch-free, non-blocking algorithms to take advantage of trends in modern hardware, like large on-chip caches and heavily threaded architectures. By drawing on both academic research and their extensive commercial experience, the WiredTiger team built a storage engine that can underpin the next 20 years of data storage applications. With WiredTiger, MongoDB 3.0 introduces document-level concurrency control, so performance remains fast and predictable under concurrent, write-intensive workloads. Transparent on-disk compression reduces storage requirements by up to 80%, and a choice of compression algorithms means that developers can tailor the performance/space trade-off to suit the needs of particular components in their applications. MongoDB's original storage engine has been enhanced, including collection-level concurrency control and more efficient journaling. Now called MMAPv1, the default storage engine is a binary drop in that provides more efficient and concurrent handling of workloads. We are currently running the production infrastructure of MongoDB Management Service on MongoDB 3.0 replica sets, mixing both MMAPv1 and WiredTiger storage engines. This type of flexibility is possible because MongoDB 3.0’s pluggable storage engine API addresses the reality that there is no single storage engine perfect for all applications. Through this architecture, MongoDB provides an easy to use, high-level layer for application development, durability, and horizontal scale, while allowing lower-level storage engines to offer solutions engineered for specialized use cases. The storage engine API means that MongoDB can thrive in any environment, such as in-memory, encrypted, HDFS, or hardware-optimized, to name but a few. In fact, MongoDB can now be used as a laboratory for storage engine development. Delegating higher-level concerns to MongoDB lets storage engine developers focus on their domain of expertise, and lets a massive installation base of MongoDB users take the new engine for a spin just by adding a replica set member. As with all infrastructure, MongoDB needs to be managed. We know this can be a challenge at large scale, so with MongoDB 3.0, we are giving enterprises Ops Manager, an on-premise product that makes MongoDB a first-class citizen of the mature operations ecosystem. Ops Manager makes routine tasks push-button, eliminating the vast majority of work that operations teams must do to manage MongoDB. It doesn't try to replace existing automation or provisioning tools, rather, it supports and supplements them, so ops can focus on architecture, not plumbing. Ops Manager's API means it can be integrated into existing tooling for monitoring, backup and management. MongoDB 3.0 is a landmark release. It brings comprehensive improvements in performance, flexibility, ease of management, and cost to operate. MongoDB has always made it fast and easy to iterate -- from prototype to production to maturity, keeping pace in the most agile environments. Taken as a whole, the changes in 3.0 remove bottlenecks and obviate workarounds, suiting MongoDB to an even wider variety of applications, and providing teams the confidence to run it in more mission critical situations. MongoDB 3.0 is a true contender for the title of "default database" in any organization. This is just the beginning. We will strengthen the case for MongoDB as your default database with every iteration. We will continue to surface advanced features of WiredTiger such as Log Structured Merge Trees. We will continue to push the envelope in data interaction semantics, by implementing a transaction system for the distributed document model. And we will iterate aggressively on Ops Manager, to make sure that operating MongoDB is as effortless as it is to develop with. MongoDB 3.0 will be generally available in March, when we finish putting it through its paces. Stay tuned for our latest release candidate , we would love it if you would try it out and give us feedback. If you’re considering updating your version of MongoDB, take a look at our Major Version Upgrade consulting services: Upgrade with Confidence

February 2, 2015

Renaming Our Upcoming Release to MongoDB 3.0

Something out of the ordinary is happening today: we’re renaming our upcoming MongoDB release to 3.0. Version numbers are usually set well in advance on a product roadmap, and we’ve already put out 6 release candidates under the 2.8 moniker, so I’m sure some eyebrows are being raised right about now. I’m going to explain the reasoning behind this change, starting with a discussion of our versioning scheme. At MongoDB, we use a versioning system composed of three components. Consider the version number 2.6.3. Two of those components are purely technical in their meaning.The rightmost number (3) is a revision number, denoting fixes for issues, which are always backwards compatible. The middle number (6) is a release series identifier, which we increment as we release new features; these may break backwards-compatibility, but not necessarily. That leaves the leftmost number, the ‘2’ in 2.6.3. This number is not a technical designation; it corresponds to an overall phase of MongoDB as a product. In fact, the most accurate description of this number is that we increment it when our community tells us we’re entering a new phase. To date, MongoDB has gone through three phases: 0.x: Science Project. Non-relational databases had little traction, but we were convinced that developers would jump at the chance to build their applications without the contortions required by existing data stores. 1.x: Early Adopters. Fully half of the features now present in MongoDB did not yet exist, and there were many growing pains to endure. For many teams, the trade-off was worth it, but for others, it was not yet ready to meet their needs. 2.x: Mainstream. MongoDB passed enterprise requirements and was deployed to handle massive data processing loads; customers built clusters housing hundreds of Terabytes of data. MongoDB saw adoption beyond the wildest dreams of our Science Project days, and our ever-growing community demanded: “More features! Better performance! Under more workloads! And easier to manage!” We’ve done our best to comply, and throughout, we have tried to build a database that can be used in as many places as our users want to deploy it. Now we're ready to herald a new phase of MongoDB: the go-to database for modern application development. Last summer, we committed to document-level locking, a pluggable storage engine API, and an on-premises version of our automation product known as "Ops Manager". We were well on the way to delivering that feature set, but as ambitious as it was, it would not have been the dawn of a new phase. Then we had a windfall, welcoming the WiredTiger team to MongoDB. Now we are delivering beyond what we expected: our product roadmap is accelerated by the capabilities present in WiredTiger, and by the firepower that that team brings to MongoDB. This potential to leapfrog our prior release plan is game-changing. Feedback on the release candidates from our beta users and community members showed that '2.8' was now a stale reference for something that had become much bigger. "This isn't an incremental release," they said, "it is a step-function release." This is the kind of feedback a team dreams of hearing. We've decided to act on what you're telling us, and give this release a number appropriate for its impact. Our team is hard at work readying MongoDB 3.0. Thank you for your continued support -- we’re looking forward to meeting, and hopefully exceeding, your expectations. To learn more about what's new in MongoDB 3.0, download the white paper below. MongoDB 3.0

January 21, 2015

Welcome WiredTiger to MongoDB

This blog post was updated on January 22, 2014 to reflect the renaming of MongoDB 2.8 to 3.0. Today, we are announcing our acquisition of WiredTiger , an open source, high-performance storage engine company co-founded and led by world-renowned experts Keith Bostic and Dr. Michael Cahill. Keith is a co-founder of Sleepycat Software, where he and Michael both architected Berkeley DB, “the most widely used database toolkit in the world, with hundreds of millions of deployed copies running in everything from routers and browsers to mailers and operating systems.” [1] We posted previously about the inclusion of WiredTiger in the upcoming 3.0 release of MongoDB, and how it will enable users to achieve great concurrency, better utilize the resources of their hardware, improve storage efficiency, and see much better performance under write-intensive workloads. If all we wanted was a great storage engine, we could have stopped there. But WiredTiger is more than great technology, it's also great people. Working together on the integration of WiredTiger and our pluggable storage engine API (another banner feature of MongoDB 3.0), we discovered how aligned we are on mission, vision, and philosophy. It wasn’t long before we realized -- WiredTiger wasn’t just a product we admired, it was a group of people who embody everything we value in our team. I can illustrate how this happened with an anecdote from early on in our association. We first met the WiredTiger team while we were still seeking out a preliminary set of storage engines to pilot the pluggable storage engine API. At this point we were still developing the API, and thinking about the types of storage engines that we want to make available to users. We wanted a flagship engine, one that could offer our users advanced features and performance improvements for write-heavy workloads. It was during this time that we started getting questions from the WiredTiger team about the storage engine API… for the integration that they had already largely built! When we saw what great work the WiredTiger team did on the MongoDB codebase, we knew right away that they were the kind of engineers we'd love to work with. We went up to Boston for an integration summit. Working closely with the WiredTiger team didn’t just reinforce our impressions, it sealed them. In our first, 4 hour long, in-person meeting, we saw eye to eye on every key issue. Communication was easy, we agreed on priorities, and we learned first hand that they are at least as passionate about open source software as we are. As a matter of fact, Keith is legitimately a pioneer of open source software, having led the effort to produce the first complete, freely available BSD Unix -- 4.4BSD-Lite . This pioneering carried on as Keith and Michael developed Berkeley DB as free software at Sleepycat. While these bona fides speak for themselves, our in-person interactions underscored them in a profound way. The WiredTiger team was equally enthusiastic about working with MongoDB, it turned out. To sum up: Keith and Michael are both luminaries in the fields of data storage and transaction management, and they and the entire WiredTiger team are on exactly the same page as we are. They are true paragons of the free software movement, and on their team, 15 year’s experience in the field makes you a relative newcomer. As much as the synergy was obvious from the standpoint of complementary technologies, it was even more obvious from a team composition standpoint. Following the acquisition, WiredTiger will live on as its own project, under active development, and we will drive towards making it the default storage engine for MongoDB 3.2. For those that want to adopt WiredTiger under 3.0, there is a no-downtime migration path. As for MMAPv1, it will continue to see active development and support, even after WiredTiger becomes the default. It will become one of a wide array of options made possible by our pluggable API, and for certain workloads, it will be the best choice. The WiredTiger storage engine is available today in the latest release candidate for MongoDB 3.0 . Have a look at the release notes , which contain instructions for upgrading, as well as for migrating to the WiredTiger storage engine. As we fine-tune the release, we are encouraging members of the community to test the RC through our Bug Hunt , offering prizes to those that identify critical issues. As of this announcement, we’re extending the hunt through the holidays! If you’re ready to test out just how amazing our new storage engine is, we’d love to have your feedback on how it works for your application. [1] Bostic, Keith, and Margo Seltzer. "Berkeley DB." Ed. Amy Brown and Greg Wilson. The Architecture of Open Source Applications. To learn more about upgrading your deployment of MongoDB to 3.0, explore our consulting services Upgrade with Confidence

December 16, 2014