GIANT Stories at MongoDB

MongoDB now released under the Server Side Public License

Eliot Horowitz

Company

MongoDB has created a new software license called the Server Side Public License, or SSPL. The SSPL clarifies the conditions for making MongoDB publicly available as a service, to ensure we can continue to invest in building MongoDB for our users rather than in costly litigation over enforcing the AGPL. All subsequent versions and patch releases to prior versions of MongoDB made after October 16th, 2018 will be issued under the new SSPL.

Software as a service is one of the most significant and far-reaching paradigm shifts the technology industry has ever seen, and the market is moving to adopt it rapidly. This change is motivated by one simple truth: it is inefficient to operate software at scale when you just want to make use of its capabilities.

This should be a time of incredible opportunity for open source. The revenue generated by a service can be a great source of funding for open source projects, far greater than what has historically been available. The reality, however, is that once an open source project becomes interesting, it is too easy for large cloud vendors to capture most of the value while contributing little or nothing back to the community. As a result, smaller companies are understandably unwilling to wager their existence against the strategic interests of the large cloud vendors, and most new software is being written as closed source.

The best current solution for an open source company is to license their software under the AGPL, which requires a management stack used to operate that software as a service to be made available under the terms of the AGPL. This approach was believed to be good enough, as most people understood their obligations to comply with AGPL. However, as AGPL-licensed software like MongoDB has become more popular, organizations like the international cloud providers have begun to test the boundaries of this license. We would prefer to avoid litigation to defend the AGPL but instead devote our time to build great products for the community and our customers.

Since we own 100% of the copyright of MongoDB, one option available to us was to convert our open source license to a closed source license. We chose not to because we fundamentally believe in open source software. We believe an open source approach leads to more valuable, robust, and secure software, and it directly enables a stronger community and better products. We also could have licensed most of the code for the MongoDB server as AGPL while applying a closed license to some critical files. We chose not to do that because we believe that a uniform licensing approach, where all the code in a repository shares a single license, makes it easier to understand the obligations of using that code, leading to a stronger development community.

The community needs an updated open source license that builds on the spirit of the AGPL, but makes explicit the conditions for providing the licensed software as a service.

The SSPL is designed to make sure that companies who do run a publicly available MongoDB (or any software subject to the SSPL) as a service are giving back to the community.

It should be noted that the new license maintains all of the same freedoms the community has always had with MongoDB under the AGPL - they are free to use, review, modify, and redistribute the source code. We based the SSPL on GPL v3, with one new section that sets out an explicit condition for offering the licensed software as a service. We believe that the SSPL meets the standards for an open source license and are working to have it approved by the OSI.

Obviously, this new license helps our business, and that is one of the primary reasons we made this decision, but it is also critical for the MongoDB community. Over the past decade MongoDB has invested approximately $300M in research and development to offer an open source database to everyone, and this change protects our ability to invest aggressively in R&D to drive further innovation and value for the community.

I am convinced that the SSPL will have a positive effect on the open source community by encouraging increased investment into open source server-side software. As always, we are listening closely for your feedback, and we are eager to work with the community to drive towards a more open future.

Resources

16 Cities in 5 Months: The MongoDB team is coming to an AWS Summit near you

As our community of users continues to grow and become more diverse, we want to ensure all of our customers are fully equipped to be successful on MongoDB Atlas. To that end, we have partnered with AWS, committing to 16 of their regional Summits. These 16 events span 13 different countries and expect to draw thousands of members of the AWS and MongoDB communities.

Stack Overflow Research of 100,000 Developers Finds MongoDB is the Most Wanted Database

Michael Lynn

Company

It’s well established that developers want to work with a database that offers flexibility, versatility, and ease of use. Scalability and reliability certainly don’t hurt either. For an increasing number of developers, MongoDB is the desired solution to meet all these requirements. And for the second year in a row, this has been validated by the opinions of those who matter most — developers themselves.

Today, we’re excited to announce that, for the second year in a row, MongoDB is the most wanted database in the Stack Overflow Developer Survey 2018, the world’s largest developer survey with over 100,000 respondents.

Steady Increases in Popularity

Since its debut, MongoDB’s popularity among developers has steadily increased. This popularity has been driven primarily by the platform’s ease of use and flexibility; more recently, the release of the fully managed database service MongoDB Atlas has made it easier than ever to run the database in any of the major cloud platforms.

MongoDB continues to innovate, listening and reacting to the demands of developers from small startups to the largest of enterprises. New and exciting features to the platform include change streams, which push real-time data updates to downstream applications, retryable writes, which enhance reliability without increasing complexity, and the announcement of multi-document ACID transactions, coming in version 4.0 later this year.

Why MongoDB?

MongoDB’s document model offers developers massive increases in productivity by allowing developers to store data in a way that is consistent with how they think and create applications. Additionally, MongoDB’s native replication and horizontal partitioning give developers the confidence and freedom to concentrate on differentiating code without having to be concerned about challenges associated with data locality, reliability and scalability. Lastly, MongoDB offers the freedom to run consistently all the way from local development environments to the largest mainframe deployments. The database can be deployed on-premises, in a hybrid cloud, or in any of the public clouds.

How can you benefit from this news?

Hiring managers seeking talented developers may do well to review some of the additional statistics included in today’s release. One interesting statistic can be found in the chart entitled “How Developers Assess Potential Jobs”. If you are a hiring manager and you’re looking to attract top talent, considering the most wanted database platform might be a way to enhance the attractiveness of your opportunities.

Javascript and Python developers reviewing the Stack Overflow report will be encouraged as these development environments continue to grow in popularity. Popularity of web application stacks like MEAN or MERN have provided Javascript developers with frameworks for building applications quickly.

Not using Javascript or Python? MongoDB offers full support for these and all popular development languages. Want to use Golang with MongoDB? A fully supported MongoDB Golang driver has reached Alpha stage in development, giving even more options to developers to build with the latest and most popular languages.

In summary

Developers are the new Kingmakers and their opinions are vitally important. The results of the Stack Overflow survey are an exciting validation of MongoDB’s efforts to build a resilient, flexible, and easy-to-use platform wanted by developers the world over.


Are you looking to learn more?

MongoDB University offers free courses for beginners, developers, database administrators, and operations personnel. Get online education about MongoDB Atlas, Compass and other products to help developers build great modern applications.

Want to build right away with MongoDB? Get started today with a 512 MB database managed by MongoDB Atlas here.

MongoDB Drops ACID

MongoDB 4.0 will add support for multi-document transactions, making it the only database to combine the speed, flexibility, and power of the document model with ACID data integrity guarantees. Through snapshot isolation, transactions provide a globally consistent view of data, and enforce all-or-nothing execution to maintain data integrity.

Transactions in MongoDB will feel just like transactions developers are familiar with from relational databases. They will be multi-statement, with similar syntax (e.g. start_transaction and commit_transaction), making them familiar to anyone with prior transaction experience. The changes to MongoDB that enable multi-document transactions will not impact performance for workloads that do not require them. In MongoDB 4.0, which will be released this summer*, transactions will work across a single replica set, and MongoDB 4.2* will support transactions across a sharded deployment.

Because documents can bring together related data that would otherwise be modeled across separate parent-child tables in a relational schema, MongoDB’s atomic single-document operations already provide transaction semantics that meet the data integrity needs of the majority of applications. But multi-document transactions will make it easier than ever for developers to address a complete range of use-cases, while for many, simply knowing that they are available will provide critical peace of mind. With MongoDB 4.0, you’ll be able to rely on transactional integrity, regardless of how you model your data.

The imminent arrival of transactions is the culmination of a multi-year engineering effort, beginning over 3 years ago with the integration of the WiredTiger storage engine. We’ve laid the groundwork in almost every part of the server – from the storage layer itself, to the replication consensus protocol, to the sharding architecture. We’ve built out fine-grained consistency and durability guarantees, introduced a global logical clock, refactored cluster metadata management, and more. We’ve also exposed all of these enhancements through APIs that are fully consumable by our drivers. We’re now about 85% of the way through the backlog of features that enable transactions, as this diagram summarizes:

You can read more about our drive to multi-document transactions here. And if you can’t wait to take transactions for a spin, we’d love to have you join our beta program; all the details are at http://mongodb.com/transactions/.


About the Author

Eliot Horowitz is the CTO and Co-Founder of MongoDB. He wrote the core code base for MongoDB starting in 2007, and subsequently built the engineering and product teams. Today, Eliot oversees those teams, and continues to drive technology innovations at MongoDB. Prior to MongoDB, Eliot co-founded and built ShopWiki, a groundbreaking online retail search engine. He built its technology, its team, and presided over its private sale in 2010. Before that, Eliot was a software developer in the R&D group at DoubleClick.

Eliot is on the board of the NY Tech Talent Pipeline. In 2006, he was selected as one of BusinessWeek’s Top 25 Entrepreneurs Under Age 25, and in 2015 was named to the Business Insider “Under 35 and Crushing it” list. He was also recently named to Crain’s NY Business 40 Under 40 Class of 2017 list. Eliot received a B.S. in Computer Science from Brown University.


* Safe Harbour Statement

This post contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward-looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation.

In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.

MongoDB’s Drive to Multi-Document Transactions

Transactions are important. Any database needs to offer transactional guarantees to enforce data integrity. But they don’t all do it in the same way – different database technologies take different approaches:

  • Relational databases model an entity’s data across multiple rows and parent-child tables, and so transactions need to span those rows and tables.
  • With subdocuments and arrays, document databases allow related data to be unified hierarchically inside a single data structure. The document can be updated with an atomic operation, giving it the same data integrity guarantees as a multi-table transaction in a relational database.

Because of this fundamental difference in data modeling, MongoDB’s existing atomicity guarantees are able to meet the data integrity needs of most applications. In fact, we estimate 80%-90% of applications don’t need multi-document transactions at all. However, there are some legitimate use cases and workloads where transactions across multiple documents are needed. In those cases, without transactions, a developer would have to implement complex logic on their own in the application layer. Also, some developers and DBAs have been conditioned by 40 years of relational data modeling to assume multi-table/document transactions are a requirement for any database, irrespective of the data model they are built upon. Others are concerned that while multi-document transactions aren’t needed by their apps today, they might be in the future and they don’t want to outgrow their database.

And so, the addition of multi-document ACID transactions makes it easier than ever for developers to address a complete range of use-cases on MongoDB.

As one can imagine, multi-document transactions are a much more complex thing to build in a distributed database than in a monolithic, scale-up database. In fact, we have been working on bringing multi-document transactions to MongoDB as part of a massive multi-year engineering investment. We have made enhancements to practically every part of the system – the storage layer itself, our replication consensus protocol, sharding architecture, consistency and durability guarantees, the introduction of a global logical clock, and refactored cluster metadata management and more. And we’ve exposed all of these enhancements through APIs that are fully consumable by our drivers.

The figure below represents the evolution of these enhancements as well as the work in progress to enable multi-document transactions. As you can see, we are nearly done.

In MongoDB 4.0, coming in summer 2018*, multi-document transactions will work across a replica set. We will extend support for transactions across a sharded deployment in the following release.

Importantly, the green boxes highlight all of the critical dependencies to transactions that have already been delivered over the past 3 years. And, frankly, that was the hardest part of the project – how to balance building the stepping stones we needed to get to transactions with delivering useful features to our users straightaway to improve their development experience along this journey. Wherever we could, we built components that suited both goals. For example, the introduction of the global logical clock and timestamps in the storage layer enforces consistent time across every operation in a distributed cluster. These enhancements are needed for transactions in order to provide snapshot isolation, but they also allowed us to implement change stream resumability and causal consistency in MongoDB 3.6, which are immediately valuable on their own. Change streams enable developers to build reactive applications that can view, filter, and act on data changes as they occur in the database in real-time, and recover from transient failures. Causal consistency allows developers to maintain the benefits of strong data consistency with “read your own write” guarantees, while taking advantage of scalability and availability of our intelligent distributed data platform.

The global logical clock is just one example. A selection of other key enhancements along the way illustrates how our engineering team deliberately laid the groundwork for transactions in such a way that we consistently surfaced additional benefits to our users:

  • The acquisition of WiredTiger Inc. and integration of its storage engine way back in MongoDB 3.0 brought massive scalability gains with document level concurrency control and compression to MongoDB. And with MVCC support, it also provided the storage layer foundations for transactions coming in MongoDB 4.0.
  • In MongoDB 3.2, the enhanced consensus protocol allowed for faster and more deterministic recovery from the failure or network partition of the primary replica set member, along with stricter durability guarantees for writes. These enhancements were immediately useful to MongoDB users then, and they are also essential capabilities for transactions.
  • The introduction of readConcern in 3.2 allowed applications to specify read isolation level on a per operation basis, providing powerful and granular consistency controls.
  • Logical sessions in MongoDB 3.6 gave our users causal consistency and retryable writes, but as a foundation for transactions, they provide MongoDB the ability to coordinate client and server operations across the nodes of a distributed cluster, managing the execution context for each statement in a transaction.
  • Similarly, retryable writes, implemented in MongoDB 3.6, simplify the development of applications in the face of elections (or other transient failures) while the server enforces at most once processing semantics.
  • Replica set point in time reads in 4.0 are essential for transactional consistency, but it’s also highly valuable to regular read operations that don’t need to be executed in a transaction. With this feature, reads will only show a view of the data that is consistent at the point the find() operation starts, irrespective of which replica serves the read, or what data has been modified by concurrent operations.

The number of remaining pieces on the roadmap to transactions is small. Once complete, multi-document distributed transactions will provide a globally consistent view of data (both in replica set and sharded deployments) through snapshot isolation and maintain all-or-nothing guarantees in cases of node failures. This will greatly simplify your application code. After all, MongoDB’s job is to take hard problems and solve them for as many developers as possible, so that you can focus on adding value to your applications and not dealing with the underlying plumbing.

We’re really excited about the release of multi-document transactions, and what they will allow you to build with MongoDB going forward. You should view our multi-document transactions page to learn more, and we invite you to sign up for the beta program so that you can start to put all of the work we’ve done through its paces.


* Safe Harbour Statement

This post contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward-looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation.

In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.

Stack Overflow & MongoDB Research Unveils Developer Productivity Struggles

Andrew Morgan

Company

As part of our mission to help developers build better apps faster by providing a database platform that doesn’t hold them back, we are always looking for new ways to better understand the challenges devs face.

We recently undertook some research with Stack Overflow, the largest, most trusted online community for developers, and the findings confirmed a lot of what we already knew and believed, such as developers’ critical role in driving innovation and growth in every type of organization. They also unveiled some troubling disconnects that still exist between business leaders and their developers.

The research, which surveyed more than 1,000 developers worldwide, revealed that despite developers being seen as key drivers of enterprise innovation – with 68% of respondents viewing developers as the primary drivers – companies are failing to capitalize on their skills and abilities. Many (33%) believe their companies aren't gaining a competitive advantage because they don’t understand the technical realities and opportunities that developers face.

What’s holding back developers?

Findings revealed three main challenges developers are dealing with that hold them back from fully delivering as innovators:

  1. Spending too much time in the wrong places: 41% of a developer’s working day goes towards the upkeep of infrastructure, instead of innovation or bringing new products to market. Al fifth of their time (20%) is spent in meetings or administration – yawn!
  2. Current job demands are heavy: 58% of developers work more than eight hours a day, 32% work weekends, and 23% fail to take all of their vacation days. The previous point plays a part in this, with developers working extra hours to catch up after wasting time on managing their infrastructure.
  3. Developer’s dilemma: Today’s application users flat out expect applications to integrate with other services to provide a richer, more complete experience. That could be the ability to make payments through Stripe or PayPal, or to log in faster with Facebook ID and send text messages from within an app via Twilio. But the monotonous, backend coding that is required to provide this improved user experience is taking the focus away from the frontend experience: 41% of a developers' time building a new application is spent connecting to backend services, rather than on developing features to makes their applications unique.
How developers spend their time each week

It was with these challenges in mind that we launched MongoDB Stitch. As developers ourselves, we recognize that web, mobile, and IoT apps today almost always include a combination of services like authentication systems, payment apps, messaging platforms, or internal microservices.

Stitch lets developers focus on building applications rather than on managing data manipulation code, service integration, or backend infrastructure. Whether you’re just starting up and want a fully managed backend as a service, or you’re part of an enterprise and want to expose existing MongoDB data to new applications, Stitch lets you focus on building the app users want, not on writing boilerplate backend logic.

56% of developers believe that developers are the main drivers of innovation within organizations today

The cloud: challenge or opportunity?

The survey also asked developers about their experiences with the cloud, resulting in some interesting insights:

  • Cloud wars: Nearly half of the developers surveyed (42%) say security regulations at their company are the primary barrier to furthering cloud adoption within the organization. Amazon Web Services (39%) is currently ahead of Microsoft Azure (19%) and Google Cloud Platform (13%) when it comes to where companies host their apps.
  • Reality check on cloud migration: In regards to migrating services to the cloud, nearly half of the developers surveyed (43%) said they currently used cloud services and had expansion plans. However, there are still considerable barriers to cloud adoption, with nearly half of those surveyed (42%) noting that security, privacy, or regulatory concerns were all hindrances to cloud migration. On top of this, less than half of today’s apps (45%) are developed primarily for the cloud.
  • Cost and productivity benefits of Cloud-hosted DBaaS are well understood: Well over half (60%) of the developers surveyed fully understand the cost benefits of cloud-hosted Database as a Service (DBaaS) in that you pay for what you use. Half (50%) noted that DBaaS results in increased developer productivity and a faster time to market.

MongoDB Atlas is MongoDB’s answer to these cloud concerns. Fully-managed with best-in-class security, Atlas is the easiest and most cost-effective way to get started building great applications on MongoDB. We take care of the operations and security best practices and let developers focus on driving innovation.

Give the devs what they want

The rise of the developer within organizations has been an ongoing trend for years, and business leaders are increasingly recognizing the value that developers provide in keeping their companies competitive.

But to truly maximize the innovation that developers can provide, software developers will need the right technology in place – technology that is intuitive and natural to work with instead of creating additional speed bumps in the development process. For developers to truly take their rightful place as Kingmakers of the enterprise, business leaders will need to continue to grow their understanding of how to best arm their developer teams to drive a competitive edge in a software-centric world.


About the author - Andrew Morgan

Andrew is part of the MongoDB product team, responsible for building the vision, positioning, and content for MongoDB’s products and services, including the analysis of market trends and customer requirements. Before joining MongoDB, Andrew was director of product management for MySQL at Oracle – with a particular focus on distributed, highly available databases. Prior to Oracle, Andrew worked in software development for telecoms with a focus on HA, in-memory, real-time databases.

Internal Mobility: Changing Roles, not Companies

Andrea Dooley

Company, Culture

What’s the first thing you would do if you started to feel a lack of fulfillment or challenge in your current role? You might begin investigating opportunities at other companies. Perhaps you’d start researching your current organization’s competitors, or the company your colleague left for last quarter, or change your LinkedIn status so recruiters know you’re willing to having a discussion.

Why do employers let their best talent get to the point where they enter the rabbit hole of potential opportunity, instead of finding a way to retain them?

Internal mobility, or the opportunity to move teams and roles within one organization, is still not as feasible an option as some may like it to be. In fact, according to a recent Gartner study, 61% of employees think it would be beneficial for their careers to work in different parts of the organization, and future career opportunity is consistently one of the top reasons employees leave.

Tenured employees have already proven themselves, so you can assume they’re able to take on bigger projects and encourage greater collaboration between departments. Over the last few years, we as an organization acted on this conjecture, providing opportunities for our employees to develop new skills.

Jay Gordon is a MongoDB Developer Advocate and a major asset in providing information and assistance regarding all things MongoDB. He enables our developer community by creating technical content, answering questions, attending events, and providing as much assistance as possible to those using (or learning to use) MongoDB.

Jay was hired in 2016 as a Technical Account Manager on the Cloud Team to provide MongoDB customers with onboarding and integration assistance. While tackling his daily responsibilities, Jay became involved in helping the MongoDB user community understand new releases, products, and tools, acting as an advocate and evangelist. A little over a year into his role as a TAM, a Developer Advocate role opened up and Jay was the perfect fit.

“It was a difficult transition because I was taking on an entirely new role. There was some self-doubt that I was really capable of the position I had taken on, but in the end, much of that was just worry for the sake of worry. The MongoDB team was by my side to help with the shift, which had a lot to do with why I stayed instead of looking elsewhere for an opportunity, and the culture had a lot to do with my decision. Being part of MongoDB is a special experience and I was not ready to walk away from that quite yet. I love my job, I love the people I work with, and I love the opportunities to be part of something huge.”

Angshuman Bagchi joined the MongoDB team in 2013 as a Technical Service Engineer in Palo Alto. For over three years he worked on front line customer support helping customers become successful with MongoDB, but once he was struck with a desire to do something different, Angshuman went back to his development roots as a lead on our Technical Services Tools team, which is responsible for all tools used by our Technical Services Engineers. The role not only allowed him to do something different, it also provided the opportunity to learn managerial and administrative skills.

“My management was very supportive of my desire for a change. They gave me the freedom to explore opportunities both outside and within the broader Technical Services organization with the emphasis that I should be allowed to work on something I want. The move has afforded me a tremendous amount of personal and professional growth.”

Marie Vito is now the Program Lead and Coordinator for our award-winning Sales Enablement team – responsible for training, process implementation, tools, and coaching to make sales representatives and leaders the most productive they can be in their roles. Marie plans and leads the monthly MongoDB Sales Bootcamp training for all sales new hires, as well as anyone else in the company interested in learning about our sales strategy.

When she first joined MongoDB as a Recruiting Coordinator, Marie supported our sales organization by scheduling, coordinating, and facilitating interviews for all sales-related roles. During that time, she was able to form strong relationships with the sales team and quickly became interested in their goals and initiatives.

An initial conversation with the Senior Director of Sales Enablement helped to flesh out the role and determine that it was a position Marie wanted to pursue.

“Following that meeting I spoke with my then manager to express my aspirations. Once it was determined I was a good fit for the role, we developed a transition plan to ensure I would be leaving the recruiting team with a more than capable replacement, while still dedicating enough time to enablement training to make the ramp process as efficient as possible. It was incredible to have so much support from my new team, old team, and everyone else in the company.”

Providing employees the opportunity to try new things, explore new roles, and broaden their skills is a great way to foster both professional and personal growth. At MongoDB, we’ve found that supporting internal mobility is a key factor in retaining our best people and keeping them content.

Interested in learning more about what we’re doing at MongoDB? Click here.

What’s New in MongoDB 3.6. Part 1 – Speed to Develop

Mat Keep

Company, MongoDB 3.6

MongoDB 3.6 is now Generally Available (GA), and ready for production deployment. In this short blog series, I’ll be taking you on a whirlwind tour of what’s new in this latest release:

  • Today, we’ll take a look at the new capabilities designed specifically to help developers build apps faster. We’ll take a look at change streams, retryable writes, developer tools, and fully expressive array manipulation
  • In part 2, we’ll dive into the world of DevOps and distributed systems management, exploring Ops Manager, schema governance, and compression
  • Part 3 will cover what’s new for developers, data scientists, and business analysts with the new SQL-based Connector for BI, richer in-database analytics and aggregations, and the new recommended driver for R
  • In our final part 4, we’ll look at all of the new goodness in our MongoDB Atlas fully managed database service available on AWS, Azure, and GCP, including cross-region replication for globally distributed clusters, auto-scaling, and more.

If you want to get the detail now on everything the new release offers, download the Guide to what’s New in MongoDB 3.6.

Developer-First

MongoDB has always been a developer-first technology. Its document data model maps naturally to objects in application code, making it simple for developers to learn and use. A document’s schema can be dynamically created and modified without downtime, making it fast to build and evolve applications. Native, idiomatic drivers are provided for 10+ languages – and the community has built dozens more – enabling ad-hoc queries, real-time aggregation and rich indexing to provide powerful programmatic ways to access and analyze data of any structure.

MongoDB 3.6 builds upon these core capabilities to allow developers to create rich apps and customer experiences, all with less code.

Change Streams

Change streams enable developers to build reactive, real-time, web, mobile, and IoT apps that can view, filter, and act on data changes as they occur in the database. Change streams enable seamless data movement across distributed database and application estates, making it simple to stream data changes and trigger actions wherever they are needed, using a fully reactive programming style.

Implemented as an API on top of MongoDB’s operation log (oplog), consumers can open change streams against collections and filter on relevant events using the $match, $project, and $redact aggregation pipeline stages. The application can register for notifications whenever a document or collection is modified, enabling downstream applications and consumers to act on new data in real time, without constantly querying the entire collection to identify changes. Applications can consume change streams directly, via a message queue, or through a backend service such as MongoDB Stitch (coming soon).

Use cases enabled by MongoDB change streams include:

  • Powering trading applications that need to be updated in real time as stock prices rise and fall.
  • Synchronizing updates across serverless and microservices architectures by triggering an API call when a document is inserted or modified. For example, new customer orders written to the database may automatically trigger functions to generate invoices and delivery schedules.
  • Updating dashboards, analytics systems, and search engines as operational data changes.
  • Creating powerful IoT data pipelines that can react whenever the state of physical objects change. For example, generating alarms whenever a connected vehicle moves outside of a geo-fenced area.
  • Pushing new credit card transactions into machine learning training models to re-score fraud classifications.
  • Refreshing scoreboards in multiplayer games.

MongoDB change streams enable consumers to react to data changes in real time Figure 1: MongoDB change streams enable consumers to react to data changes in real time

Some MongoDB users requiring real-time notifications have built their own change data capture processes that “tail” the oplog. By migrating to change streams, these users can reduce development and operational overhead, improve usability, and increase data reliability. When compared to both oplog tailing and change notifications implemented by alternative databases, MongoDB change streams offer a number of advantages:

  • Change streams are flexible – users can register to receive just the individual deltas from changes to a document, or receive a copy of the full document.
  • Change streams are consistent – by utilizing a global logical clock, change streams ensure a total ordering of event notifications across shards. As a result, MongoDB guarantees the order of changes will be preserved, and can be safely processed by the consuming application in the order received from the stream.
  • Change streams are secure – users are able to create change streams only on collections to which they have been granted read access.
  • Change streams are reliable – notifications are only sent on majority committed write operations, and are durable when nodes or the network fails.
  • Change streams are resumable – when nodes recover after a failure, change streams can be automatically resumed, assuming that the last event received by the application has not rolled off the oplog.
  • Change streams are familiar – the API syntax takes advantage of the established MongoDB drivers and query language, and are independent of the underlying oplog format.
  • Change streams are highly concurrent – up to 1,000 change streams can be opened against each MongoDB instance with minimal performance degradation.

Review the MongoDB change streams documentation to learn more.

Retryable Writes

The addition of retryable writes to MongoDB moves the complexity of handling temporary system failures from the application to the database. Now, rather than the developer having to implement custom, client-side code, the MongoDB driver can automatically retry writes in the event of transient network failures or a primary replica election, while the MongoDB server enforces exactly-once processing semantics.

By assigning a unique transaction identifier to each write operation, the driver re-sends that ID to enable the server to evaluate success of the previous write attempt, or retry the write operation as needed. This implementation of retryable writes offers a number of benefits over approaches taken by other databases:

  • Retryable writes are not limited to idempotent operations only. They can also be applied to operations such as incrementing or decrementing a counter, or processing orders against stock inventory.
  • Retryable writes are safe for operations that failed to acknowledge success back to the application due to timeout exceptions, for example due to a transient network failure.
  • Retryable writes do not require developers to add any extra code to their applications, such as retry logic or savepoints.

Applications that cannot afford any loss of write availability, such as e-commerce applications, trading exchanges, and IoT sensor data ingestion, immediately benefit from retryable writes. When coupled with self-healing node recovery – typically within 2-seconds or less – MongoDB’s retryable writes enable developers to deliver always-on, global availability of write operations, without the risks of data loss and stale reads imposed by eventually consistent, multi-master systems.

Tunable Consistency

With tunable consistency, MongoDB affords developers precise control over routing queries across a distributed cluster, balancing data consistency guarantees with performance requirements. MongoDB 3.4 added linearizable reads, which were central to MongoDB passing Jepsen – some of the most stringent data safety and correctness tests in the database industry.

Now the MongoDB 3.6 release introduces support for causal consistency – guaranteeing that every read operation within a client session will always see the previous write operation, regardless of which replica is serving the request. By enforcing strict, causal ordering of operations within a session, causal consistency ensures every read is always logically consistent, enabling monotonic reads from a distributed system – guarantees that cannot be met by most multi-node databases.

Causal consistency allows developers to maintain the benefits of strict data consistency enforced by legacy single node relational databases, while modernizing their infrastructure to take advantage of the scalability and availability benefits of modern distributed data platforms.

Developer Tooling: MongoDB Compass

As the GUI for MongoDB, Compass has become an indispensable tool for developers and DBAs, enabling graphical schema discovery and query optimization. Compass now offers several new features:

  • Auto-complete: Enables developers to simplify query development with Compass providing suggestions for field names and MongoDB operators, in addition to matching braces and quotes as they code.
  • Query History: Allows developers to re-run their most recently executed queries, and save common queries to run on-demand.
  • Table View: Now developers can view documents as conventional tables, as well as JSON documents.

MongoDB Compass is not just a single tool – it’s a framework built to allow for the addition of modular components. Compass now exposes this as the Compass Plugin Framework, making Compass extensible by any user with the same methods used by MongoDB’s software engineers. Using the plugin API, users can build plugins to add new features to Compass. Examples include a GridFS viewer, a sample data generator, a hardware stats viewer, a log collector/analyzer, and more.

You can learn more about these new features in the MongoDB Compass documentation.

MongoDB Compass Community

With the MongoDB 3.6 release, the Compass family has expanded to now include the new, no-cost Compass Community edition.

Compass Community provides developers an intuitive visual interface to use alongside the MongoDB shell. It includes the core features of Compass, enabling users to review the hierarchy and size of databases and collections, inspect documents, and insert / update / delete documents. Developers can use the GUI to build queries, examine how they’re executed, and add or drop indexes to improve performance. Compass Community also supports the latest Compass functionality available with MongoDB 3.6, making developers even more productive.

MongoDB Compass Community Figure 2: MongoDB Compass Community, new no-cost GUI for MongoDB developers

MongoDB Compass Community is available from the MongoDB download center.

Fully Expressive Array Updates

Arrays are a powerful construct in MongoDB’s document data model, allowing developers to represent complex objects in a single document that can be efficiently retrieved in one call to the database. Before MongoDB 3.6, however, it was only possible to atomically update the first matching array element in a single update command.

With fully expressive array updates, developers can now perform complex array manipulations against matching elements of an array – including elements embedded in nested arrays – all in a single atomic update operation. MongoDB 3.6 adds a new arrayFilters option, allowing the update to specify which elements to modify in the array field. This enhancement allows even more flexibility in data modeling. It also delivers higher performance than alternative databases supporting JSON data as entire documents do not need to be rewritten when only selective array elements are updated.

Learn more from the array update documentation.

Next Steps

That wraps up the first part of our what’s new blog series. Remember, if you want to get the detail now on everything the new release offers, download the Guide to what’s New in MongoDB 3.6.

Alternatively, if you’d had enough of reading about it and want to get started now, then:

Accenture Insights Platform now offering MongoDB

Alan Chhabra

Company

For years, Accenture and MongoDB have been committed to enabling organizations to leverage the power of data to gain a competitive edge. Today, we are excited to announce the availability of MongoDB Enterprise Server as part of the Accenture Insights Platform (AIP).

Announcing the General Availability of MongoDB 3.6

We announced MongoDB 3.6 in November. Following great community feedback on the 3.6 release candidates, we’re happy to say that 3.6 is now generally available and ready for production deployments. You can download the community version and MongoDB Enterprise Server today.

MongoDB 3.6 is also available on MongoDB Atlas, so you can try out 3.6 or upgrade your existing Atlas clusters to 3.6.

MongoDB 3.6 makes it easier than ever to work with data in the most natural, efficient, and frictionless way possible. In short, MongoDB helps you go faster when building and scaling apps. Key 3.6 features include:

Change streams enable you to build reactive web, mobile and IoT applications that can view, filter, and act on data changes as they occur in the database. Whenever data is changed in MongoDB, downstream systems are automatically notified of the updates in real time. Change streams provide an easy and efficient way to build reactive, event driven apps.

Retryable writes move the complexity of handling transient systems failures from the application to the database. Instead of you having to implement masses of custom, client-side code, MongoDB automatically retries write operations using exactly-once semantics.

With Schema validation, using syntax derived from the proposed IETF JSON Schema standard, we’ve extended the document validation capabilities originally introduced in MongoDB 3.2. Now, DevOps and DBA teams can define a prescribed document structure for each collection, down to the level of individual fields within nested arrays. And you’re able to tune this as you need: lock the schema down, open it up, apply it to a subset of fields – whatever you need for each app or stage of your project.

Binding to localhost by default: with MongoDB 3.6 all MongoDB packages across all platforms refuse all external connections to the database unless explicitly configured otherwise by the administrator. Combined with new IP whitelisting support, administrators can configure MongoDB to only accept external connections on approved IP addresses. These enhancements greatly reduce the risk of unsecured MongoDB instances unintentionally being deployed into production.

Aggregation enhancements support more expressive queries, giving you faster access to data-driven insights. MongoDB’s document data model allows you to model entities in the same way you represent them in code - as complete objects - so you don't need to worry about JOINs. But for analytics it’s useful to join data across multiple collections. We introduced left outer equijoins in MongoDB 3.2, but now we are expanding this with a more powerful $lookup operator to support the equivalent of SQL subqueries and non-equijoins. MongoDB's Connector for BI, which enables MongoDB to be used as a data source in SQL-based analytics and data visualization tools, takes advantage of these enhancements to deliver higher performance, with more analytic operations pushed natively to the database.

MongoDB Atlas is the best way to run MongoDB in the public cloud. MongoDB 3.6 is available as a fully managed service on Atlas, including important new features to support global applications, and with automated scalability and performance optimizations.

Cross-region replication allows Atlas clusters to span multiple cloud provider regions, maintaining continuous availability in the event of geographic outages, and providing optimal customer experience by distributing data closer to users. Atlas now also supports automatic scaling for storage associated with a cluster, making it easier for you to manage capacity. The new performance advisor continuously highlights slow-running queries and provides intelligent index recommendations to improve performance.

Community Contribution

We’d like to acknowledge these users who contributed to this release: adrien petel, aftab muhammed khan, Andrew, atish, Ben Calev, Bodenhaltung, Christian, Curtis Hovey, daniel moqvist, Dawid Esterhuizen, Denis Shkirya, Deyoung Hong, Dmitri Shubin, Edik Mkoyan, Eugene Ivanov, Evan Broder, Gian Maria Ricci, hotdog929, Igor Canadi, Igor Solodovnikov, Igor Wiedler, Ivy Wang, James Reitz, Jelle van der Waa, Jim Van Fleet, Joe, KarenHuang, kurevo18, Marek Skalický, Markus Penttil, Matt Berry, may, Meni Livne, Michael Coleman, Michael Liu, Mike Zraly, Renaud Allard, Richard Hutta, Rob Clancy, ryankall, Sergey, Steven Green, Thales Ceolin, Tom Leibu, Ultrabug, Wayne Wang, and Yuriy Solodkyy.

Thank you! Please keep the feedback coming.

Learn More

With over a hundred new and updated features, MongoDB 3.6 is our biggest release yet. Download the Guide to What’s New in MongoDB 3.6 to learn more.

Or, to get started now:

Download MongoDB 3.6 Enterprise
Download MongoDB 3.6 Community