GIANT Stories at MongoDB

New to MongoDB Atlas — Full CRUD Support in Data Explorer

As a fully managed database service, MongoDB Atlas makes life simpler for anyone interacting with MongoDB, whether you’re deploying a cluster on demand, restoring a snapshot, evaluating real-time performance metrics, or inspecting data.

Today, we’re taking it one step further by allowing developers to manipulate their data right from within the Atlas UI. The embedded Data Explorer, which has historically allowed you to run queries, view metadata regarding your deployments, and retrieve information such as index usage statistics, now supports full CRUD functionality.

To support these capabilities, new Project-level roles with different permission levels have been added.

You can assign users these new roles in the Users and Teams settings.

In addition, all Data Explorer operations are tracked and presented in the Atlas Activity Feed (found in the Alerts menu for each Project), allowing you to see who did what, and when.

When you click into the Data Explorer in Atlas, you should see new controls for interacting with your documents, collections, databases, and indexes. For example, modify existing documents using the intuitive visual editor, or insert new documents and clone or delete existing ones in just a few clicks. A comprehensive list of available Data Explorer operations can be found in the Atlas documentation.

The Data Explorer is currently available for M10 Atlas clusters and higher.

2018 MongoDB Innovation Award Winners

We received an overwhelming number of nominations for the fifth annual MongoDB Innovation Awards, recognizing companies who are using MongoDB to dream big and deliver incredibly bold, innovative solutions that are moving forward industries and changing lives for the better.

We are thrilled to announce our 12 winners who will be honored at MongoDB World, New York, June 26 and 27.

See the full list and read a bit more about how they are disrupting the status quo here:

Global Go to Market Partner of the Year: Accenture

Accenture is a leading global professional services company, providing a broad range of services and solutions strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions, Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. The company partners with more than three-quarters of the Fortune Global 500, driving innovation to improve the way the world works and lives. Accenture and MongoDB have worked together to help organizations leverage the power of data to gain a competitive edge.



The Enterprise: Charles Schwab

Charles Schwab is one of the largest financial services firms in the United States. To improve customer experience, speed up development cycles, and prepare for cloud portability, Charles Schwab is modernizing a significant portion of its applications by migrating to MongoDB-powered microservices. Multiple applications are built on MongoDB, including an authentication app leveraged by retail customers as well as a portfolio management solution utilized by registered investment advisors.



Launch Fast: Coinbase

Coinbase is dedicated to creating an open financial system for the world and defining what the future of finance will look like. To do this, they built the most trusted and regulatory compliant global cryptocurrency trading platform to broker exchanges of Bitcoin, Bitcoin Cash, Ethereum and Litecoin as well as pioneering cryptocurrency indexes and institutional cryptocurrency trading. In 2017, they experienced exponential growth with over 20M+ users and $150B+ being traded on their platform in over 190 countries. The Coinbase engineering team scaled and optimized MongoDB to respond to this unprecedented volume of traffic and to prepare for future waves of cryptocurrency enthusiasm.



Scale: Epic Games

Epic Games develops cutting-edge games and cross-platform gaming engine technology. Their massively popular, multi-platform game, Fortnite, has been played by more than 125 million gamers around the globe. The Epic team has implemented a number of best practices and performance improvements to get the best scaling and availability characteristics out of MongoDB.



Data-Driven Business: Freddie Mac

Freddie Mac set out to modernize a number of applications that were previously built on legacy relational databases. One mission-critical application, a property appraisal tool, held massive amounts of property and loan information, but was increasingly expensive and time consuming to update. Turning to MongoDB, Freddie Mac was able to collect information from a variety of different sources in a variety of formats to build a single view of all the information needed to accurately appraise a property. In the months since using MongoDB, Freddie Mac has seen an increase in developer productivity.



Customer Experience: Fresenius Medical Care North America

Fresenius Medical Care North America is the premier health care company focused on providing the highest quality care to people with renal and other chronic conditions. Through its industry-leading network of dialysis facilities, outpatient cardiac and vascular labs, and urgent care centers Fresenius Medical Care North America (FMCNA) provides coordinated health care services at pivotal care points for hundreds of thousands of chronically ill customers throughout the continent.

Since 2015, FMCNA has used MongoDB Enterprise Advanced for a variety of projects to help support their mission to deliver superior care that improves the quality of life of every patient. These projects have included analytics platforms, a data lake and the FHIR platform (a healthcare standard for exchanging medical records securely and at scale). However, the most impactful application has been a single view of the patient platform built on MongoDB. This platform brings together a variety of data sources to ensure the patient, doctors and other caregivers all have a complete understanding of the treatments required and can make adjustments with confidence.



Healthcare: Genomics England

Genomics England, a company owned by the UK government's Department of Health and Social Care, is working with the NHS to sequence 100,000 genomes from patients with rare diseases and their families, as well as patients with common cancer. In the future, there may be a diagnosis where there wasn't one before and, in time, there is the potential of new and more effective personalized treatments for patients.

On average, 1,000 genomes are sequenced per week, which amounts to around 10 terabytes of data per day. To manage this immense and sensitive data set as well as power the data science that makes it all possible, Genomics England used MongoDB Enterprise Advanced. The partnership with MongoDB allows the processing time for complex queries to be reduced from hours to milliseconds, which means scientists can discover new insights more quickly.



Internet of Things: Humana Inc.

With a variety of applications built on MongoDB, Humana is changing healthcare for the better. One of their IoT applications called Go365 is a corporate wellness and rewards program which helps employees live healthier lives, which in turn increases productivity and reduces overall health claims costs for employers. Go365 features a personalized program that inspires, supports, and rewards members for taking steps to improve and continue healthy behavior. Users are able to compete in challenges, connect their fitness devices and mobile apps to log healthy activities and earn points, reward themselves through the Go365 Mall, and track their progress. In fact, by year 3, people who engaged with the program saw that the cost of their health claims were reduced by over 10%, relative to those of unengaged members.



Delivery Partner of the Year: Infosys

A perennial winner, this is the third year in a row Infosys has won a MongoDB Innovation Award. As a global leader in consulting, technology and next-generation services, this year Infosys has been working closely with MongoDB to accelerate application modernization for client organizations. A key part of this is the joint delivery of single view and mainframe modernization offerings to migrate and digitize business-critical applications away from rigid tabular databases and on to next-generation technology. In this long standing partnership, Infosys and MongoDB are already helping many large enterprises with renewing and modernizing their IT landscape.



The William Zola Award for Community Excellence: Ken W. Alger

Ken Alger is one of the most prolific bloggers on MongoDB's technology with dozens of posts in the past two years. He is a self-taught programmer and a teacher at Treehouse. An avid follower of open-source, he has previously sat on the board of directors of the Django Software Foundation. He is delighted to share his extensive MongoDB knowledge via his blog, Twitter, and his GitHub account. He exemplifies the true community spirit of MongoDB and The William Zola Award for Community Excellence.



Savvy Startup: Radar

Radar, a seed-stage startup and member of the MongoDB Startup Accelerator program, has built iOS and Android SDKs on MongoDB Atlas and AWS. As the location platform for modern apps, Radar allows developers to easily add location context and tracking to their applications. Radar currently runs on more than 25 million devices around the globe, processing billions of locations each week.



Atlas: 7-Eleven

7-Eleven is continuing to redefine what convenience is. By leveraging MongoDB Atlas on AWS and a microservices architecture, 7-Eleven has built an e-commerce application called 7-Now which allows consumers to browse a product catalog connected to their local store’s inventory, make purchases on their mobile phones, and schedule in-store pick up or delivery through services like Postmates. This application not only streamlines the consumer’s experience, but also gives the 7-Eleven team extensive analytics capabilities allowing them to improve the overall customer experience. This is sure to have a major impact in their 10,000 stores in the US and Canada, and with 60% of the US population living within one mile of a 7-Eleven.

Scaling Your Replica Set: Non-Blocking Secondary Reads in MongoDB 4.0

MongoDB 4.0 adds the ability to read from secondaries while replication is simultaneously processing writes. To see why this is new and important let's look at secondary read behavior in versions prior to 4.0.

Background

From the outset MongoDB has been designed so that when you have sequences of writes on the primary, each of the secondary nodes must show the writes in the same order. If you change field "A" in a document and then change field "B", it is not possible to see that document with changed field "B" and not changed field "A". Eventually consistent systems allow you to see it, but MongoDB does not, and never has.

On secondary nodes, we apply writes in batches, because applying them sequentially would likely cause secondaries to fall behind the primary. When writes are applied in batches, we must block reads so that applications cannot see data applied in the "wrong" order. This is why when reading from secondaries, periodically the readers have to wait for replication batches to be applied. The heavier the write load, the more likely that your secondary reads will have these occasional "pauses", impacting your latency metrics. Given that applications frequently use secondary reads to reduce the latency of queries (for example when they use "nearest" readPreference) having to wait for replication batches to be applied defeats the goal of getting lowest latency on your reads.

In addition to readers having to wait for replication batch writes to finish, the writing of batches needs a lock that requires all reads to complete before it can be taken. That means that in the presence of high number of reads, the replication writes can start lagging – an issue that is compounded when chain replication is enabled.

What was our goal in MongoDB 4.0?

Our goal was to allow reads during oplog application to decrease read latency and secondary lag, and increase maximum throughput of the replica set. For replica sets with a high write load, not having to wait for readers between applying oplog batches allows for lower lag and quicker confirmation of majority writes, resulting in less cache pressure on the primary and better performance overall.

How did we do it?

Starting with MongoDB 4.0 we took advantage of the fact that we implemented support for timestamps in the storage engine, which allows transactions to get a consistent view of data at a specific "cluster time". For more details about this see the video: WiredTiger timestamps.

Secondary reads can now also take advantage of the snapshots, by reading from the latest consistent snapshot prior to the current replication batch that's being applied. Reading from that snapshot guarantees a consistent view of the data, and since applying current replication batch doesn't change these earlier records, we can now relax the replication lock and allow all these secondary reads at the same time the writes are happening.

How much difference does this make?

A lot! The range of performance improvements for throughput could range from none (if you were not impacted by the replication lock - that is your write load is relatively low) to 2X.

Most importantly, this improves latency for secondary reads – for those who use readPreference "nearest" because they want to reduce latency from the application to the database – this feature means their latency in the database will also be as low as possible. We saw significant improvement in 95 and 99th percentile latency in these tests.

Thread levels 8 16 32 64
Feature off 1 2 3 5
Feature on 0 1 1 0

95th percentile read latency (ms)

Best part of this new feature? You don't need to do anything to enable it or opt-into it. All secondary reads in 4.0 will read from snapshot without waiting for replication writes.

This is just one of a number of great new features coming in MongoDB 4.0. Take a look at our blog on the 4.0 release candidate to learn more. And don’t forget, you’ve still got time to register for MongoDB World where you can meet with the engineers who are building all of these great new features.

Introducing the MongoDB Masters Program for 2018

My name is Michael Lynn and I’m the Worldwide Director of Developer Advocacy at MongoDB. I’m incredibly proud to be a part of the Developer Relations and Marketing team here at MongoDB.

A majority of what we do in Developer Advocacy is related to increasing awareness of MongoDB within the community of developers and data scientists. We do this through involvement in a variety of user groups, industry conferences, and events as well as through management of the MongoDB Masters Program.

This program was created to recognize leaders within their community, experts in MongoDB, and professionals who freely share their knowledge. This year’s class includes returning Masters, as well as new members who have distinguished themselves in the past year.

MongoDB Masters in years past have provided valuable product feedback and driven thought leadership in their fields. We look forward to deepening this relationship over the coming year. This year’s class of Masters will also be encouraged to participate in beta testing programs, share their experiences with MongoDB, and continue to expand and broaden their own voices as leaders in the technical community.

The Masters program has been an incredibly rewarding and valuable program for MongoDB and we greatly appreciate the efforts of our most vocal, and most active supporters. This is why we’ve put so much time and effort into creating a program to recognize these individuals and thank them for their contributions.

Master Honorees enjoy benefits ranging from access to the MongoDB Engineering and Product Management teams to discounted MongoDB Atlas Credits.

Preparations are underway for the MongoDB Masters Summit, which will be held on Tuesday, June 26th as part of MongoDB World 2018. We’ll have several speakers and a special Q&A session with Eliot Horowitz, our Co-Founder, and CTO. We encourage all members of our community to register for MongoDB World 2018, meet the Masters in person, and join our Advocacy Hub to start their own path to becoming a MongoDB Master.

So, all this talk of Masters – how, you might be thinking, do I become a Master?

Before I dive into an explanation of the requirements, please take a moment to review the bios of some of the existing Masters. You’ll easily spot some things in common across all of these incredibly talented and accomplished individuals.

Passion

Masters are passionate about technology and about solutions to technical problems. This passion drives these individuals to do things that few technologists will do. While this attribute is common among the existing and past Masters, it’s not easy to measure. You know it when you see it and it’s woven into the careers of many of the people I’ve encountered surrounding this program.

Impact

If passion is fuel, then impact is fire. Impact is the result of the passionate pursuit of worthy causes. Again, this is an attribute easily found in common across our Masters membership. Measuring impact is also difficult because in many cases, especially when dealing with the Masters, the impact of their actions, projects, and even their careers is widespread. Masters are individuals that positively impact their families, teams, companies, and their communities.

Execution

Execution is the spark that ignites fire. Elegant, efficient and effective solutions to technical challenges rarely, if ever, happen by accident. Rather, truly successful solutions require intelligent, deliberate execution – and in most cases, hard work. I strongly encourage you to spend time with any of the Masters and it will become clear that these individuals know how to execute. They know how to get things accomplished.

These are the attributes of a MongoDB Master and to achieve membership, an individual should be passionate about great technology and about solving technical problems. These individuals should have demonstrated, through successful execution, a massively beneficial impact on their company, team and/or community.

Are you interested in becoming a MongoDB Master, or do you think you may already meet the requirements? I would like to invite you to join us at MongoDB World in New York to learn more; consider completing the nomination form below to have yourself or a colleague considered for a MongoDB Masters membership.

MongoDB Masters membership nomination →

Stratifyd & MongoDB: AI-Driven Business Insights to Keep Customers Happy

2017 was a banner year for MongoDB's partner ecosystem. We remain strategic about engaging with our channels, and the results are validating our approach. Our strong network of ISVs, global system integrators, cloud, resellers, and technology partners is a competitive differentiator that helps us scale.

We are especially excited about the innovation and growth in store for our ISV business in 2018. It's already off to a great start. Our newest ISV partner Stratifyd is a fantastic example of how platforms built around MongoDB address serious market needs with the most cutting-edge, innovative technology.

Stratifyd is an end-to-end customer analytics platform powered by AI. The platform provides competitive advantages to some of the most recognized brands in the world. LivePerson, Etsy, MASCO, Kimberly-Clark, and many more rely on Stratifyd for a 360-degree view of their end customers.

Stratifyd analyzes customer interactions such as online reviews, social media posts, phone calls, emails, chats, surveys, CRM data, and more to turn them into actionable business insights which increase customer acquisition and retention, which is critical to the continued success of Stratifyd’s clients. In addition to these benefits, Stratifyd is just a flat-out cool implementation of AI.

I caught up with Stratifyd's CTO, Kevin O'Dell, to discuss the data technology behind the platform, and how MongoDB drives value for their customers.

---

For anyone that isn’t familiar with Stratifyd yet, how do you describe the platform?

Stratifyd uses human generated data to analyze, categorize, and understand intent with the purpose of changing human behavior. This changes the way brands interact with their customers, but also the way customers interact with brands, increasing customer acquisition and raising retention rates.

What was the genesis of the company? Why did you set out to build this?

Stratifyd was a result of postdoctoral research done at the University of North Carolina, Charlotte. Our founders were researching how AI could analyze unstructured data. During their research, they discovered strong business and government use cases. The founding team was working with numerous three letter agencies on predicting terrorist and disease movements globally. They were able to raise millions of dollars in funding from these agencies. The demand for a product organically grew from there, which led to the development of the Stratifyd platform.

I love the insights Stratifyd can provide – how would you describe the unique advantages that Stratifyd gives its customers?

Stratifyd provides near real-time business intelligence for contact center, marketing, product, and customer experience teams, all based on customer interactions. These insights enable businesses to be proactive rather than reactive in regard to business strategy. Stratifyd customers are able to respond to customer requests, complaints, or general feedback in near real time, changing the way companies interact with their end users. For example, we have empowered a customer with the knowledge to launch a new product line. Another customer gained insights that fundamentally changed how they are rolling out a 700+ million-dollar brand in a new continent.

What kind of feedback are you getting from customers that have deployed Stratifyd in their businesses?

Our customers love using our platform. They are surprised at how simple it is to use, and how powerful it is. They really appreciate how Stratifyd is making AI and machine learning meaningful for them in their day-to day-lives. Stratifyd helps ensure measurable results from day one. Speaking of implementation, they REALLY love that we don’t just use the term day one figuratively – customers are up and running in less than a day.

Talk to me about how you landed on MongoDB. What were you looking for in a database, and what problems were you having before moving to MongoDB?

That's an easy one to answer: speed and flexibility. Stratifyd ingests data from hundreds of sources. We needed a database that could keep up with high read and write request rates while handling a flexible schema. The hardest problem we were trying to solve was lack of secondary indexes; with those, MongoDB accelerated our query response times by at least 100x.

Can you share any best practices for scaling your MongoDB infrastructure? Any impressive metrics around the number of interactions, the volume of reads / writes per second, response times?

As a SaaS-first platform, being always-on is a HUGE best practice for us. MongoDB’s innate replication and failover abilities ensured less than 17 minutes of total downtime last year! Using MongoDB as our backend system, our AI can process a quarter of a million words in less than a minute.

How do you measure the impact of MongoDB on your business?

Our business wouldn’t be able to succeed without MongoDB. The uptime, failover, query response times, secondary indexes, and dynamic schemas have empowered most of Stratifyd’s key differentiators.

What advice would you give someone who is considering using MongoDB for their next project?

With all projects, I recommend truly understanding the requirements for the end results. There is a ton of excellent technology out there, but picking the wrong one can be detrimental to project success. Always run numerous tests, comparing different stacks to make sure you find the right one and fail fast on the wrong technology stack.

Stratifyd has some impressive customers, from the Fortune 500 to some really innovative startups: what’s next for the company?

We have some pretty big things planned for 2018. We're now providing more than just actionable intelligence; we are now streamlining and automating workflows. We are closing the customer feedback loop, which enables us to plug Stratifyd into any business process quickly to deliver measurable results.

New to MongoDB Atlas on AWS — AWS Cloud Provider Snapshots, Free Tier Now Available in Singapore & Mumbai


AWS Cloud Provider Snapshots

MongoDB Atlas is an automated cloud database service designed for agile teams who’d rather spend their time building apps than managing databases, backups, and restores. Today, we’re happy to announce that Cloud Provider Snapshots are now available for MongoDB Atlas replica sets on AWS. As the name suggests, Cloud Provider Snapshots provide fully managed backup storage and recovery using the native snapshot capabilities of the underlying cloud service provider.

Choosing a backup method for a database cluster in MongoDB Atlas

When this feature is enabled, MongoDB Atlas will perform snapshots against the primary in the replica set; snapshots are stored in the same cloud region as the primary, granting you control over where all your data lives. Please visit our documentation for more information on snapshot behavior.

Cloud Provider Snapshots on AWS have built-in incremental backup functionality, meaning that a new snapshot only saves the data that has changed since the previous one. This minimizes the time it takes to create a snapshot and lowers costs by reducing the amount of duplicate data. For example, a cluster with 10 GB of data on disk and 3 snapshots may require less than 30 GB of total snapshot storage, depending on how much of the data changed between snapshots.

Cloud Provider Snapshots are available for M10 clusters or higher in all of the 15 AWS regions where you can deploy MongoDB Atlas clusters.

Consider creating a separate Atlas project for database clusters where a different backup method is required. MongoDB Atlas only allows one backup method per project. Once you select a backup method — whether it’s Continuous Backup or Cloud Provider Snapshots — for a cluster in a project, Atlas locks the backup service to the chosen method for all subsequent clusters in that project. To change the backup method for the project, you must disable backups for all clusters in the project, then re-enable backups using your preferred backup methodology. Atlas deletes any stored snapshots when you disable backup for a cluster.


Free, $9, and $25 MongoDB Atlas clusters now available in Singapore & Mumbai

We’re committed to lowering the barrier to entry to MongoDB Atlas and allowing developers to build without worrying about database deployment or management. Last week, we released a 14% price reduction on all MongoDB Atlas clusters deployed in AWS Mumbai. And today, we’re excited to announce the availability of free and affordable database cluster sizes in South and Southeast Asia on AWS .

Free M0 Atlas clusters, which provide 512 MB of storage for experimentation and early development, can now be deployed in AWS Singapore and AWS Mumbai. If more space is required, M2 and M5 Atlas clusters, which provide 2 GB and 5 GB of storage, respectively, are now also available in these regions for just $9 and $25 per month.

Introducing the Aggregation Pipeline Builder in MongoDB Compass

Building MongoDB aggregations has never been so easy.

The most efficient way to analyze your data is where it already lives. That’s why we have MongoDB’s built-in aggregation framework. Have you tried it yet? If so, you know that it’s one of the most powerful MongoDB tools at your disposal. If not, you’re missing out on the ability to query your data in incredibly powerful ways. In fact, we like to say that “aggregate is the new find”. Built on the concept of data processing pipelines (like in Unix or PowerShell), the aggregation framework lets users “funnel” their documents through a multi-stage pipeline that filters, transforms, sorts, computes, aggregates your data, and more. The aggregation framework enables you to perform extensive analytics and statistical analysis in real time and generate pre-aggregated reports for dashboarding.

There are no limits to the number of stages an aggregation pipeline can have – pipelines can be as simple or as complex as you wish. In fact, the only limit is one’s imagination when it comes to deciding how to aggregate data. We’ve seen some very comprehensive pipelines!

With a rich library of over 25 stages and 100 operators (and growing with every release), the aggregation framework is an amazingly versatile tool. To help you be even more successful with it, we decided to build an aggregation construction user interface. The new Aggregation Pipeline Builder is now available with the latest release of Compass for beta testing. It’s available under the Aggregations tab.

The screenshot below depicts a sample pipeline on a movies collection that produces a listing of the title, year, and rating of all movies except for crime or horror, in English and Japanese which are rated either PG or G, starting with most recent, and sorted alphabetically within each year. Each stage was added gradually, with an ability to preview the result of our aggregation.

This easy-to-use UI lets you build your aggregation queries faster than ever before. There’s no need to worry about bracket matching, reordering stages, or remembering operator syntax with its intuitive drag-and-drop experience and code skeletons. You also get auto-completion for aggregation operators as well as query operators and even document field names.

If you need help understanding a particular operator, click on the info icon next to it and you’ll be taken directly to the appropriate guidance.

As you are building your pipeline, you can easily preview your results. This, in combination with an ability to rearrange and toggle stages on and off, makes it easy to troubleshoot your pipelines. When you are satisfied with the results, the constructed pipeline can be copied to the clipboard for easy pasting in your code, or simply saved in your favorites list for re-use later!

The aggregation authoring experience just got even more incredible with the new Compass aggregation pipeline builder. Why not check it out today?

  • Download the latest beta version of Compass
  • See the documentation for the aggregation pipeline builder in Compass
  • See the aggregation framework quick reference
  • To learn or brush up your aggregation framework skills, take M121 from our MongoDB University – it’s well worth it!

Also, please remember to send us your feedback by filing JIRA tickets or emailing it to: compass@mongodb.com.

MongoDB Atlas Price Reduction - AWS Mumbai

Developers use MongoDB Atlas, the fully automated cloud service for MongoDB, to quickly and securely create database clusters that scale effortlessly to meet the needs of a new generation of applications.

We recognize that the developer community in India is an incredibly vibrant one, one that is growing rapidly thanks to startups like Darwinbox. The team there built a full suite of HR services online, going from a standing start to a top-four sector brand in the Indian market in just two years.

As part of our ongoing commitment to support the local developer community and lower the barrier to entry to using a MongoDB service that removes the need for time-consuming administration tasks, we are excited to announce a price reduction for MongoDB Atlas. Prices are being reduced by up to 14% on all MongoDB Atlas clusters deployed in AWS Mumbai. With this, we aim to give more developers access to the best way to work with data, automated with built-in best practices.

MongoDB Atlas is available in India on AWS Mumbai and GCP Mumbai. It is also available on Microsoft Azure in Pune, Mumbai and Chennai. Never tried MongoDB Atlas? Click here to learn more.

MongoDB 4.0 Release Candidate 0 Has Landed

MongoDB enables you to meet the demands of modern apps with a technology foundation that enables you through:

  1. The document data model – presenting you the best way to work with data.
  2. A distributed systems design – allowing you to intelligently put data where you want it.
  3. A unified experience that gives you the freedom to run anywhere – future-proofing your work and eliminating vendor lock-in.

Building on the foundations above, MongoDB 4.0 is a significant milestone in the evolution of MongoDB, and we’ve just shipped the first Release Candidate (RC), ready for you to test.

Why is it so significant? Let’s take a quick tour of the key new features. And remember, you can learn about all of this and much more at MongoDB World'18 (June 26-27).

Multi-Document ACID Transactions

Previewed back in February, multi-document ACID transactions are part of the 4.0 RC. With snapshot isolation and all-or-nothing execution, transactions extend MongoDB ACID data integrity guarantees to multiple statements and multiple documents across one or many collections. They feel just like the transactions you are familiar with from relational databases, are easy to add to any application that needs them, and and don't change the way non-transactional operations are performed. With multi-document transactions it’s easier than ever for all developers to address a complete range of use cases with MongoDB, while for many of them, simply knowing that they are available will provide critical peace of mind that they can meet any requirement in the future. In MongoDB 4.0 transactions work within a replica set, and MongoDB 4.2 will support transactions across a sharded cluster*.

To give you a flavor of what multi-document transactions look like, here is a Python code snippet of the transactions API.

with client.start_session() as s:
    s.start_transaction():
    try:
        collection.insert_one(doc1, session=s)
        collection.insert_one(doc2, session=s)
    except:
        s.abort_transaction()
        raise
    s.commit_transaction()

And now, the transactions API for Java.

try (ClientSession clientSession = client.startSession()) {
          clientSession.startTransaction();
           try {
                   collection.insertOne(clientSession, docOne);
                   collection.insertOne(clientSession, docTwo);
                   clientSession.commitTransaction();
          } catch (Exception e) {
                   clientSession.abortTransaction();
           }
    }

Our path to transactions represents a multi-year engineering effort, beginning over 3 years ago with the integration of the WiredTiger storage engine. We’ve laid the groundwork in practically every part of the platform – from the storage layer itself to the replication consensus protocol, to the sharding architecture. We’ve built out fine-grained consistency and durability guarantees, introduced a global logical clock, refactored cluster metadata management, and more. And we’ve exposed all of these enhancements through APIs that are fully consumable by our drivers. We are feature complete in bringing multi-document transactions to replica sets, and 90% done on implementing the remaining features needed to deliver transactions across a sharded cluster.

Take a look at our multi-document ACID transactions web page where you can hear directly from the MongoDB engineers who have built transactions, review code snippets, and access key resources to get started.

Aggregation Pipeline Type Conversions

One of the major advantages of MongoDB over rigid tabular databases is its flexible data model. Data can be written to the database without first having to predefine its structure. This helps you to build apps faster and respond easily to rapidly evolving application changes. It is also essential in supporting initiatives such as single customer view or operational data lakes to support real-time analytics where data is ingested from multiple sources. Of course, with MongoDB’s schema validation, this flexibility is fully tunable, enabling you to enforce strict controls on data structure, type, and content when you need more control.

So while MongoDB makes it easy to ingest data without complex cleansing of individual fields, it means working with this data can be more difficult when a consuming application expects uniform data types for specific fields across all documents. Handling different data types pushes more complexity to the application, and available ETL tools have provided only limited support for transformations. With MongoDB 4.0, you can maintain all of the advantages of a flexible data model, while prepping data within the database itself for downstream processes.

The new $convert operator enables the aggregation pipeline to transform mixed data types into standardized formats natively within the database. Ingested data can be cast into a standardized, cleansed format and exposed to multiple consuming applications – such as the MongoDB BI and Spark connectors for high-performance visualizations, advanced analytics and machine learning algorithms, or directly to a UI. Casting data into cleansed types makes it easier for your apps to to process, sort, and compare data. For example, financial data inserted as a long can be converted into a decimal, enabling lossless and high precision processing. Similarly, dates inserted as strings can be transformed into the native date type.

When $convert is combined with over 100 different operators available as part of the MongoDB aggregation pipeline, you can reshape, transform, and cleanse your documents without having to incur the complexity, fragility, and latency of running data through external ETL processes.

Non-Blocking Secondary Reads

To ensure that reads can never return data that is not in the same causal order as the primary replica, MongoDB blocks readers while oplog entries are applied in batches to the secondary. This can cause secondary reads to have variable latency, which becomes more pronounced when the cluster is serving write-intensive workloads. Why does MongoDB need to block secondary reads? When you apply a sequence of writes to a document, then MongoDB is designed so that each of the nodes must show the writes in the same causal order. So if you change field "A" in a document and then change field "B", it is not possible to see that document with changed field "B" and not changed field "A". Eventually consistent systems suffer from this behavior, but MongoDB does not, and never has.

By taking advantage of storage engine timestamps and snapshots implemented for multi-document ACID transactions, secondary reads in MongoDB 4.0 become non-blocking. With non-blocking secondary reads, you now get predictable, low read latencies and increased throughput from the replica set, while maintaining a consistent view of data. Workloads that see the greatest benefits are those where data is batch loaded to the database, and those where distributed clients are accessing low latency local replicas that are geographically remote from the primary replica.

40% Faster Data Migrations

Very few of today’s workloads are static. For example, the launch of a new product or game, or seasonal reporting cycles can drive sudden spikes in load that can bring a database to its knees unless additional capacity can be quickly provisioned. If and when demand subsides, you should be able to scale your cluster back in, rightsizing for capacity and cost.

To respond to these fluctuations in demand, MongoDB enables you to elastically add and remove nodes from a sharded cluster in real time, automatically rebalancing the data across nodes in response. The sharded cluster balancer, responsible for evenly distributing data across the cluster, has been significantly improved in MongoDB 4.0. By concurrently fetching and applying documents, shards can complete chunk migrations up to 40% faster, allowing you to more quickly bring new nodes into service at just the moment they are needed, and scale back down when load returns to normal levels.

Extensions to Change Streams

Change streams, released with MongoDB 3.6, enable developers to build reactive, real-time, web, mobile, and IoT apps that can view, filter, and act on data changes as they occur in the database. Change streams enable seamless data movement across distributed database and application estates, making it simple to stream data changes and trigger actions wherever they are needed, using a fully reactive programming style.

With MongoDB 4.0, Change Streams can now be configured to track changes across an entire database or whole cluster. Additionally, change streams will now return a cluster time associated with an event, which can be used by the application to provide an associated wall clock time for the event.

Getting Started with MongoDB 4.0

Hopefully this gives you a taste of what’s coming in 4.0. There’s a stack of other stuff we haven’t covered today, but you can learn about it all in the resources below.

To get started with the RC now:

  1. Head over to the MongoDB download center to pick up the latest development build.
  2. Review the 4.0 release notes.
  3. Sign up for the forthcoming MongoDB University training on 4.0.

And you can meet our engineering team and other MongoDB users at MongoDB World'18 (June 26-27).

---

* Safe Harbor Statement

This blog post contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward-looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation.

In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.

MongoDB Celebrates Reaching 1,000 Employees

MongoDB
May 18, 2018
Culture

We’re excited to announce that MongoDB has officially reached 1,000 employees across the globe.

For some of us, it’s a time to reflect on how far we’ve come.

For others, this milestone helps to generates excitement for what is to come in the future.

And for the rest of us, it was another moment to be proud of, and a new reason to celebrate.

Were proud of how far we’ve come, and we’re looking forward to the next 1,000.