What’s New in MongoDB 3.2, Part 2: New Support for Mission-Critical Applications
Welcome to part 2 of our 3-part MongoDB 3.2 blog series.
- In part 1 we covered the new storage engines and the use-cases they served.
- In part 3 we’ll cover new tools and integrations supporting data analysts, DBAs, and operations teams.
In this post, I’ll cover features designed to support mission-critical applications, including document validation and the enhanced replication protocol. Remember, you can get the detail now on everything MongoDB 3.2 offers by downloading the What’s New white paper.
Document Validation: Data Governance for Dynamic Schema
Dynamic schemas bring great agility, but it is also important that controls can be implemented to maintain data quality, especially if the database is powering multiple applications, or is integrated into a larger data management platform that feeds into upstream and downstream systems. Rather than delegating enforcement of these controls back into application code, MongoDB provides Document Validation within the database. Users can enforce checks on document structure, data types, data ranges, and the presence of mandatory fields. As a result, DBAs can apply data governance standards, while developers maintain the benefits of a flexible document model.
Validating Documents in MongoDB 3.2
There is significant flexibility to customize which parts of the documents are and are not validated for any collection. For any key it might be appropriate to check:
- That it exists
- If it does exist, that the value is of the correct type
- That the value is in a particular format (regular expressions can be used to check if the contents of the string matches a particular pattern – that it’s a properly formatted email address, for example)
- That the value falls within a given range
As an example, the following snippet adds a rule to the contacts collection that validates:
- The year of birth is no later than 1994
- The document contains a phone number and/or an email address
- When present, the phone number and email addresses are strings
db.runCommand({
collMod: "contacts",
validator: {
$and: [
{year_of_birth: {$lte: 1994}},
{$or: [
{phone: { $type: "string" }},
{email: { $type: "string" }}
]}]
}})Adding the validation checks to a collection is very intuitive to any developer or DBA familiar with MongoDB, as Document Validation uses the standard MongoDB Query Language.
For a more in-depth discussion of this feature, including code samples and a tutorial, please see the Document Validation blog post or the documentation.
Enhanced Replication Protocol: Fast Failover and Stricter Durability Guarantees
In a distributed system like MongoDB, it should be assumed that individual nodes can and will fail. MongoDB’s priorities in the event of a node failing are to:
- Ensure data consistency
- Resume full service as quickly as possible to maximize availability
MongoDB 3.2 introduces an enhanced replication protocol that delivers faster service recovery in the event of a primary failure, as well as stricter durability guarantees. The enhanced replication protocol extends the Raft consensus algorithm to offer greater deployment flexibility while maintaining compatibility with replication constructs offered in earlier MongoDB releases. Specifically, the protocol maintains support for replica set arbiters, replica set member election priorities and secondary members replicating from other secondaries to enable chained replication.
The enhanced replication protocol reduces the failover interval by optimizing the algorithms used to detect replica set primary failures and elect a new primary. Failover time is dependent on several factors, including network latency. It is important for the system to avoid unnecessary failovers, and to provide flexibility for the needs of different deployments. MongoDB 3.2 introduces a new parameter called electionTimeoutMillis to allow users to configure their systems for optimal failover behavior:
- Higher values result in slower failovers but decreased sensitivity to network latency and load on the primary node
- Lower values result in faster failover, but increased sensitivity to network latency and load on the primary.
electionTimeoutMillis defaults to 10,000 milliseconds (10 seconds).
Review the documentation for more information.
Durability Guarantees
In addition to faster failover, the enhanced protocol offers stricter consistency and durability controls for write operations across replica sets. With a write concern configured to apply writes to one or more secondaries before acknowledging the operation, the enhanced protocol will now commit operations to both the memory and the journal on those secondary members before reporting successful completion to the application. Additionally, when using the majority write concern, the change will also be committed to the memory and journal on the primary before the acknowledgment is made.
By default, any replica set upgraded from a pre-MongoDB 3.2 release will use the previous replication protocol, while new replica sets will use the enhanced protocol. It is possible to manually switch to the old or new protocol by setting protocolVersion to 0 or 1 respectively.
Simplified Sharded Cluster Management
MongoDB scales out databases on commodity hardware using a technique called sharding. The config servers are a critical component in a sharded cluster, storing the metadata used by the mongos query router to direct read and write operations to the appropriate shards, thereby abstracting complexity from applications accessing the database.
Prior to MongoDB 3.2, the config servers were implemented as three special-purpose mongod instances using their own write protocols, coordinators, and consistency checking. This architecture complicated the management of sharded clusters, and increased latency when deploying MongoDB across more than three data centers.
Starting with MongoDB 3.2, the config servers will, by default, be deployed as a MongoDB replica set. This change improves metadata consistency and manageability as the config servers can now take advantage of the standard replica set read and write protocols. Furthermore, config server replica sets can now span more than three data centers with up to 50 replica set members supported, providing higher levels of availability and lower cross-region latency. Review the documentation to learn more.
Next Steps
That wraps up the second part of our 3-part blog series. Remember, you can get the detail now on everything MongoDB 3.2 offers by downloading the What’s New white paper.
Alternatively, if you’d had enough of reading about it and want to get your hands on the code now, then:
To start using MongoDB 3.2 as quickly and efficiently as possible, bring in the experts. MongoDB’s consulting engineers can deliver a private training on 3.2 features tailored to your needs, then work with you to develop a customized upgrade plan for your deployment. Interested? Learn more.
To start using MongoDB 3.2 as quickly and efficiently as possible, bring in the experts. MongoDB’s consulting engineers can deliver a private training on 3.2 features tailored to your needs, then work with you to develop a customized upgrade plan for your deployment. Interested?
About the Author - Mat Keep
Mat is a director within the MongoDB product marketing team, responsible for building the vision, positioning and content for MongoDB’s products and services, including the analysis of market trends and customer requirements. Prior to MongoDB, Mat was director of product management at Oracle Corp. with responsibility for the MySQL database in web, telecoms, cloud and big data workloads. This followed a series of sales, business development and analyst / programmer positions with both technology vendors and end-user companies.