Improved Error Messages for Schema Validation in MongoDB 5.0
Rate this announcement
Many MongoDB users rely on schema
validation to
enforce rules governing the structure and integrity of documents in
their collections. But one of the challenges they faced was quickly
understanding why a document that did not match the schema couldn't be
inserted or updated. This is changing in the upcoming MongoDB 5.0
release.
Schema validation ease-of-use will be significantly improved by
generating descriptive error messages whenever an operation fails
validation. This additional information provides valuable insight into
which parts of a document in an insert/update operation failed to
validate against which parts of a collection's validator, and how. From
this information, you can quickly identify and remediate code errors
that are causing documents to not comply with your validation rules. No
more tedious debugging by slicing your document into pieces to isolate
the problem!
If you would like to evaluate this feature and provide us early
feedback, fill in this
form to
participate in the preview program.
The most popular way to express the validation rules is JSON
Schema.
It is a widely adopted standard that is also used within the REST API
specification and validation. And in MongoDB, you can combine JSON
Schema with the MongoDB Query Language (MQL) to do even more.
In this post, I would like to go over a few examples to reiterate the
capabilities of schema validation and showcase the addition of new
detailed error messages.
First, let's look at the new error message. It is a structured message
in the BSON format, explaining which part of the document didn't match
the rules and which validation rule caused this.
Consider this basic validator that ensures that the price field does not
accept negative values. In JSON Schema, the property is the equivalent
of what we call "field" in MongoDB.
When trying to insert a document with
{price: -2}, the following error
message will be returned.Some of the key fields in the response are:
failingDocumentId- the _id of the document that was evaluatedoperatorName- the operator used in the validation rulepropertiesNotSatisfied- the list of fields (properties) that failed validation checkspropertyName- the field of the document that was evaluatedspecifiedAs- the rule as it was expressed in the validatorreason - explanationof how the rule was not satisfiedconsideredValue- value of the field in the document that was evaluated
The error may include more fields depending on the specific validation
rule, but these are the most common. You will likely find the
propertyName and reason to be the most useful fields in the
response.Now we can look at the examples of the different validation rules and
see how the new detailed message helps us identify the reason for the
validation failure.
As an example, we'll use a collection of real estate properties in NYC
managed by a team of real estate agents.
Here is a sample document:
Our real estate properties are identified with property id (PID) that
has to follow a specific naming format: It should start with two letters
followed by five digits, and some letters and digits after, like this:
WS10011FG4 or EV10010A1.
We can use JSON Schema
pattern operator to create a rule for this as a
regular expression.Validator:
If we try to insert a document with a PID field that doesn't match the
pattern, for example
{ PID: "apt1" }, we will receive an error.The error states that the field
PID had the value of "apt1" and it
did not match the regular expression, which was specified as
"^[A-Z]{2}[0-9]{5}[A-Z]+[0-9]+$".The description may be localized into several languages. Currently, our
application only supports Spanish, German, and French, so the
localization object can only contain fields
description_es,
description_de, or description_fr. Other fields will not be allowed.We can use operator
patternProperties to describe this requirement as
regular expression and indicate that no other fields are expected here
with "additionalProperties": false.Validator:
Document like this can be inserted successfully:
Document like this will fail the validation check:
The error below indicates that field
localization contains additional
property description_cz. description_cz does not match the expected
pattern, so it is considered an additional property.Each real estate property in our collection has a type, and we want to
use one of the four types: "Residential," "Commercial," "Industrial," or
"Land." This can be achieved with the operator
enum.Validator:
The following document will be considered invalid:
The error states that field
type failed validation because "value was
not found in enum."Agents who manage each real estate property are stored in the
agents
array. Let's make sure there are no duplicate elements in the array, and
no more than three agents are working with the same property. We can use
uniqueItems and maxItems for this.The following document violates both if the validation rules.
The error returns information about failure for two rules: "array did
not match specified length" and "found a duplicate item," and it also
points to what value was a duplicate.
Now, we want to make sure that there's contact information available for
the agents. We need each agent's name and at least one way to contact
them: phone or email. We will use
requiredand anyOf to create this
rule.Validator:
The following document will fail validation:
Here the error indicates that the third element of the array
(
"itemIndex": 2) did not match the rule.Let's create another rule to ensure that if the document contains the
saleDate field, saleDetails is also present, and vice versa: If
there is saleDetails, then saleDate also has to exist.Now, let's try to insert the document with
saleDate but with no
saleDetails:The error now includes the property with dependency
saleDate and a
property missing from the dependencies: saleDetails.Notice that in JSON Schema, the field
dependencies is in the root
object, and not inside of the specific property. Therefore in the error
message, the details object will have a different structure:In the previous examples, when the JSON Schema rule was inside of the
"properties" object, like this:
the details of the error message contained
"operatorName": "properties" and a "propertyName":You can use MongoDB Query Language (MQL) in your validator right next to
JSON Schema to add richer business logic to your rules.
As one example, you can use
$expr
to add a check for a
discountPrice to be less than originalPrice
just like this:$expr
resolves to
true or false, and allows you to use aggregation
expressions to create sophisticated business rules.For a little more complex example, let's say we keep an array of bids in
the document of each real estate property, and the boolean field
isWinner indicates if a particular bid is a winning one.Sample document:
Let's make sure that only one of the
bids array elements can be marked
as the winner. The validator will have an expression where we apply a
filter to the array of bids to only keep the elements with "isWinner":
true, and check the size of the resulting array to be less or equal to
1.Validator:
Let's try to insert the document with few bids having
"isWinner": true.The produced error message will indicate which expression evaluated to
false.
As the last example, let's see how we can use the geospatial features of
MQL to ensure that all the real estate properties in the collection are
located within the New York City boundaries. Our documents include a
geoLocation field with coordinates. We can use $geoWithin to check
that these coordinates are inside the geoJSON polygon (the polygon for
New York City in this example is approximate).Validator:
A document like this will be inserted successfully.
The following document will fail.
The error will indicate that validation failed the
$geoWithin
operator, and the reason is "none of the considered geometries were
contained within the expression's geometry."Schema validation is a great tool to enforce governance over your data
sets. You have the choice to express the validation rules using JSON
Schema, MongoDB Query Language, or both. And now, with the detailed
error messages, it gets even easier to use, and you can have the rules
be as sophisticated as you need, without the risk of costly maintenance.
If you would like to evaluate this feature and provide us early
feedback, fill in this
form to
participate in the preview program.
More posts on schema validation:
Questions? Comments? We'd love to connect with you. Join the
conversation on the MongoDB Community
Forums.
Safe Harbor
The development, release, and timing of any features or functionality
described for our products remains at our sole discretion. This
information is merely intended to outline our general product direction
and it should not be relied on in making a purchasing decision nor is
this a commitment, promise or legal obligation to deliver any material,
code, or functionality.
Rate this announcement