- Working With Data >
- Queries
Queries¶
On this page
Mongoid provides a rich query DSL inspired by ActiveRecord. A trivial query looks as follows:
A more complex query utilizing various Mongoid features could be as follows:
The query methods return Mongoid::Criteria
objects, which are chainable
and lazily evaluated wrappers for MongoDB query language (MQL).
The queries are executed when their result sets are iterated. For example:
Methods like first
and last
return the individual documents immediately.
Otherwise, iterating a Criteria object with methods like each
or map
retrieves the documents from the server. to_a
can be used to force
execution of a query that returns an array of documents, literally converting
a Criteria object to an Array.
When a query method is called on a Criteria instance, the method returns a new Criteria instance with the new conditions added to the existing conditions:
Condition Syntax¶
Mongoid supports three ways of specifying individual conditions:
- Field syntax.
- MQL syntax.
- Symbol operator syntax.
All syntaxes support querying embedded documents using the dot notation. All syntaxes respect field types, if the field being queried is defined in the model class, and field aliases.
The examples in this section use the following model definition:
Field Syntax¶
The simplest querying syntax utilizes the basic Ruby hashes. Keys can be symbols or strings, and correspond to field names in MongoDB documents:
MQL Syntax¶
An MQL operator may be specified on any field using the hash syntax:
Symbol Operator Syntax¶
MQL operators may be specified as methods on symbols for the respective field name, as follows:
Embedded Documents¶
To match values of fields of embedded documents, use the dot notation:
Note
Queries always return top-level model instances, even if all of the conditions are referencing embedded documents.
Field Types¶
In order to query on a field, it is not necessary to add the field to the model class definition. However, if a field is defined in the model class, the type of the field is taken into account when constructing the query:
Aliases¶
Queries take into account storage field names and field aliases:
Since id and _id fields are aliases, either one can be used for queries:
Logical Operations¶
Mongoid supports and
, or
, nor
and not
logical operations on
Criteria
objects. These methods take one or more hash of conditions
or another Criteria
object as their arguments, with not
additionally
having an argument-free version.
For backwards compatibility with earlier Mongoid versions, all of the logical operation methods also accept arrays of parameters, which will be flattened to obtain the criteria. Passing arrays to logical operations is deprecated and may be removed in a future version of Mongoid.
The following calls all produce the same query conditions:
Operator Combinations¶
As of Mongoid 7.1, logical operators (and
, or
, nor
and not
)
have been changed to have the the same semantics as those of ActiveRecord.
To obtain the semantics of or
as it behaved in Mongoid 7.0 and earlier,
use any_of
which is described below.
When conditions are specified on the same field multiple times, all conditions are added to the criteria:
any_of
, nor
and not
behave similarly, with not
producing
different query shapes as described below.
When and
, or
and nor
logical operators are used, they
operate on the criteria built up to that point and its argument.
where
has the same meaning as and
:
and
Behavior¶
The and
method will add new simple conditions to the top level of the
criteria, unless the receiving criteria already has a condition on the
respective fields, in which case the conditions will be combined with $and
.
As of Mongoid 7.1, specifying multiple criteria on the same field with and
combines all criteria so specified, whereas in previous versions of Mongoid
conditions on a field sometimes replaced previously specified conditions on
the same field, depending on which form of and
was used.
or
/nor
Behavior
_———————-
or
and nor
produce $or
and $nor
MongoDB operators, respectively,
using the receiver and all of the arguments as operands. For example:
If the only condition on the receiver is another or
/nor
, the new
conditions are added to the existing list:
Use any_of
to add a disjunction to a Criteria object while maintaining
all of the conditions built up so far as they are.
any_of
Behavior¶
any_of
adds a disjunction built from its arguments to the existing
conditions in the criteria. For example:
The conditions are hoisted to the top level if possible:
not
Behavior¶
not
method can be called without arguments, in which case it will negate
the next condition that is specified. not
can also be called with one
or more hash conditions or Criteria
objects, which will all be negated and
added to the criteria.
Note
$not
in MongoDB server cannot be used with a string argument.
Mongoid uses $ne
operator to achieve such a negation:
Similarly to and
, not
will negate individual conditions for simple
field criteria. For complex conditions and when a field already has a condition
defined on it, since MongoDB server only supports the $not
operator on
a per-field basis rather than globally, Mongoid emulates $not
by using
an {'$and' => [{'$nor' => ...}]}
construct:
If using not
with arrays or regular expressions, please note the
caveats/limitations of $not
stated in the MongoDB server documentation.
Incremental Query Construction¶
By default, when conditions are added to a query, Mongoid considers each
condition complete and independent from any other conditions potentially
present in the query. For example, calling in
twice adds two separate
$in
conditions:
Some operator methods support building the condition incrementally. In this case, when an condition on a field which uses one of the supported operators is being added, if there already is a condition on the same field using the same operator, the operator expressions are combined according to the specified merge strategy.
Merge Strategies¶
Mongoid provides three merge strategies:
- Override: the new operator instance replaces any existing conditions on the same field using the same operator.
- Intersect: if there already is a condition using the same operator on the same field, the values of the existing condition are intersected with the values of the new condition and the result is stored as the operator value.
- Union: if there already is a condition using the same operator on the same field, the values of the new condition are added to the values of the existing condition and the result is stored as the operator value.
The following snippet demonstrates all of the strategies, using in
as the
example operator:
The strategy is requested by calling override
, intersect
or union
on a Criteria
instance. The requested strategy applies to the next
condition method called on the query. If the next condition method called does
not support merge strategies, the strategy is reset, as shown in the following
example:
Since ne
does not support merge strategies, the union
strategy was
ignored and reset and when in
was invoked the second time there was no
strategy active.
Warning
Merge strategies currently assume the previous condition(s) have been added
to the top level of the query, however this is not always the case
(conditions may be nested under an $and
clause). Using merge strategies
with complex criteria may cause incorrect queries to be constructed.
This misbehavior is intended to be fixed in the future.
Supported Operator Methods¶
The following operator methods support merge strategies:
all
in
nin
The set of methods may be expanded in future releases of Mongoid. For future compatibility, only invoke a strategy method when the next method call is an operator that supports merge strategies.
Note that the merge strategies are currently only applied when conditions are
added through the designated methods. In the following example merge strategy
is not applied because the second condition is added via where
, not via
in
:
This behavior may change in a future release of Mongoid and should not be relied upon.
In contrast, it does not matter how the existing query was built when a
merge strategy-supporting operator method is invoked. In the following
example, the first condition was added through where
but the strategy
mechanism still applies:
Operator Value Expansion¶
Operator methods that support merge strategies all take Array
as their value
type. Mongoid expands Array
-compatible types, such as a Range
,
when they are used with these operator methods:
Additionally, Mongoid has historically wrapped non-Array
values in arrays,
as the following example demonstrates:
This wrapping behavior is deprecated and should not be relied on. It may be removed in a future release of Mongoid.
Query Methods¶
elem_match¶
This matcher finds documents with array fields where one of the array values matches all of the conditions. For example:
elem_match
also works with embedded associations:
elem_match
does not work with non-embedded associations because MongoDB
does not have joins - the conditions would be added to the collection
that is the source of a non-embedded association rather than the collection
of the association’s target.
elem_match
can also be used with recursively embedded associations,
as the following example shows:
Projection¶
Mongoid provides two projection operators: only
and without
.
only
¶
The only
method retrieves only the specified fields from the database. This
operation is sometimes called “projection”.
Attempting to reference attributes which have not been loaded results in
ActiveModel::MissingAttributeError
.
Even though Mongoid currently allows writing to attributes that have not been loaded, such writes will not be persisted (MONGOID-4701) and should therefore be avoided.
only
can also be used with embedded associations:
Note
Server versions 4.2 and lower allowed projecting both an association and the association’s fields in the same query, as follows:
The most recent projection specification overrides the earlier one. For example, the above query was equivalent to:
Server versions 4.4 and higher prohibit specifying an association and its fields in projection in the same query.
only
can be specified with referenced associations (has_one, has_many,
has_and_belongs_to_many) but is currently ignored for referenced associations -
all fields of referenced associations will be loaded
(MONGOID-4704).
Note that if a document has has_one
or has_and_belongs_to_many
associations,
the fields with foreign keys must be included in the list of attributes
loaded with only
for those associations to be loaded. For example:
without
¶
The opposite of only
, without
causes the specified fields to be omitted:
Because Mongoid requires the _id
field for various operations, it (as well
as its id
alias) cannot be omitted via without
:
Ordering¶
Mongoid provides the order
method on Criteria
objects and its alias,
order_by
, to specify the ordering of documents. These methods take a
hash indicating which fields to order the documents by, and whether to use
ascending or descending order for each field.
The direction may be specified as integers 1
and -1
for ascending
and descending, respectively, or as symbols :asc
and :desc
, or as
strings "asc"
and "desc"
.
Alternatively, order
accepts an array of two-element arrays specifying
the ordering. Field names and directions may be strings or symbols.
Another way of providing the order is to use #asc
and #desc
methods
on symbols, as follows:
The arguments can be provided as a string using SQL syntax:
Finally, there are asc
and desc
methods that can be used instead of
order
/order_by
:
order
calls can be chained, in which case the oldest calls define the
most significant criteria and the newest calls define the least significant
ones (since in Ruby hashes maintain the order of their keys):
This can sometimes lead to surprising results if there are scopes, including
the default scope, that use order
/order_by
. For example, in the
following snippet bands are ordered by name first because the order in the
default scope takes precedence over the order given in the query, due to
the default scope being evaluated first:
Pagination¶
Mongoid provides the pagination operators limit
, skip
, and batch_size
on Criteria
.
limit
¶
limit
sets the total number of documents to be returned by a query:
skip
¶
skip
(alias: offset
) sets the number of query results to skip
before returning documents. The limit
value, if specified, will be applied
after documents are skipped. When performing pagination, skip
is recommended
to be combined with ordering to ensure consistent results.
batch_size
¶
When executing large queries, or when iterating over query results with an enumerator method such as
Criteria#each
, Mongoid automatically uses the MongoDB getMore command to load results in batches.
The default batch_size
is 1000, however you may set it explicitly:
Finding By _id
¶
Mongoid provides the find
method on Criteria
objects to find documents
by their _id
values:
The find
method performs type conversion, if necessary, of the argument
to the type declared in the model being queried for the _id
field.
By default, the _id
type is BSON::ObjectId
, thus the query above
is equivalent to:
Note
When querying collections directly using the driver, type conversion is not automatically performed:
The find
method can accept multiple arguments, or an array of arguments.
In either case each of the arguments or array elements is taken to be an _id
value, and documents with all of the specified _id
values are returned in
an array:
If the same _id
value is given more than once, the corresponding document
is only returned once:
The documents returned are not ordered, and may be returned in a different
order from the order of provided _id
values, as illustrated in the above
examples.
If any of the _id
values are not found in the database, the behavior of
find
depends on the value of the raise_not_found_error
configuration
option. If the option is set to true
, find
raises
Mongoid::Errors::DocumentNotFound
if any of the _id``s are not found.
If the option is set to ``false
and find
is given a single _id
to
find and there is no matching document, find
returns nil
. If the
option is set to false
and find
is given an array of ids to find
and some are not found, the return value is an array of documents that were
found (which could be empty if no documents were found at all).
Additional Query Methods¶
Mongoid also has some helpful methods on criteria.
Operation | Example |
---|---|
Get the total number of documents matching a filter, or the total number of documents in a collection. Note this will always hit the database for the count. As of Mongoid 7.2, the |
|
Get an approximate number of documents in the collection using the
collection metadata. The |
|
Get a list of distinct values for a single field. Note this will always hit the database for the distinct values. This method accepts the dot notation, thus permitting referencing fields in embedded associations. This method respects :ref:`field aliases <field-aliases>`, including those defined in embedded documents. |
|
Iterate over all matching documents in the criteria. |
|
Determine if any matching documents exist. Will return true if there are 1 or more. |
|
Find a document by the provided attributes. If not found,
raise an error or return nil depending on the value of the
|
|
Find a document by the provided attributes, and if not found create and return a newly persisted one. Note that attributes provided in the arguments to this method will override any set in ``create_with``. |
|
Find a document by the provided attributes, and if not found return a new one. |
|
Finds a single document given the provided criteria. Get a list of documents by passing in a limit argument. This method automatically adds a sort on _id. This can cause performance issues, so if the sort is undesirable, Criteria#take can be used instead. |
|
Find the first document by the provided attributes, and if not found create and return a newly persisted one. |
|
Find the first document by the provided attributes, and if not found
create and return a newly persisted one using |
|
Find the first document by the provided attributes, and if not found return a new one. |
|
Find documents for a provided JavaScript expression, optionally with
the specified variables added to the evaluation scope. The scope
argument is supported in MongoDB 4.2 and lower.
In MongoDB 3.6 and higher, prefer $expr over |
|
Same as count but caches subsequent calls to the database |
|
Get the values from one document for the provided fields. Returns nil for unset fields and for non-existent fields. This method does not apply a sort to the documents, so it will not necessarily return the values from the first document. This method accepts the dot notation, thus permitting referencing fields in embedded associations. This method respects :ref:`field aliases <field-aliases>`, including those defined in embedded documents. |
|
Get all the values for the provided field. Returns nil for unset fields and for non-existent fields. This method accepts the dot notation, thus permitting referencing fields in embedded associations. This method respects :ref:`field aliases <field-aliases>`, including those defined in embedded documents. |
|
Get a list of n documents from the database or just one if no parameter is provided. This method does not apply a sort to the documents, so it can return different document(s) than #first and #last. |
|
Get a document from the database or raise an error if none exist. This method does not apply a sort to the documents, so it can return different document(s) than #first and #last. |
|
Get a mapping of values to counts for the provided field. This method accepts the dot notation, thus permitting referencing fields in embedded associations. This method respects :ref:`field aliases <field-aliases>`, including those defined in embedded documents. |
Eager Loading¶
Mongoid provides a facility to eager load documents
from associations to prevent the n+1 issue when
iterating over documents with association access. Eager loading is supported on
all associations with the exception of polymorphic belongs_to
associations.
Regular Expressions¶
MongoDB, and Mongoid, allow querying documents by regular expressions.
Given the following model definitions:
… we can query using simple Ruby regular expressions in a natural way:
It is also possible to query using PCRE syntax by constructing
BSON::Regexp::Raw
objects explicitly:
Conditions On Fields¶
When a condition uses a field defined in the model, the value being specified
in the condition is converted according to the rules of the field, if any.
For example, consider the following model definition that contains a Time
field, a Date
field and an implicit Object
field, and also
intentionally does not define a field called deregistered_at
:
Queries on born_on
and registered_at
fields using Date
and Time
values, respectively, are straightforward:
But, note the differences in behavior when providing a Date
instance
in all possible scenarios:
When using the registered_at
field which is of type Time
, the date
was interpreted to be in local time (as per the configured time zone). When using the born_on
field which is of type Date
,
the date was interpreted to be in UTC. When using the voted_at
field
which was defined without a type (hence implicitly as an Object
),
the date was used unmodified in the constructed query. When using a
nonexistent field deregistered_at
the date was interpreted to be in UTC
and converted to a time, matching the behavior of querying a Date
field.
Scoping¶
Scopes provide a convenient way to reuse common criteria with more business domain style syntax.
Named Scopes¶
Named scopes are simply criteria defined at class load that are referenced by a provided name. Just like normal criteria, they are lazy and chainable.
Named scopes can take procs and blocks for accepting parameters or extending functionality.
By default, Mongoid allows defining a scope that would shadow an existing class method, as the following example shows:
To have Mongoid raise an error when a scope would overwrite an existing class
method, set the scope_overwrite_exception
configuration option to true
.
Default Scopes¶
Default scopes can be useful when you find yourself applying the same criteria to most queries, and wish to specify these criteria as the default. Default scopes are procs that return criteria objects.
Specifying a default scope also initializes the fields of new models to the values given in the default scope, if the values are simple literals:
Note that if a default value is provided both in the field definition and in the default scope, the value in the default scope takes precedence:
Because a default scope initializes fields in new models as just described, defining a default scope with a dotted key and a simple literal value is not possible:
A workaround is to define the default scope as a complex query:
You can tell Mongoid not to apply the default scope by using
unscoped
, which can be inline or take a block.
You can also tell Mongoid to explicitly apply the default scope again later to always ensure it’s there.
If you are using a default scope on a model that is part of an association, you must reload the association to have scoping reapplied. This is important to note if you change a value of a document in the association that would affect its visibility within the scoped association.
Note
After the default scope is applied, it is no longer distinguished from
other query conditions. This can lead to surprising behavior when using
or
and nor
operators in particular:
In the last example, you might expect the two conditions
(active: true
and touring: true
) to be combined with an $and
,
but because the Band
class already has the scope applied to it,
it becomes one of the disjunction branches of the or
.
Runtime Default Scope Override¶
You can use the with_scope
method to change the default scope in a block
at runtime:
Note
If with_scope calls are nested, when the nested with_scope block completes
Mongoid 7 sets the current scope to nil instead of the parent scope.
Mongoid 8 will set the current scope to the correct parent scope.
To get Mongoid 8 behavior in Mongoid 7.4 and higher, set the
Mongoid.broken_scoping
global option to false.
Class Methods¶
Class methods on models that return criteria objects are also treated like scopes, and can be chained as well.
Queries + Persistence¶
Mongoid supports persistence operations off of criteria in a light capacity for when you want to expressively perform multi document inserts, updates, and deletion.
Warning
Criteria ordering and pagination conditions, including order
, limit
,
offset
, and batch_size
, will be ignored on the following operations.
Operation | Example |
---|---|
Create a newly persisted document. |
|
Create a newly persisted document and raise an exception on validation failure. |
|
Create a new (unsaved) document. |
|
Update attributes of the first matching document. |
|
Update attributes of all matching documents. |
|
Perform an $addToSet on all matching documents. |
|
Perform a $bit on all matching documents. |
|
Perform an $inc on all matching documents. |
|
Perform a $pop on all matching documents. |
|
Perform a $pull on all matching documents. |
|
Perform a $pullAll on all matching documents. |
|
Perform a $push on all matching documents. |
|
Perform a $push with $each on all matching documents. |
|
Perform a $rename on all matching documents. |
|
Perform a $set on all matching documents. |
|
Perform a $unset on all matching documents. |
|
Deletes all matching documents in the database. |
|
Deletes all matching documents in the database while running callbacks for all. This loads all documents into memory and can be an expensive operation. |
Query Cache¶
The Ruby MongoDB driver provide query caching functionality. When enabled, the query cache saves the results of previously executed find and aggregation queries and reuses them in the future instead of performing the queries again, thus increasing application performance and reducing database load.
The query cache has historically been provided by Mongoid but as of driver version 2.14.0, the query cache implementation has been moved into the driver. When Mongoid is used with driver version 2.14.0 or later, Mongoid delegates all work to the driver’s query cache implementation.
Please review the driver query cache documentation for details about the driver’s query cache behavior.
The rest of this section assumes that driver 2.14.0 or later is being used and Mongoid delegates to the driver’s query cache.
Enabling Query Cache¶
The query cache may be enabled by using the driver’s namespace or Mongoid’s namespace.
Enabling Query Cache Via Middleware¶
Applications that utilize a Rack-based web framework (Rails, etc.) can take advantage of the Rack middleware provided by Mongoid to automatically enable the query cache for each web request, and clear the query cache after each request completes. Please see the Query Cache Rack Middleware section on the configuration page for instructions.
Note that the Query Cache Middleware does not apply to code executed outside web requests.
Enabling Query Cache Manually¶
To enable the Query Cache manually for a code segment, use:
The Query Cache can also be explicitly enabled and disabled, although we recommend to use the block form described above:
Alternatively, you can reference the driver’s query cache module directly:
Caching the Result of #first
¶
Calling the first
method on a model class imposes an ascending sort by
the _id
field on the underlying query. This may produce unexpected behavior
with query caching.
For example, when calling all
on a model class and then first
,
one would expect the second query to use the cached results from the first.
However, because of the sort imposed on the second query, both methods
will query the database and separately cache their results.
To use the cached results, call all.to_a.first
on the model class.