Working with the MongoDB Single-Collection Pattern in Swift
Rate this quickstart
It's a MongoDB axiom that you get the best performance and scalability by storing together the data that's most commonly accessed together.
The simplest and most obvious approach to achieve this is to embed all related data into a single document. This works great in many cases, but there are a couple of scenarios where it can become inefficient:
- (Very) many to many relationships. This can lead to duplicated data. This duplication is often acceptable — storage is comparatively cheap, after all. It gets more painful when the duplicated data is frequently modified. You then have the cost of updating every document which embeds that data.
- Reading small parts of large documents. Even if your query is only interested in a small fraction of fields in a document, the whole document is brought into cache — taking up memory that could be used more effectively.
- Large, mutable documents. Whenever your application makes a change to a document, the entire document must be written back to disk at some point (could be combined with other changes to the same document). WiredTiger writes data to disk in 4 KB blocks after compression — that typically maps to a 16-20 KB uncompressed document. If you're making lots of small edits to a 20+ KB document, then you may be wasting disk IO.
If embedding all of the data in a single document isn't the right pattern for your application, then consider the single-collection design. The single-collection pattern can deliver comparable read performance to embedded documents, while also optimizing for updates.
There are variants on the single-collection pattern, but for this post, I focus on the key aspects:
- Related data that's queried together is stored in the same collection.
- The documents can have different structures.
- Indexes are added so that all of the data for your frequent queries can be fetched with a single index lookup.
At this point, your developer brain may be raising questions about how your application code can cope with this. It's common to read the data from a particular collection, and then have the MongoDB driver convert that document into an object of a specific class. How does that work if the driver is fetching documents with different shapes from the same collection? This is the primary thing I want to demonstrate in this post.
I'll be using Swift, but the same principles apply to other languages. To see how to do this with Java/Spring Data, take a look at Single-Collection Designs in MongoDB with Spring Data.
I recently started using the MongoDB Swift Driver for the first time. I decided to build a super-simple Mac desktop app that lets you browse your collections (which MongoDB Compass does a much better job of) and displays Change Stream events in real time (which Compass doesn't currently do).
Provide your connection-string and then browse your collections. Select the "Enable change streams" option to display change events in real time.
The app will display data from most collections as generic JSON documents, with no knowledge of the schema. There's a special case for a collection named "Collection" in a database named "Single" — we'll look at that next.
The Simple.Collection collection needs to contain these (or similar) documents:
This data represents a shopping basket with an
_id of "basket1". There are two items associated with basket1 — basket1-item1 and basket1-item2. A single query will fetch all three documents for the basket (find all documents where _id starts with "basket1"). There is always an index on the _id attribute, and so that index will be used.Note that all of the data for a basket in this dataset is extremely small — well below the 16-20K threshold — and so in a real life example, I'd actually advise embedding everything in a single document instead. The single-collection pattern would make more sense if there were a large number of line items, and each was large (e.g., if they embedded multiple thumbnail images).
Each document also has a
docType attribute to identify whether the document refers to the basket itself, or one of the associated items. If your application included a common query to fetch just the basket or just the items associated with the basket, then you could add a composite index: { _id: 1, docType: 1}.Other uses of the
docType field include:- A prompt to help humans understand what they're looking at in the collection.
- Filtering the data returned from a query to just certain types of documents from the collection.
- Filtering which types of documents are included when using MongoDB Compass to examine a collection's schema.
- Allowing an application to identify what type of document its received. The application code can then get the MongoDB driver to unmarshal the document into an object of the correct class. This is what we'll look at next.
We'll use the same desktop app to see how your code can discriminate between different types of documents from the same collection.
The app has hardcoded knowledge of what a basket and item documents looks like. This allows it to render the document data in specific formats, rather than as a JSON document:
The code to determine the document
docType and convert the document to an object of the appropriate class can be found in CollectionView.swift.CollectionView fetches all of the matching documents from MongoDB and stores them in an array of
BSONDocuments:The application can then loop over each document in
docs, checks the docType attribute, and then decides what to do based on that value:If
docType == "basket", then the code converts the generic doc into a Basket object and passes it to BasketView for rendering.This is the
Basket class, including initializer to create a Basket from a BSONDocument:Similarly for
Items:The sub-views can then use the attributes from the properly-typed object to render the data appropriately:
The single-collection pattern is a way to deliver read and write performance when embedding or other design patterns aren't a good fit.
This pattern breaks the 1-1 mapping between application classes and MongoDB collections that many developers might assume. This post shows how to work around that:
- Extract a single docType field from the BSON document returned by the MongoDB driver.
- Check the value of docType and get the MongoDB driver to map the BSON document into an object of the appropriate class.