Docs Home → Atlas App Services
Optimize Sync Storage in Atlas
On this page
Overview
Atlas Device Sync uses space in your app's synced Atlas cluster to store metadata for sync. This includes a history of changes to each realm. Atlas App Services minimizes this space usage in your Atlas cluster. Minimizing metadata is necessary to reduce the time and data needed for sync.
History
The App Services backend keeps a history of changes to underlying data for each realm, similar to the MongoDB oplog. App Services uses this history to synchronize data between the backend and clients. App Services stores history in your synced Atlas cluster.
Trimming
When you set a client maximum offline time in an App that uses Flexible Sync, trimming deletes changes older than the client maximum offline time.
Note
Flexible Sync uses trimming to reduce Device Sync history stored in Atlas. Partition-Based Sync uses backend compaction.
Client Maximum Offline Time
Used in trimming, the client maximum offline time controls the age limit of history. This indirectly changes how long a client can remain offline between synchronization sessions with the backend. Clients that do not synchronize for more than the specified number of days may experience a client reset the next time they connect with the backend.
Setting the client maximum offline time to a lower value will decrease the amount of history required by sync. The resulting optimization lowers storage usage in the synced Atlas cluster.
New Apps automatically enable client maximum offline time with a default value of 30 days.
Warning
Client Maximum Offline Time Causes Permanent Changes to History
Client maximum offline time enables trimming for older history. This permanently changes affected history, and can cause client resets in the future even after disabling the feature.
Key Concepts
Sync should always converge at the same end state on all clients. In order to converge during a sync, clients require the full history of changes beginning immediately after their last sync. When a client does not sync for a long period of time, trimming can alter the history in ways that prevent the client from converging. Since synchronization relies on all clients converging on a common result, such a client cannot synchronize.
As a result, the client must complete a client reset before it can resume synchronization. In a client reset scenario, the client deletes the client-local copy of a realm and downloads the current state of that realm from the backend. Synchronization then resumes using the new copy of the realm.
The client maximum offline time controls how long your backend waits before applying trimming. After the specified number of days without syncing, clients may experience a client reset the next time they connect with the backend.
Applications that do not specify the client maximum offline time never apply trimming. This means that clients can connect after any period of time offline -- weeks, months, or even years -- and synchronize changes. As time passes, frequently-edited realms accumulate many changes. With a large changeset, synchronization requires more time and data usage.
Client Maximum Offline Time Does Not Immediately Influence Client Resets
Trimming causes permanent, irreversible changes to history. As a result, increasing the client maximum offline time does not immediately change the length of time before clients experience a client reset. Existing history has already been changed by trimming, requiring a client reset. New history needs time to accumulate up to the new client maximum offline time.
Disabling the client maximum offline time feature stops additional trimming, but history that has already been changed by trimming will permanently cause client resets in clients.
Decreasing client maximum offline time also does not immediately change the length of time before clients experience a client reset. Client resets begin taking place earlier once the regularly scheduled trimming job applies trimming to the newly eligible history.
Set the Client Maximum Offline Time
Optimizing Performance and Storage When Using Flexible Sync
For Flexible Sync configuration, the amount of Atlas storage space used is directly proportional to the number of queryable fields you have set up. Queryable fields use storage on the backing Atlas cluster. The more queryable fields you configure, the more storage you use on the backing cluster.
If you have a large number of collections in an App, you may need to use the same queryable field name across multiple collections. Combine this with permissions for more granular control over who can access which collections.
Example
Your app may contain 20 or 30 collections, but you want to minimize the
number of queryable fields. You can re-use global queryable fields
across collections in order to sync objects from every collection.
For example, owner_id might be a field you want to query in multiple
collections.
Alternately, you may have owner_id in multiple collections, but only
need to query on it in one collection. In this case, you might make
owner_id a collection queryable field. This means Sync only has to
maintain metadata about this field for one collection, instead of storing
metadata for all of the collections where you're not querying on this
field.
Finally, for Apps where devices want to query one specific facet of the
data, such as owner_id == user.id, you may want to designate the
field an indexed queryable field. Indexed queryable fields provide more
efficient performance for Apps where the client only needs to sync on a
small subset of their data - a group of stores or a single user, for
example.
You can have one indexed queryable field per App. An indexed queryable field is a global queryable field that must be present and use the same eligible data type in each collection you sync.
For more information, refer to Queryable Field Scopes and Indexed Queryable Fields.
For best performance, open a synced realm with a broad query. Then, add more refined queries to expose targeted sets of data in the client application. Slicing off working sets from a broad query provides better performance than opening multiple synced realms using more granular queries.
When you configure queryable fields, consider the broad queries you use for Sync, and select fewer fields that support those broad queries.
Example
In a to-do list app, prefer broad queries such as assignee == currentUser
or projectName == selectedProject for a Sync query. This gives you a
couple of broad fields against which to Sync documents. In the client, you
can further refine your query for things like tasks of a certain priority
or completion status to slice off a working set.
Summary
Device Sync uses space in your synced Atlas cluster to store change history.
Trimming reduces the space usage for Flexible Sync apps, but can cause client resets for clients who have not connected to the backend in more than the client maximum offline time (in days).
Flexible Sync apps that have disabled a client maximum offline time do not apply trimming, so clients of any age can sync without experiencing a client reset.
Adding additional queryable fields to a flexible sync configuration will increase storage consumed on an Atlas Cluster. Using broad queries and selecting fewer fields that support broad queries decreases storage consumed.