Docs Home → Atlas App Services
Optimize Sync Atlas Usage
On this page
Overview
Atlas Device Sync uses space in your app's synced Atlas cluster to store metadata for sync. This includes a history of changes to each realm. Atlas App Services minimizes this space usage in your Atlas cluster via trimming for Flexible Sync apps or backend compaction for Partition Based Sync apps. Minimizing metadata is necessary to reduce the time and data needed for sync.
History
The App Services backend keeps a history of changes to underlying data for each realm, similar to the MongoDB oplog. App Services uses this history to synchronize data between the backend and clients. App Services stores history in your synced Atlas cluster.
Optimization Methods
There are two methods for optimizing utility data synced in your cluster depending on whether your App uses Flexible or Partition-Based Sync:
Trimming: Used for Flexible Sync apps.
Backend Compaction: Used for Partition Based Sync apps.
Trimming
When you set a client maximum offline time in an App that uses Flexible Sync, trimming deletes changes older than the client maximum offline time.
Note
Flexible Sync uses trimming instead of backend compaction.
Backend Compaction
Backend compaction is a maintenance process that automatically runs for all apps using Partition Based Sync. Apps using Partition Based Sync store the entire history of data changes. Using compaction, the backend optimizes a realm's history by removing unneeded changesets. The process removes any instruction whose effect is later overwritten by a newer instruction.
Example
Consider the following realm history:
CREATE table1.object1 UPDATE table1.object1.x = 4 UPDATE table1.object1.x = 10 UPDATE table1.object1.x = 42
The following history would also converge to the same state, but without unneeded interim changes:
CREATE table1.object1 UPDATE table1.object1.x = 42
Note
Partition Based Sync uses backend compaction rather than trimming and client maximum offline time.
Client Maximum Offline Time
Used in trimming, the client maximum offline time controls the age limit of history. This indirectly changes how long a client can remain offline between synchronization sessions with the backend. Clients that do not synchronize for more than the specified number of days may experience a client reset the next time they connect with the backend.
Setting the client maximum offline time to a lower value will decrease the amount of history required by sync. The resulting optimization lowers storage usage in the synced Atlas cluster.
New Apps automatically enable client maximum offline time with a default value of 30 days.
Warning
Client Maximum Offline Time Causes Permanent Changes to History
Client maximum offline time enables trimming for older history. This permanently changes affected history, and can cause client resets in the future even after disabling the feature.
Key Concepts
Sync should always converge at the same end state on all clients. In order to converge during a sync, clients require the full history of changes beginning immediately after their last sync. When a client does not sync for a long period of time, trimming can alter the history in ways that prevent the client from converging. Since synchronization relies on all clients converging on a common result, such a client cannot synchronize.
As a result, the client must complete a client reset before it can resume synchronization. In a client reset scenario, the client deletes the client-local copy of a realm and downloads the current state of that realm from the backend. Synchronization then resumes using the new copy of the realm.
The client maximum offline time controls how long your backend waits before applying trimming. After the specified number of days without syncing, clients may experience a client reset the next time they connect with the backend.
Applications that do not specify the client maximum offline time never apply trimming. This means that clients can connect after any period of time offline -- weeks, months, or even years -- and synchronize changes. As time passes, frequently-edited realms accumulate many changes. With a large changeset, synchronization requires more time and data usage.
Client Maximum Offline Time Does Not Immediately Influence Client Resets
Trimming causes permanent, irreversible changes to history. As a result, increasing the client maximum offline time does not immediately change the length of time before clients experience a client reset. Existing history has already been changed by trimming, requiring a client reset. New history needs time to accumulate up to the new client maximum offline time.
Disabling the client maximum offline time feature stops additional trimming, but history that has already been changed by trimming will permanently cause client resets in clients.
Decreasing client maximum offline time also does not immediately change the length of time before clients experience a client reset. Client resets begin taking place earlier once the regularly scheduled trimming job applies trimming to the newly eligible history.
Set the Client Maximum Offline Time
Optimizing Performance and Storage When Using Flexible Sync
For Flexible Sync configuration, the amount of Atlas storage space used is directly proportional to the number of queryable fields you have set up. Queryable fields use storage on the backing Atlas cluster. The more queryable fields you configure, the more storage you use on the backing cluster.
If you have a large number of collections in an App, you may need to use the same queryable field name across multiple collections. Combine this with permissions for more granular control over who can access which collections.
Example
Your app may contain 20 or 30 collections, but you want to minimize the number of queryable fields. You can re-use queryable fields across collections in order to sync objects from every collection. For example, "owner_id" might be a field you want to query in multiple collections, but by having the same field name across collections, only one queryable field is used.
For best performance, open a synced realm with a broad query. Then, add more refined queries to expose targeted sets of data in the client application. Slicing off working sets from a broad query provides better performance than opening multiple synced realms using more granular queries.
When you configure queryable fields, consider the broad queries you use for Sync, and select fewer fields that support those broad queries.
Example
In a to-do list app, prefer broad queries such as assignee == currentUser
or projectName == selectedProject for a Sync query. This gives you a
couple of broad fields against which to Sync documents. In the client, you
can further refine your query for things like tasks of a certain priority
or completion status to slice off a working set.
Summary
Device Sync uses space in your synced Atlas cluster to store change history.
Backend compaction reduces the space usage of this history for Partition Based Sync apps.
Trimming reduces the space usage for Flexible Sync apps, but can cause client resets for clients who have not connected to the backend in more than the client maximum offline time (in days).
Flexible Sync apps that have disabled a client maximum offline time do not apply trimming, so clients of any age can sync without experiencing a client reset.
Adding additional queryable fields to a flexible sync configuration will increase storage consumed on an Atlas Cluster. Using broad queries and selecting fewer fields that support broad queries decreases storage consumed.