Docs Menu
Docs Home
/ /
Atlas App Services
/ /

Optimize Sync Storage in Atlas

On this page

  • History
  • Trimming
  • Client Maximum Offline Time
  • Key Concepts
  • Client Maximum Offline Time Does Not Immediately Influence Client Resets
  • Set the Client Maximum Offline Time
  • Optimizing Performance and Storage When Using Flexible Sync
  • Summary

Atlas Device Sync uses space in your app's synced Atlas cluster to store metadata for sync. This includes a history of changes to each synced database. Atlas App Services minimizes this space usage in your Atlas cluster. Minimizing metadata is necessary to reduce the time and data needed for sync.

The App Services backend keeps a history of changes to underlying data for each realm, similar to the MongoDB oplog. App Services uses this history to synchronize data between the backend and clients. App Services stores history in your synced Atlas cluster.

When you set a client maximum offline time in an App, trimming deletes changes older than the client maximum offline time.

Used in trimming, the client maximum offline time controls the age limit of history. This indirectly changes how long a client can remain offline between synchronization sessions with the backend. Clients that do not synchronize for more than the specified number of days may experience a client reset the next time they connect with the backend.

Setting the client maximum offline time to a lower value will decrease the amount of history required by sync. The resulting optimization lowers storage usage in the synced Atlas cluster.

New Apps automatically enable client maximum offline time with a default value of 30 days.

Warning

Client Maximum Offline Time Causes Permanent Changes to History

Client maximum offline time enables trimming for older history. This permanently changes affected history, and can cause client resets in the future even after disabling the feature.

Sync should always converge at the same end state on all clients. In order to converge during a sync, clients require the full history of changes beginning immediately after their last sync. When a client does not sync for a long period of time, trimming can alter the history in ways that prevent the client from converging. Since synchronization relies on all clients converging on a common result, such a client cannot synchronize.

As a result, the client must complete a client reset before it can resume synchronization. In a client reset scenario, the client deletes the client-local copy of a realm and downloads the current state of that realm from the backend. Synchronization then resumes using the new copy of the realm.

The client maximum offline time controls how long your backend waits before applying trimming. After the specified number of days without syncing, clients may experience a client reset the next time they connect with the backend.

Applications that do not specify the client maximum offline time never apply trimming. This means that clients can connect after any period of time offline -- weeks, months, or even years -- and synchronize changes. As time passes, frequently-edited realms accumulate many changes. With a large changeset, synchronization requires more time and data usage.

Trimming causes permanent, irreversible changes to history. As a result, increasing the client maximum offline time does not immediately change the length of time before clients experience a client reset. Existing history has already been changed by trimming, requiring a client reset. New history needs time to accumulate up to the new client maximum offline time.

Disabling the client maximum offline time feature stops additional trimming, but history that has already been changed by trimming will permanently cause client resets in clients.

Decreasing client maximum offline time also does not immediately change the length of time before clients experience a client reset. Client resets begin taking place earlier once the regularly scheduled trimming job applies trimming to the newly eligible history.

  1. Select the Device Sync menu in the sidebar.

  2. Under Sync Type, make sure Flexible is selected.

  3. Scroll down to the Advanced Configuration (optional) dropdown. Click the dropdown to open.

    The client maximum offline time configuration in a disabled state.
  4. Under the Client Max Offline Time section, click the Enable Max Offline Time toggle.

  5. Specify a number of days for your application's client maximum offline time. The default value is 30 days. The minimum value is 1.

  6. Click the Save Changes button at the bottom of the screen once you are ready to save.

  7. In the confirmation window, click the Save Changes button again to confirm changes.

  1. Pull a local copy of the latest version of your app with the following pull command:

    Pull
    appservices pull --remote="<Your App ID>"
  2. You can configure the number of days for your application's client maximum offline time with the client_max_offline_days property in your app's sync/config.json file:

    ``sync/config.json``
    {
    "client_max_offline_days": 42,
    }
  3. Deploy the updated app configuration with the following push command:

    Push
    appservices push --remote="<Your App ID>"

For Flexible Sync configuration, the amount of Atlas storage space used is directly proportional to the number of queryable fields you have set up. Queryable fields use storage on the backing Atlas cluster. The more queryable fields you configure, the more storage you use on the backing cluster.

If you have a large number of collections in an App, you may need to use the same queryable field name across multiple collections. Combine this with permissions for more granular control over who can access which collections.

Example

Your app may contain 20 or 30 collections, but you want to minimize the number of queryable fields. You can re-use global queryable fields across collections in order to sync objects from every collection. For example, owner_id might be a field you want to query in multiple collections.

Alternately, you may have owner_id in multiple collections, but only need to query on it in one collection. In this case, you might make owner_id a collection queryable field. This means Sync only has to maintain metadata about this field for one collection, instead of storing metadata for all of the collections where you're not querying on this field.

Finally, for Apps where devices want to query one specific facet of the data, such as owner_id == user.id, you may want to designate the field an indexed queryable field. Indexed queryable fields provide more efficient performance for Apps where the client only needs to sync on a small subset of their data - a group of stores or a single user, for example.

You can have one indexed queryable field per App. An indexed queryable field is a global queryable field that must be present and use the same eligible data type in each collection you sync.

For more information, refer to Queryable Field Scopes and Indexed Queryable Fields.

For best performance, open a synced realm with a broad query. Then, add more refined queries to expose targeted sets of data in the client application. Slicing off working sets from a broad query provides better performance than opening multiple synced realms using more granular queries.

When you configure queryable fields, consider the broad queries you use for Sync, and select fewer fields that support those broad queries.

Example

In a to-do list app, prefer broad queries such as assignee == currentUser or projectName == selectedProject for a Sync query. This gives you a couple of broad fields against which to Sync documents. In the client, you can further refine your query for things like tasks of a certain priority or completion status to slice off a working set.

  • Device Sync uses space in your synced Atlas cluster to store change history.

  • Trimming reduces the space usage for Flexible Sync apps, but can cause client resets for clients who have not connected to the backend in more than the client maximum offline time (in days).

  • Flexible Sync apps that have disabled a client maximum offline time do not apply trimming, so clients of any age can sync without experiencing a client reset.

  • Adding additional queryable fields to a flexible sync configuration will increase storage consumed on an Atlas Cluster. Using broad queries and selecting fewer fields that support broad queries decreases storage consumed.

Back

Production Load Testing

Next

Compact an Atlas Volume