On this page
- Key Concepts
- Operational Transformation
- Client File Identifier
- Network Security
- Sync Session Process
- Realm SDK Connects to the App Server
- Realm SDK Initiates a Sync Session
- App Services Allocates a New Client File Identifier
- Realm SDK Sends a Client Identifier
- Realm SDK Uploads & Downloads Sync Changesets
- Realm SDK Terminates the Sync Session
- Request Types
- Client -> Server Messages
- Server -> Client Messages
Atlas Device Sync uses a protocol to correctly and efficiently sync data changes in real time across multiple clients that each maintain their own local Device Sync files. The protocol defines a set of pre-defined request types as well as a process by which a client, like a Realm SDK, can connect to an App Services application server and sync data.
The a Realm SDKs internally implement and manage the sync protocol, so for most applications you don't need to understand the sync protocol to use Device Sync. This page covers the protocol at a high level and is not an implementation spec.
A changeset is a list of instructions that describe granular modifications made to a known object state or version by one or more write operations. Changesets are the base unit of the sync protocol. Synced realm clients send changesets to the App Services server whenever they perform a write operation. The server sends each connected client the changesets for write operations executed by other clients.
The App Services sync server accepts changesets from any connected sync client (including changes in a synced MongoDB cluster) at any time and uses an operational transformation algorithm to serialize changes into a linear order and resolve conflicting changesets before sending them to connected clients.
When you make a change to a synced object, App Services does not re-upload the entire object. Instead, App Services sends only the difference ("delta") between before and after. The service compresses the deltas with zlib compression. This reduces network load, which is especially useful in mobile network conditions.
An operational transformation is a function that, given two changesets, produces a third changeset that represents logically applying one of the given changesets after the other. Device Sync uses operational transformation to resolve conflicts between changesets from different sync clients that apply to the same base state.
Realm is an offline-first local database even when sync is enabled, which means that any device may perform offline writes and upload the corresponding changesets later when network connectivity is re-established. The operational transformation algorithm is designed to gracefully handle changesets that arrive "out of order" with respect to the logical server clock such that every synced Realm file converges to the same version of each changed object.
An operational transformation on Realm changesets is analogous to a rebase operation in Git.
A client file identifier is a value that uniquely identifies a synced client Realm file and its corresponding server file. The server generates a client file identifier whenever an SDK requests one during its initial sync of a Realm file. Each identifier is a 64-bit, non-zero, positive signed integer strictly less than 2^63.
The server guarantees that all identifiers generated on behalf of a particular server file are unique with respect to each other. The server is free to generate identical identifiers for two client files if they are associated with different server files.
The SDK synchronizes with the application server over a WebSocket connection secured by HTTPS using TLS 1.3.
To initiate, execute, and terminate an App Services Database sync session, a Realm SDK and application server send and receive a set of protocol-specific requests.
The SDK negotiates a WebSocket connection over HTTP and then establishes a sync
session by sending
requests to the server over the WebSocket connection.
Once the session is established, the SDK and server send synced changesets for a given
Realm file to each other via
DOWNLOAD messages. To end the session, the SDK sends an
Realm SDK App Server | | | <---- 1. HTTP Handshake -----> | | | | --------- 2. BIND -----------> | | | | <-- 3. IDENT (first time) ---- | | | | --------- 4. IDENT ----------> | | | | <---- 5. UPLOAD/DOWNLOAD ----> | | | | --------- 6. UNBIND ---------> |
The sync protocol is primarily handled over a WebSocket connection between the SDK and the server. To establish a connection, the SDK sends a handshake HTTP request that includes the following:
- a protocol version
- a WebSocket key
- a valid access token for an authenticated App Services application user
The server sends an HTTP 101 Switching Protocols response that specifies a WebSocket connection for the SDK. The rest of the sync protocol occurs over this connection.
To begin a sync session, a Realm SDK sends a
request to an Atlas App Services application server. The request identifies a
specific local Realm Database file to sync and includes a WebSocket connection key that
the server will use to open a bidirectional connection to the SDK.
If the SDK is attempting to sync a particular Realm Database file for the first time,
it does not yet possess a server-generated client identifier for the file. In
this case, the
BIND request also indicates that the
App Services server should allocate one.
BIND request indicates that the SDK needs a client
file identifier, the App Services server generates a unique value for the specified
Realm Database file and sends it to the SDK in an
response. When the SDK receives the
IDENT, it stores
the new client identifier persistently in the local Realm Database file.
An SDK only needs to request a client file identifier the first time it syncs each Realm Database file. For subsequent sync sessions, the SDK can use the persisted identifier.
Once an SDK has initiated a sync session with a
request, it must identify the local Realm Database file that it wants to sync. To do
this, the SDK sends the application server an
message that contains the client file identifier. If the SDK has previously
synced the realm with the server, it can specify the most recently synced
server version to optimize the sync process.
When it receives the
IDENT message, the server
establishes the session. The SDK and server can can now freely send upload and
download sync changesets at any time.
When the server receives an
UPLOAD message, it applies
operational transformations to resolve
any conflicts with other changesets and then applies the transformed changeset
to the server version of the realm. This triggers the server to send
DOWNLOAD messages to other connected clients, including
the synced Atlas cluster which mirrors the server realm. A
DOWNLOAD message groups one or more transformed
changesets in chronological order from oldest to most recent according to the
server's history. The SDK applies the changesets in the same order.
The following table describes the request types that a sync client can send to an App Services server:
Starts a new sync session on the server and provides a signed authorization token for the current application user. If the client does not yet possess a client file identifier for the Realm file it wants to sync, this also indicates that the server should generate one and send it back to the client.
A client must send a BIND before it can send any other requests.
Provides the client file identifier that indicates the following:
This request is related to but distinct from the
Specifies one or more changesets for operations that occurred on the client. The changesets are listed by client version in increasing order.
Specifies a changeset that describes a serialized transaction that occurred on the client. The client may not upload any other changesets until the server confirms or rejects the transaction.
Ends a running sync session.
A client may not send any other requests for an
Requests that the server notify the client when it has synced the latest changeset in the server history (at the time of the request).
Re-authorizes a current sync session with a new user token.
Requests that the server send one or more
Requests that the server send the client version of the latest changeset that was sent by the client and processed by the server. This is most commonly used when an SDK executes a client reset.
Indicates that the client is still connected and that the server should
maintain the sync session. A client must send at least one PING to the
server every 10 minutes. The server acknowledges to each PING with a
If the server has not received a PING from a client in more than 10 minutes, it considers the client to be disconnected and may automatically end the session.
The following table describes the request types that the App Services server can send to a sync client:
Specifies one or more changesets (up to 16MB total) for operations that occurred on other clients. The changesets are listed by server version in increasing order.
The changesets in a DOWNLOAD may not be the exact changesets uploaded by other clients. Instead, they may be equivalent changesets output by Device Sync's operational transformation algorithm.
Specifies that the server ended a sync session in response to an
Indicates whether or not the server successfully processed a changeset specified in a
Contains one or more segments of encoded data that the client can concatenate to construct the latest server version of the realm. Sent in response to a
Specifies the client version of the latest changeset that was sent by the client and processed by the server. Sent in response to a
Indicates that the server encountered an issue that appears to have been caused by the connected client. For details, see Sync Client Errors.