Ian Ward

## Introduction

When MongoDB acquired Realm in 2019, we knew we wanted to give developers the easiest and fastest way to synchronize data on-device with a backend in the cloud.

:youtube[]{vid=6WrQ-f0dcIA}

In an offline-first environment, edge-to-cloud data sync typically requires thousands of lines of complex conflict resolution and networking code, and leaves developers with code bloat that slows the development of new features in the long-term.MongoDB Realm’s sync simplifies moving data between the Realm Mobile Database and MongoDB Atlas. With huge amounts of boilerplate code eliminated, teams are able to focus on the features that drive 5-star app reviews and happy users. 

Since bringing Realm Sync GA in February 2021, we’ve seen it transform the way developers are building data synchronization into their mobile applications. But we’ve also seen developers creating workarounds for complex sync use cases. With that in mind, we’ve been hard at work building the next iteration of Realm Sync, which we’re calling Realm Flexible Sync.

Realm Flexible Sync takes into account a year’s worth of user feedback on partition-based sync, and aims to make syncing data to MongoDB Atlas a simple and idiomatic process by using a client-defined query to define the data synced to user applications.

## How Flexible Sync Works

Flexible Sync lets developers start writing code that syncs data more quickly – allowing you to choose which data is synced via a language-native query and to change the queries that define your syncing data at any time.

With Flexible Sync, developers can enable devices to define a query on the client side using the Realm SDK’s query-language, which will execute on MongoDB Atlas to identify the set of data to Sync. Any documents that match the query will be translated to Realm Objects and saved to the client device’s local disk. The query will be maintained on the server, which will check in real-time to identify if new document insertions, updates, or deletions on Atlas change the query results. Relevant changes on the server-side will be replicated down to the client in real-time, and any changes from the client will be similarly replicated to Atlas.

## New Capabilities

Flexible Sync is distinctly different from the [partition-based sync](https://docs.mongodb.com/realm/sync/partitioning/#std-label-partitioning) used by Realm Sync today. 

With partition-based sync, developers must configure a partition field for their Atlas database. This partition field lives on each document within the Atlas database that the operator wants to sync. Clients can then request access to different partitions of the Atlas database, using the different values of the partition key field. When a client opens a synchronized Realm they pass in the partition key value as a parameter. The sync server receives the value from the client, and sends any documents down to the client that match the partition key value. These documents are automatically translated as Realm Objects and stored on the client’s disk for offline access. 

Partition-based sync works well for applications where data is static and compartmentalized, and where permissions models rarely need to change. With Flexible Sync, we’re making fine-grained and flexible permissioning possible, and opening up new app use cases through simplifying the syncing of data that requires ranged or dynamic queries.

## Flexible Permissions

Unlike with partition-based sync, Flexible Sync makes it seamless to implement the [document-level permission model](https://docs.mongodb.com/realm/mongodb/define-roles-and-permissions/) when syncing data - meaning synced fields can be limited based on a user’s role. We expect this to be available at beta, and with field-level permissions coming after that.

Consider a healthcare app, with different field-level permissions for Patients, Doctors, and Administrative staff using the application. A patient collection contains user data about the patient, their health history, procedures undergone, and prognosis. The patient accessing the app would only be able to see their full healthcare history, along with their own personal information. Meanwhile, a doctor using the app would be able to see any patients assigned to their care, along with healthcare history and prognosis. But doctors viewing patient data would be unable to view certain personal identifying information, like social security numbers. Administrative staff who handle billing would have another set of field-level permissions, seeing only the data required to successfully bill the patient. 

Under the hood, this is made possible when Flexible Sync runs the query sent by the client, obtains the result set, and then subtracts any data from the result set sent down to the client based on the permissions. The server guards against clients receiving data they aren’t allowed to see, and developers can trust that the server will enforce compliance, even if a query is written with mistakes. In this way, Flexible Sync simplifies sharing subsets of data across groups of users and makes it easier for your application's permissions to mirror complex organizations and business requirements.

Flexible Sync also allows clients to share some documents but not others, based on the ResultSet of their query. Consider a company where teams typically share all the data within their respective teams, but not across teams. When a new project requires teams to collaborate, Flexible Sync makes this easy. The shared project documents could have a field called allowedTeams: [ marketing, sales]. Each member of the team would have a client-side query, searching for all documents on allowedTeams matching marketing or sales using an $in operator, depending on what team that user was a member of.

## Ranged & Dynamic Queries

One of Flexible Sync's primary benefits is that it allows for simple synchronization of data that falls into a range – such as a time window – and automatically adds and removes documents as they fall in and out of range.   

Consider an app used by a company’s workforce, where the users only need to see the last seven days of work orders. With partition-based sync, a time-based trigger needed to fire daily to move work orders in and out of the relevant partition. With Flexible Sync, a developer can write a ranged query that automatically includes and removes data as time passes and the 7-day window changes. By adding a time based range component to the query, code is streamlined. The sync resultset gets a built-in TTL, which previously had to be implemented by the operator on the server-side. 

Flexible Sync also enables much more dynamic queries, based on user inputs. Consider a shopping app with millions of products in its Inventory collection. As users apply filters in the app – viewing only pants that are under $30 dollars and size large – the query parameters can be combined with logical ANDs and ORs to produce increasingly complex queries, and narrow down the search result even further. All of these query results are combined into a single realm file on the client’s device, which significantly simplifies code required on the client-side. 

## Looking Ahead

Ultimately, our decision to build Flexible Sync is driven by the Realm team’s desire to eliminate every possible piece of boilerplate code for developers. We’re motivated by delivering a sync service that can fit any use case or schema design pattern you can imagine, so that you can spend your time building features rather than implementing workarounds. 

The Flexible Sync project represents the next evolution of Realm Sync. We’re working hard to get to a public beta by the end of 2021, and believe this query-based sync has the potential to become the standard for Realm Sync enabled applications. We won’t have every feature available on day one, but iterative releases over the course of 2022 will continuously bring you more query operators and permissions integrations.

Interested in joining the early access program? [Sign-up here](https://www.mongodb.com/realm/opt-in) and we’ll let you know when Realm Flexible Sync is available in beta. 



Flexible Sync lets developers start writing code that syncs data more quickly – allowing you to choose which data is synced via a language-native query and to change the queries that define your syncing data at any time.

A Preview of Realm Flexible Sync

Andrew Morgan


## Introduction

In the [previous post in this series](https://www.mongodb.com/developer/how-to/realm-schema-migration), we saw how to migrate your Realm data when you upgraded your iOS app with a new schema. But, that only handled the data in your local, standalone Realm database. What if you're using [MongoDB Realm Sync](https://docs.mongodb.com/realm/sync/) to replicate your local Realm data with other instances of your mobile app and with MongoDB Atlas? That's what this article will focus on.

We'll start with the original [RChat app](https://www.mongodb.com/developer/how-to/building-a-mobile-chat-app-using-realm-new-way/). We'll then extend the iOS app and backend Realm schema to add a new feature that allows chat messages to be tagged as high priority. The next (and perhaps surprisingly more complicated from a Realm perspective) upgrade is to make the `author` attribute of the existing `ChatMessage` object non-optional.

You can find all of the code for this post in the RChat repo under these branches:

- [Starting point](https://github.com/realm/RChat)
- [Upgrade #1](https://github.com/realm/RChat/tree/new-schema)
- [Upgrade #2](https://github.com/realm/RChat/tree/v2-schema)

## Prerequisites

[Realm Cocoa 10.13.0](https://github.com/realm/realm-cocoa/releases) or later (for versions of the app that you're upgrading **to**)

## Catch-Up — The RChat App

RChat is a basic chat app:

- Users can register and log in using their email address and a password.
- Users can create chat rooms and include other users in those rooms.
- Users can post messages to a chat room (optionally including their location and photos).
- All members of a chatroom can see messages sent to the room by themselves or other users.

:youtube[Existing RChat iOS app functionality]{vid=BlV9El_MJqk}

## Upgrade #1: Add a High-Priority Flag to Chat Messages

The first update is to allow a user to tag a message as being high-priority as they post it to the chat room:

![Screenshot showing the option to click a thermometer button to tag the message as urgent](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Set_Priority_a2c8f60550.png)

That message is then highlighted with bold text and a "hot" icon in the list of chat messages:

![Screenshot showing that a high-priority message has bold text and a hot thermometer icon](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/View_Priority_1991c9d4bf.png)

### Updating the Backend Realm Schema

Adding a new field is an [additive change](https://docs.mongodb.com/realm/sync/synced-schema-overview/)—meaning that you don't need to restart sync (which would require every deployed instance of the RChat mobile app to recognize the change and start sync from scratch, potentially losing local changes).

We add the new `isHighPriority` bool to our Realm schema through the Realm UI:

![Screenshot from the RealmUI showing that the isHighPriority bool has been added to the schema](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Update_Schema_v1_b9ff24b45c.png)

We also make `isHighPriority` a required (non-optional field).

The resulting schema looks like this:

```js
{
   "bsonType": "object",
   "properties": {
     "_id": {
       "bsonType": "string"
     },
     "author": {
       "bsonType": "string"
     },
     "image": {
       "bsonType": "object",
       "properties": {
         "_id": {
           "bsonType": "string"
         },
         "date": {
           "bsonType": "date"
         },
         "picture": {
           "bsonType": "binData"
         },
         "thumbNail": {
           "bsonType": "binData"
         }
       },
       "required": [
         "_id",
         "date"
       ],
       "title": "Photo"
     },
     "isHighPriority": {
       "bsonType": "bool"
     },
     "location": {
       "bsonType": "array",
       "items": {
         "bsonType": "double"
       }
     },
     "partition": {
       "bsonType": "string"
     },
     "text": {
       "bsonType": "string"
     },
     "timestamp": {
       "bsonType": "date"
     }
   },
   "required": [
     "_id",
     "partition",
     "text",
     "timestamp",
     "isHighPriority"
   ],
   "title": "ChatMessage"
 }
```
Note that existing versions of our iOS RChat app can continue to work with our updated backend Realm app, even though their local `ChatMessage` Realm objects don't include the new field.

### Updating the iOS RChat App

While existing versions of the iOS RChat app can continue to work with the updated Realm backend app, they can't use the new `isHighPriority` field as it isn't part of the `ChatMessage` object.

To add the new feature, we need to update the mobile app after deploying the updated Realm backend application.

The first change is to add the `isHighPriority` field to the `ChatMessage` class:

```swift
class ChatMessage: Object, ObjectKeyIdentifiable {
   @Persisted(primaryKey: true) var _id = UUID().uuidString
   @Persisted var partition = "" // "conversation=<conversation-id>"
   @Persisted var author: String? // username
   @Persisted var text = ""
   @Persisted var image: Photo?
   @Persisted var location = List<Double>()
   @Persisted var timestamp = Date()
   @Persisted var isHighPriority = false
   ...
}
```

As seen in the [previous post in this series](https://www.mongodb.com/developer/how-to/realm-schema-migration), Realm can automatically update the local realm to include this new attribute and initialize it to `false`. Unlike with standalone realms, we **don't** need to signal to the Realm SDK that we've updated the schema by providing a schema version.

The new version of the app will happily exchange messages with instances of the original app on other devices (via our updated backend Realm app).

## Upgrade #2: Make `author` a Non-Optional Chat Message field

When the initial version of RChat was written, the `author` field of `ChatMessage` was declared as being optional. We've since realized that there are no scenarios where we wouldn't want the author included in a chat message. To make sure that no existing or future client apps neglect to include the author, we need to update our schema to make `author` a required field.

Unfortunately, changing a field from optional to required (or vice versa) is a [destructive change](https://docs.mongodb.com/realm/sync/synced-schema-overview/#destructive-changes), and so would break sync for any deployed instances of the RChat app.

Oops!

This means that there's extra work needed to make the upgrade seamless for the end users. We'll go through the process now.

### Updating the Backend Realm Schema

The change we need to make to the schema is destructive. This means that the new document schema is incompatible with the schema that's currently being used in our mobile app.

If RChat wasn't already deployed on the devices of hundreds of millions of users (we can dream!), then we could update the Realm schema for the `ChatMessage` collection and restart Realm Sync. During development, we can simply remove the original RChat mobile app and then install an updated version on our test devices.

To avoid that trauma for our end users, we leave the `ChatMessage` collection's schema as is and create a [partner collection](https://docs.mongodb.com/realm/sync/migrate-schema-partner-collection/#partner-collections). The partner collection (`ChatMessageV2`) will contain the same data as `ChatMessage`, except that its schema makes `author` a required field.

These are the steps we'll go through to create the partner collection:

- Define a Realm schema for the `ChatMessageV2` collection.
- Run an [aggregation](https://docs.mongodb.com/manual/aggregation/) to copy all of the documents from `ChatMessage` to `ChatMessageV2`. If `author` is missing from a `ChatMessage` document, then the aggregation will add it.
- Add a trigger to the `ChatMessage` collection to propagate any changes to `ChatMessageV2` (adding `author` if needed).
- Add a trigger to the `ChatMessageV2` collection to propagate any changes to `ChatMessage`.

#### Define the Schema for the Partner Collection

From the Realm UI, copy the schema from the `ChatMessage` collection. 

Click the button to create a new schema:

![Showing a plus button in the Realm UI to add a new collection](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/add_collection_c16ec9a063.png)

Set the database and collection name before clicking "Add Collection":

![Setting the database and collection names in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/config_collection_324bc0182e.png)

Paste in the schema copied from `ChatMessage`, add `author` to the `required` section, change the `title` to `ChatMessageV2`, and the click the "SAVE" button:

![Adding "author" to the required attribute list and naming the class ChatMessageV2 in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/define_schema_be6ae7c555.png)

This is the resulting schema:

```js
{
   "bsonType": "object",
   "properties": {
     "_id": {
       "bsonType": "string"
     },
     "author": {
       "bsonType": "string"
     },
     "image": {
       "bsonType": "object",
       "properties": {
         "_id": {
           "bsonType": "string"
         },
         "date": {
           "bsonType": "date"
         },
         "picture": {
           "bsonType": "binData"
         },
         "thumbNail": {
           "bsonType": "binData"
         }
       },
       "required": [
         "_id",
         "date"
       ],
       "title": "Photo"
     },
     "isHighPriority": {
       "bsonType": "bool"
     },
     "location": {
       "bsonType": "array",
       "items": {
         "bsonType": "double"
       }
     },
     "partition": {
       "bsonType": "string"
     },
     "text": {
       "bsonType": "string"
     },
     "timestamp": {
       "bsonType": "date"
     }
   },
   "required": [
     "_id",
     "partition",
     "text",
     "timestamp",
     "isHighPriority",
     "author"
   ],
   "title": "ChatMessageV2"
 }
```

#### Copy Existing Data to the Partner Collection

We're going to use an [aggregation pipeline](https://docs.mongodb.com/manual/aggregation/) to copy and transform the existing data from the original collection (`ChatMessage`) to the partner collection (`ChatMessageV2`).

You may want to pause sync just before you run the aggregation, and then unpause it after you enable the trigger on the `ChatMessage` collection in the next step:

![Pressing a button to pause Realm sync in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/pause_sync_707605a511.png)

The end users can continue to create new messages while sync is paused, but those messages won't be published to other users until sync is resumed. By pausing sync, you can ensure that all new messages will make it into the partner collection (and so be visible to users running the new version of the mobile app).

If pausing sync is too much of an inconvenience, then you could create a temporary trigger on the `ChatMessage` collection that will copy and transform document inserts to the `ChatMessageV2` collection (it's a subset of the `ChatMessageProp` trigger we'll define in the next section.).

From the Atlas UI, select "Collections" -> "ChatMessage", "New Pipeline From Text":

![Navigating through "Atlas/ChatMessage/Collections/New Pipeline from Text" in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/add_aggregation_2e84c4a8df.png)

Paste in this aggregation pipeline and click the "Create New" button:

```js
[
 {
   '$addFields': {
     'author': {
       '$convert': {
         'input': '$author',
         'to': 'string',
         'onError': 'unknown',
         'onNull': 'unknown'
       }
     }
   }
 },
 {
   '$merge': {
     into: "ChatMessageV2",
     on: "_id",
     whenMatched: "replace",
     whenNotMatched: "insert"
   }
 }
]
```

This aggregation will take each `ChatMessage` document, set `author` to "unknown" if it's not already set, and then add it to the `ChatMessageV2` collection.

Click "MERGE DOCUMENTS":

![Clicking the "Merge Documents" button in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/merge_documents_8b40ae6ce6.png)

`ChatMessageV2` now contains a (possibly transformed) copy of every document from `ChatMessage`. But, changes to one collection won't be propagated to the other. To address that, we add a database trigger to each collection…

#### Add Database Triggers

We need to create two [Realm Functions](https://docs.mongodb.com/realm/functions/)—one to copy/transfer documents to `ChatMessageV2`, and one to copy documents to `ChatMessage`.

From the "Functions" section of the Realm UI, click "Create New Function":

![Clicking the "Create New Function" button in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/new_function_3fd3b90a11.png)

Name the function `copyToChatMessageV2`. Set the authentication method to "System"—this will circumvent any access permissions on the `ChatMessageV2` collection. Ensure that the "Private" switch is turned on—that means that the function can be called from a trigger, but not directly from a frontend app. Click "Save":

![Set the name to "copyToChatMessageV2", authentication to "System" and "Private" to "On". Then click the "Save" button in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/configure_function_6acb29f42b.png)

Paste this code into the function editor and save:

```js
exports = function (changeEvent) {
   const db = context.services.get("mongodb-atlas").db("RChat");


   if (changeEvent.operationType === "delete") {
     return db.collection("ChatMessageV2").deleteOne({ _id: changeEvent.documentKey._id });
   }


   const author = changeEvent.fullDocument.author ? changeEvent.fullDocument.author : "Unknown";
   const pipeline = [
     { $match: { _id: changeEvent.documentKey._id } },
     {
       $addFields: {
         author: author,
       }
     },
     { $merge: "ChatMessageV2" }];


   return db.collection("ChatMessage").aggregate(pipeline);
};
```

This function will receive a `ChatMessage` document from our trigger. If the operation that triggered the function is a delete, then this function deletes the matching document from `ChatMessageV2`. Otherwise, the function either copies `author` from the incoming document or sets it to "Unknown" before writing the transformed document to `ChatMessageV2`. We could initialize `author` to any string, but I've used "Unknown" to tell the user that we don't know who the author was.

Create the `copyToChatMessage` function in the same way:

```js
exports = function (changeEvent) {
   const db = context.services.get("mongodb-atlas").db("RChat");


   if (changeEvent.operationType === "delete") {
     return db.collection("ChatMessage").deleteOne({ _id: changeEvent.documentKey._id })
   }
    const pipeline = [
     { $match: { _id: changeEvent.documentKey._id } },
     { $merge: "ChatMessage" }]
   return db.collection("ChatMessageV2").aggregate(pipeline);
};
```

The final change needed to the backend Realm application is to add database triggers that invoke these functions.

From the "Triggers" section of the Realm UI, click "Add a Trigger":

![Click the "Add a Trigger" button in the Realm UI](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/configure_trigger_558a629423.png)

Configure the `ChatMessageProp` trigger as shown:

![In the Realm UI, set "Trigger Type" to "Database". Set "Name" to "ChatMessageProp". Enabled = On. Event Ordering = on. Cluster Name = Cluster 0. Database name = RChat. Collection name = ChatMessage. Check all operation types. Full Document = on. Document Preimage = off. Event Type = Function. Function = copyToChatMessageV2](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/trigger_settings_1601e02011.png)

Repeat for `ChatMessageV2Change`:

![In the Realm UI, set "Trigger Type" to "Database". Set "Name" to "ChatMessageProp". Enabled = On. Event Ordering = on. Cluster Name = Cluster 0. Database name = RChat. Collection name = ChatMessage. Check all operation types. Full Document = off. Document Preimage = off. Event Type = Function. Function = copyToChatMessage](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/trigger2_settings_c55de55738.png)

If you paused sync in the previous section, then you can now unpause it.

### Updating the iOS RChat App

We want to ensure that users still running the old version of the app can continue to exchange messages with users running the latest version.

Existing versions of RChat will continue to work. They will create `ChatMessage` objects which will get synced to the `ChatMessage` Atlas collection. The database triggers will then copy/transform the document to the `ChatMessageV2` collection.

We now need to create a new version of the app that works with documents from the `ChatMessageV2` collection. We'll cover that in this section.

Recall that we set `title` to `ChatMessageV2` in the partner collection's schema. That means that to sync with that collection, we need to rename the `ChatMessage` class to `ChatMessageV2` in the iOS app. 

Changing the name of the class throughout the app is made trivial by Xcode.

Open `ChatMessage.swift` and right-click on the class name (`ChatMessage`), select "Refactor" and then "Rename…":

![In Xcode, select the "ChatMessage" class name, right-click and select "Refactor -> Rename..."](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/refactor1_4c2d938e3d.png)

Override the class name with `ChatMessageV2` and click "Rename":

![In Xcode, overwrite the class name with "ChatMessageV2" and then click the "Rename" button](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/refactor2_03837b2472.png)

The final step is to make the author field mandatory. Remove the ? from the author attribute to make it non-optional:

```swift
class ChatMessageV2: Object, ObjectKeyIdentifiable {
   @Persisted(primaryKey: true) var _id = UUID().uuidString
   @Persisted var partition = "" // "conversation=<conversation-id>"
   @Persisted var author: String
   ...
}
```

## Conclusion

Modifying a Realm schema is a little more complicated when you're using Realm Sync for a deployed app. You'll have end users who are using older versions of the schema, and those apps need to continue to work.

Fortunately, the most common schema changes (adding or removing fields) are additive. They simply require updates to the back end and iOS schema, together.

Things get a little trickier for destructive changes, such as changing the type or optionality of an existing field. For these cases, you need to create and maintain a partner collection to avoid loss of data or service for your users.

This article has stepped through how to handle both additive and destructive schema changes, allowing you to add new features or fix issues in your apps without impacting users running older versions of your app.

Remember, you can find all of the code for this post in the RChat repo under these branches:

- [Starting point](https://github.com/realm/RChat)
- [Upgrade #1](https://github.com/realm/RChat/tree/new-schema)
- [Upgrade #2](https://github.com/realm/RChat/tree/v2-schema)

If you're looking to upgrade the Realm schema for an iOS app that **isn't** using Realm Sync, then refer to the [previous post in this series](https://www.mongodb.com/developer/how-to/realm-schema-migration).

If you have any questions or comments on this post (or anything else Realm-related), then please raise them on our [community forum](https://www.mongodb.com/community/forums/c/realm-sdks/58). To keep up with the latest Realm news, follow [@realm](https://twitter.com/realm) on Twitter and join the [Realm global community](https://live.mongodb.com/realm-global-community/).


When you add features to your app, you may need to modify your Realm schema. Here, we step through how to migrate your synced schema and data.

Migrating Your iOS App's Synced Realm Schema in Production

Dominic Frei

This article shows how to migrate from using a local Realm to MongoDB Realm Sync. We will cover everything you need to know to transform your game into a multiplayer experience.

Turning Your Local Game into an Online Experience with MongoDB Realm Sync

Shane McAllister

Missed Realm Sync in use - building and architecting a Mobile Chat App meetup event? Don't worry, you can catch up here.

Realm Sync in Use - Building and Architecting a Mobile Chat App Meetup

Today, we are excited to announce the public preview of our next version of Realm Sync: Flexible Sync. This new method of syncing puts the power into the hands of the developer. Now, developers can get more granular control over the data synced to user applications with intuitive language-native queries and hierarchical permissions.

:youtube[]{vid=aJ6TI1mc7Bs}

## Introduction 

Prior to launching the general availability of Realm Sync in February 2021, the Realm team spent countless hours with developers learning how they build best-in-class mobile applications. A common theme emerged—building real-time, offline-first mobile apps require an overwhelming amount of complex, non-differentiating work. 

Our [first version of Realm Sync](https://www.mongodb.com/developer/how-to/realm-partitioning-strategies/) addressed this pain by abstracting away offline-first, real-time syncing functionality using declarative APIs. It expedited the time-to-market for many developers and worked well for apps where data is static and compartmentalized, or where permissions rarely need to change. But for dynamic apps and complex use cases, developers still had to spend time creating workarounds instead of developing new features. With that in mind, we built the next iteration of Realm Sync: Flexible Sync. Flexible Sync is designed to help developers: 

- Get to market faster: Use intuitive, language-native queries to define the data synced to user applications instead of proprietary concepts.
- Optimize real-time collaboration between users: Utilize object-level conflict-resolution logic.
- Simplify permissions: Apply role-based logic to applications with an expressive permissions system that groups users into roles on a pe-class or collection basis.

Flexible Sync requires MongoDB 5.0+.

## Language-Native Querying

Flexible Sync’s query-based sync logic is distinctly different from how Realm Sync operates today. The new structure is designed to more closely mirror how developers are used to building sync today—typically using GET requests with query parameters. 

One of the primary benefits of Flexible Sync is that it eliminates all the time developers spend determining what query parameters to pass to an endpoint. Instead, the Realm APIs directly integrate with the native querying system on the developer’s choice of platform—for example, a predicate-based query language for iOS, a Fluent query for Android, a string-based query for Javascript, and a LINQ query for .NET. 

Under the hood, the Realm Sync thread sends the query to MongoDB Realm (Realm’s cloud offering). MongoDB Realm translates the query to MongoDB’s query language and executes the query against MongoDB Atlas. Atlas then returns the resulting documents. Those documents are then translated into Realm objects, sent down to the Realm client, and stored on disk. The Realm Sync thread keeps a queue of any changes made locally to synced objects—even when offline. As soon as connectivity is reestablished, any changes made to the server-side or client-side are synced down using built-in granular conflict resolution logic. All of this occurs behind the scenes while the developer is interacting with the data. This is the part we’ve heard our users describe as “magic.” 

Flexible Sync also enables much more dynamic queries, based on user inputs. Picture a home listing app that allows users to search available properties in a certain area. As users define inputs—only show houses in Dallas, TX that cost less than $300k and have at least three bedrooms—the query parameters can be combined with logical ANDs and ORs to produce increasingly complex queries, and narrow down the search result even further. All query results are combined into a single realm file on the client’s device, which significantly simplifies code required on the client-side and ensures changes to data are synced efficiently and in real time. 

::::tabs
:::tab[]{tabid="Swift"}
```swift
// Set your Schema
class Listing: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    @Persisted var location: String
    @Persisted var price: Int
    @Persisted var bedrooms: Int
}

// Configure your App and login
let app = App(id: "XXXX")
let user = try! await app.login(credentials:
            .emailPassword(email: "email", password: "password"))

// Set the new Flexible Sync Config and open the Realm
let config = user.flexibleSyncConfiguration()
let realm = try! await Realm(configuration: config, downloadBeforeOpen: .always)

// Create a Query and Add it to your Subscriptions
let subscriptions = realm.subscriptions

try! await subscriptions.write {
    subscriptions.append(QuerySubscription<Listing>(name: "home-search") {
        $0.location == "dallas" && $0.price < 300000 && $0.bedrooms >= 3
    })
}

// Now query the local realm and get your home listings - output is 100 listings
// in the results
print(realm.objects(Listing.self).count)

// Remove the subscription - the data is removed from the local device but stays
// on the server
try! await subscriptions.write {
    subscriptions.remove(named: "home-search")
}

// Output is 0 - listings have been removed locally
print(realm.objects(Listing.self).count)
```
:::
:::tab[]{tabid="Kotlin"}
```kotlin
// Set your Schema
open class Listing: ObjectRealm() {
  @PrimaryKey
  @RealmField("_id")
  var id: ObjectId
  var location: String = ""
  var price: Int = 0
  var bedrooms: Int = 0
}

// Configure your App and login
val app = App("<YOUR_APP_ID_HERE>")
val user = app.login(Credentials.emailPassword("email", "password"))

// Set the new Flexible Sync Config and open the Realm
let config = SyncConfiguration.defaultConfig(user)
let realm = Realm.getInstance(config)

// Create a Query and Add it to your Subscriptions
val subscriptions = realm.subscriptions
subscriptions.update { mutableSubscriptions ->
   val sub = Subscription.create(
      "home-search", 
      realm.where<Listing>()
         .equalTo("location", "dallas")
         .lessThan("price", 300_000)
         .greaterThanOrEqual("bedrooms", 3)
   )
   mutableSubscriptions.add(subscription)
}

// Wait for server to accept the new subscription and download data
subscriptions.waitForSynchronization()
realm.refresh()

// Now query the local realm and get your home listings - output is 100 listings 
// in the results
val homes = realm.where<Listing>().count()

// Remove the subscription - the data is removed from the local device but stays 
// on the server
subscriptions.update { mutableSubscriptions ->
   mutableSubscriptions.remove("home-search")
}
subscriptions.waitForSynchronization()
realm.refresh()

// Output is 0 - listings have been removed locally
val homes = realm.where<Listing>().count()
```
:::
:::tab[]{tabid=".NET"}
```csharp
// Set your Schema
class Listing: RealmObject
{
    [PrimaryKey, MapTo("_id")]
    public ObjectId Id { get; set; }
    public string Location { get; set; }
    public int Price { get; set; }
    public int Bedrooms { get; set; }
}

// Configure your App and login
var app = App.Create(YOUR_APP_ID_HERE);
var user = await app.LogInAsync(Credentials.EmailPassword("email", "password"));

// Set the new Flexible Sync Config and open the Realm
var config = new FlexibleSyncConfiguration(user);
var realm = await Realm.GetInstanceAsync(config);

// Create a Query and Add it to your Subscriptions
var dallasQuery = realm.All<Listing>().Where(l => l.Location == "dallas" && l.Price < 300_000 && l.Bedrooms >= 3);
realm.Subscriptions.Update(() =>
{
    realm.Subscriptions.Add(dallasQuery);
});

await realm.Subscriptions.WaitForSynchronizationAsync();

// Now query the local realm and get your home listings - output is 100 listings
// in the results
var numberOfListings = realm.All<Listing>().Count();

// Remove the subscription - the data is removed from the local device but stays
// on the server

realm.Subscriptions.Update(() =>
{
    realm.Subscriptions.Remove(dallasQuery);
});

await realm.Subscriptions.WaitForSynchronizationAsync();

// Output is 0 - listings have been removed locally
numberOfListings = realm.All<Listing>().Count();
```
:::
:::tab[]{tabid="JavaScript"}
```js
import Realm from "realm";

// Set your Schema
const ListingSchema = {
  name: "Listing",
  primaryKey: "_id",
  properties: {
    _id: "objectId",
    location: "string",
    price: "int",
    bedrooms: "int",
  },
};

// Configure your App and login
const app = new Realm.App({ id: YOUR_APP_ID_HERE });
const credentials = Realm.Credentials.emailPassword("email", "password");
const user = await app.logIn(credentials);

// Set the new Flexible Sync Config and open the Realm
const realm = await Realm.open({
  schema: [ListingSchema],
  sync: { user, flexible: true },
});

// Create a Query and Add it to your Subscriptions
await realm.subscriptions.update((mutableSubscriptions) => {
  mutableSubscriptions.add(
    realm
      .objects(ListingSchema.name)
      .filtered("location = 'dallas' && price < 300000 && bedrooms = 3", {
        name: "home-search",
      })
  );
});

// Now query the local realm and get your home listings - output is 100 listings
// in the results
let homes = realm.objects(ListingSchema.name).length;

// Remove the subscription - the data is removed from the local device but stays
// on the server
await realm.subscriptions.update((mutableSubscriptions) => {
  mutableSubscriptions.removeByName("home-search");
});

// Output is 0 - listings have been removed locally
homes = realm.objects(ListingSchema.name).length;
```
:::
::::
## Optimizing for Real-Time Collaboration

Flexible Sync also enhances query performance and optimizes for real-time user collaboration by treating a single object or document as the smallest entity for synchronization. Flexible Sync allows for Sync Realms to more efficiently share data and for conflict resolution to incorporate changes faster and with less data transfer.

For example, you and a fellow employee are analyzing the remaining tasks for a week. Your coworker wants to see all of the time-intensive tasks remaining (`workunits >  5`), and you want to see all the tasks you have left for the week (`owner == ianward`). Your queries will overlap where `workunits > 5` and `owner == ianward`. If your coworker notices one of your tasks is marked incorrectly as `7 workunits` and changes the value to `6`, you will see the change reflected on your device in real time. Under the hood, the merge algorithm will only sync the changed document instead of the entire set of query results increasing query performance. 

![Venn diagram showing that 2 different queries can share some of the same documents](https://mongodb-devhub-cms.s3.us-west-1.amazonaws.com/Interesecting_Tasks_380cf4962d.png)

## Permissions

Whether it’s a company’s internal application or an app on the App Store, permissions are required in almost every application. That’s why we are excited by how seamless Flexible Sync makes applying a document-level permission model when syncing data—meaning synced documents can be limited based on a user’s role.

Consider how a sales organization uses a CRM application. An individual sales representative should only be able to access her own sales pipeline while her manager needs to be able to see the entire region’s sales pipeline. In Flexible Sync, a user’s role will be combined with the client-side query to determine the appropriate result set. For example, when the sales representative above wants to view her deals, she would send a query where `opportunities.owner == "EmmaLullo"` but when her manager wants to see all the opportunities for their entire team, they would query with opportunities.team == "West”. If a user sends a much more expansive query, such as querying for all opportunities, then the permissions system would only allow data to be synced for which the user had explicit access.

```json
{
  "Opportunities": {
    "roles": [
        {
                name: "manager", 
                applyWhen: { "%%user.custom_data.isSalesManager": true},
                read: {"team": "%%user.custom_data.teamManager"}
                write: {"team": "%%user.custom_data.teamManager"}
            },
        {
                name: "salesperson",
                applyWhen: {},
                read: {"owner": "%%user.id"}
                write: {"owner": "%%user.id"}
        }
    ]
  },
{
  "Bookings": {
    "roles": [
        {
                name: "accounting", 
                applyWhen: { "%%user.custom_data.isAccounting": true},
                read: true,
                write: true
            },
        {
                name: "sales",
                applyWhen: {},
                read: {"%%user.custom_data.isSales": true},
                write: false
        }
    ]
  }
```

## Looking Ahead

Ultimately, our goal with Flexible Sync is to deliver a sync service that can fit any use case or schema design pattern imaginable without custom code or workarounds. And while we are excited that Flexible Sync is now in preview, we’re nowhere near done. 

The Realm Sync team is planning to bring you more query operators and permissions integrations over the course of 2022. Up next we are looking to expose array operators and enable querying on embedded documents, but really, we look to you, our users, to help us drive the roadmap. Submit your ideas and feature requests to our [feedback portal](https://feedback.mongodb.com/forums/923521-realm) and ask questions in our [Community forum](https://www.mongodb.com/community/forums/c/realm/realm-sync/111). Happy building!

Realm Flexible Sync (now in preview) gives developers new options for syncing data to your apps

Sync

Featured

Using Realm Flexible Sync in Your App—an iOS Tutorial

Introducing Flexible Sync (Preview) – The Next Iteration of Realm Sync

Realm Sync in Use - Building and Architecting a Mobile Chat App Meetup

All Sync Content