How to monitor the state of a flexible sync subscription

We are migrating an enterprise app from Partition Sync to Flexible Sync.

One of the features that we have in our app is showing the user the progress of the synchronization. This is a crucial feature as it gives the user a visual feedback that.

  1. They have pending unsynced changes that are being synced
  2. Gives an estimate of how much time the sync will take

Both of those points are important. However, the first point is more important. As I understand, detailed progress monitoring is not possible with Flexible sync. However, I really need to find a way to solve the first issue.

Essentially, I need to show some loading indicator whenever

  1. There is a background sync currently in progress
  2. When I add a new subscription, I want to monitor whether that “change” has been fully synced or no

We are heavily using reactive extensions, so, ideally I would need either an event or and IObservable<bool> that will indicate that there’s a current background sync in progress.

If you’re only interested in the upload path, then detailed progress notifications work there just fine. It’s only download notifications which are not available due to the unpredictability of the amount of data that needs to be downloaded (though we’re working on an a project to address this).

When you add a new subscription, you can call realm.Subscriptions.WaitForSynchronizationAsync() to be notified when the server has sent you the data that matches the new subscription. You could also check realm.Subscriptions.State when you start your application to see whether all the requested data has been synchronized or if you need to tell the user they’re not seeing the complete dataset.

Hey Nikola

Thanks for getting back.

Indeed, upload progress works fine. Download progress, is the issue.

I am checking the “State” property. But the problem is that it is not reactive, nor is there an event I can subscribe to to check whether we are currently “Actively” syncing. To overcome this, I’ve created this nasty timer-based reactive chain:

return Observable.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
                        .Select(_ => realm.Subscriptions.State == SubscriptionSetState.Pending)
                        .DistinctUntilChanged()
                        .Publish()
                        .RefCount();

It’d be great to have a means to do this reactively, like with download/upload progress notifications. If we can’t do that now, that’s also fine. But I can suggest something like this as a feature. I realize that due to the nature of data, it’s hard to estimate how much of it you will download; hence, the download progress is not there. But, I would assume, telling the fact that there’s something that’s being downloaded/uploaded is a much less complicated issue, and it’d be cool to have a reactive observable or an event just for that.

I see, this makes sense to me and I filed Have the session emit notifications whenever upload/download starts and completes · Issue #7045 · realm/realm-core · GitHub for the team to prioritize. Regarding the download path - we’re working on a project for this, but that will give you the estimated download progress, not how much data is left to download - do you think that will work for your use case or do you need to have some absolute measure for the amount of data to download (e.g. bytes/number of documents/number of changesets)?

Finally, I’m not an expert on RX, but I feel the timer-based solution is a bit of an overkill since the subscription state changes are always deterministic and they go in one direction only - i.e. Pending -> Complete/Error. Not sure if that’s clear from the API/docs, but once a subscription is “Complete”, it will never go back to “Pending” unless you update the subscription set. So if you wanted a non-timer based solution, you could build a thin wrapper around the subscription set that will emit notifications like:

public static class SubscriptionObserver
{
    public static event EventHandler<SubscriptionSetState> StateChanged;

    public static void UpdateRX(this SubscriptionSet subscriptions, Action update)
    {
        subscriptions.Update(update);
        _ = WaitForSyncAsync(subscriptions);
    }

    public static void Initialize(this SubscriptionSet subscriptions)
    {
        if (subscriptions.State == SubscriptionSetState.Pending)
        {
            _ = WaitForSyncAsync(subscriptions);
        }
    }

    private static async Task WaitForSyncAsync(SubscriptionSet subscriptions)
    {
        try
        {
            StateChanged?.Invoke(subscriptions, subscriptions.State);
            await subscriptions.WaitForSynchronizationAsync();
        }
        catch
        {
            // Log exceptions if necessary
        }
        finally
        {
            StateChanged?.Invoke(subscriptions, subscriptions.State);
        }
    }
}

The way you would use it is to call SubscriptionObserver.Initialize(realm.Subscriptions) the first time you create a Realm to trigger the initial notification (e.g. if the user updated their subscriptions while offline), then use realm.Subscriptions.UpdateRX instead of .Update. You could then wrap the event into an observable or just replace the eventhandler with an observable entirely.

Thanks a lot for raising the GitHub issue for this.

The estimated download progress is more than enough for me. I just need a way to show the user that they have pending data to sync and give them some absolute measure of how much. They don’t care about the individual bytes.

Thanks for the hint about the subscription state determinism. I actually didn’t know that. My assumption was that the subscription becomes “Pending” again once there’s some new data that will be downloaded. This raises a question though. If I have a subscription for Products. I don’t change it, but a new product comes and I need to sync it. In this case, the subscription object won’t help me detect and show “syncing” progress. This means, that whenever I change the subscription that’s the only place I can “wait” for that initial data to sync up and show some progress to the user. However, I can’t monitor the ongoing changes and show some indication to the user that there’s some data coming in unless I explicitly call “Synchronize” and wait for it to complete.

So, having the feature that you have already created would be a great addition.

Thanks, Nikola. We can close this thread for now.

When the subscription goes into Complete state, this means the server has sent all data that matches the subscriptions at the time the change happened, but there may be more data coming in after that. If you want to make sure the user is caught up with the latest changes on the server you can use realm.SyncSession.WaitForDownloadAsync, though that will not tell you whether there are changes, it’ll just wait until the server tells it its caught up, which can happen almost immediately if there are no new changes or take some time if there are.

Yep, that makes sense, Nikola. I am already using realm.SyncSession.WaitForDownloadAsync