Client Reset Process

Hi @Paolo_Manna

I just came across your example code, which was interesting to me. First off, before I go on, thanks for posting your example - I am sure it has and will continue to help many developers!

I compared your example to my setup and found in general we do a lot of the same. I did pick up your way of choosing an ‘asyncOpen’ or ‘sync’. Previously, I depended on tracking the progress on a Realm’s sync solely. Consequently, that could pause the user longer than needed in some cases, since I have some Realms not worth waiting for and benefit via ‘asyncOpen’.

I would like to bring some differences to your attention, that I felt were important for me and could be for some other developers too. If you don’t mind I would appreciate your thoughts - especially if I error-ed or have any oversights (thank you in advance). Below I have presented a question on your use of calling ‘realmSetupErrorHandler()’ per Realm open.

In my usage, I like to know which Realm the ‘SyncSession’ is reporting on, for the ‘Error’ being handled. Therefore, I determine the Realm from the ‘SyncSession’ provided (which is the code I am providing). With that, I use such Realm in places thereafter. For example, sometimes I could use it to query before I back up the (reset-needing) Realm, and then merge later. Though for larger Realms it is better to query on a per Collection basis from the backed-up Realm. Merging gets tricky for Flexible Sync (FS), since it is based on query permissions and those results can change based on client-side usage (but that is a topic for another day). Luckily, for Partition-Based-Sync (PBS), it is straight forward. Oh yeah, I also pull the array of ‘ObjectTypes’ (e.g., ‘useThisRealmsConfig.objectTypes’) to iterate through them for querying data, in order for merging into the new (downloaded) syncing Realm.
I know for your example, it was testing 1 Realm (and 1 Schema) and therefore did NOT need to provide for additional Realms nor Schemas.

Some of my usage/differences to note:
• I am running multiple Realm-Apps on 1 client-side app. Therefore, I check for Flexible-Sync (FS) and Partition-Based-Sync (PBS) - this can also work in the future when both FS and PBS can be used in 1 Realm-App [:crossed_fingers:].
• I have multiple Realms on PBS (i.e., per partition) and 1 Realm on FS (since FS currently works with only 1 Realm essentially).
• Since ‘.syncManager.errorHandler’ is per Realm-App, I start it right after I open all the Realms (per Realm-App) in a single block. Since I’m running 2 Realm-Apps (per PBS & FS) I do this process twice - and only 2x (unless reloading Realm-App) and not per error being handled.
•• I noticed you do it per error handling (via Realm opens), is that something I need to change? - meaning (e.g.) is there some issue during errors that it is better re-pass it again?


/// SOME GENERAL NOTES. 
    /// I am running multiple Realm-Apps on 1 client-side app. Therefore, I test for Flexible-Sync (FS) and Partition-Based-Sync (PBS) - this can also work in the future when both FS and PBS can be used in 1 Realm-App [🤞].
    /// I have multiple Realms on PBS (i.e., per partition) and 1 Realm on FS (since FS currently works with only 1 Realm essentially).
    /// Since '`.syncManager.errorHandler`' is per Realm-App, I start it right after I open all the Realms (per Realm-App) in a single block. Since I'm running 2 Realm-Apps (per PBS & FS) I do this process twice - and only 2x (unless reload Realm-App) and not per error being handled.
    /// My code setup is quite different, but I extracted and refactored my code for this part and to fit your example. I also named properties to be long in order to be verbose for any readers.


/// The changes could be entered here (line# 293) in your example (as of 2022-06-11). I would start by adding an unwrapped ‘SyncSession’. I renamed to use '`have`' prepended in the names too.
guard let haveSelf = self, let haveSession = session, let haveUser = haveSession.parentUser() else { return; }

guard let useThisRealmsSyncConfig = haveSession.configuration()
    else {
        // NOTE: CRASH in DEV, Error-Log in PROD. Reminder I started after opening a Realm.
        fatalError();
    }

print("During-TESTING-GOT:'useThisRealmsSyncConfig' - at this point we now know if this is 'Flexible-Sync'=='\(useThisRealmsSyncConfig.enableFlexibleSync)' - if 'true' then is 'FS'.")

let isFlexibleSyncEnabled = useThisRealmsSyncConfig.enableFlexibleSync
let useThisRealmsConfig: Realm.Configuration

if isFlexibleSyncEnabled {
    useThisRealmsConfig = useThisRealmsSyncConfig.user.flexibleSyncConfiguration()
}
else {
    guard let haveThisRealmsPartitionValue = useThisRealmsSyncConfig.partitionValue as? String
        else {
            // NOTE: This should not occur.
            fatalError();
        }
    print("During-TESTING-GOT: 'haveThisRealmsPartitionValue', which is: '\(haveThisRealmsPartitionValue)'.")
    useThisRealmsConfig = useThisRealmsSyncConfig.user.configuration(partitionValue: AnyBSON(haveThisRealmsPartitionValue), clientResetMode: .manual)
}

        
print("During-TESTING-GOT: 'useThisRealmsConfig', which its file-URL is: '\(String(describing: useThisRealmsConfig.fileURL))'.")

let useThisRealm: Realm
do {
    useThisRealm = try Realm(configuration: useThisRealmsConfig)
}
catch let haveError as Error? {
    fatalError("During-TESTING-GOT: an ERROR trying to get a 'Realm', here is the ERROR: '\(String(describing: haveError?.localizedDescription))'.");
}

print("During-TESTING-GOT: 'useThisRealm', which has: '\(useThisRealm.schema.objectSchema.count)' Schemas.")

/// This is provided to show how to get all objects to merge later.  Also, based on the size of Realm you may want to do similar from the backed-up Realm and query on a per Collection basis. I wish there was an exposed property for last sync'd to prevent unnecessary loading here. 
let allObjectTypesArray: [Object.Type] = (useThisRealm.configuration.objectTypes?.filter({ $0 is Object.Type }) ?? []) as! [Object.Type]
var mergeLaterArray: [ (Object.Type, [Object]) ] = [ ]
allObjectTypesArray.forEach({ mergeLaterArray.append(($0, Array(useThisRealm.objects($0)))) })  // may want to sort these too.
print("During-TESTING-GOT: this far! Now will test the current Realm (before backing-up) for querying before a Client-Reset. Here are values for: 'allObjectsArray' having '\(mergeLaterArray.count)' total Classes/Schemas && 'allObjectsArray' having '\(mergeLaterArray.compactMap({ $0.1 }).flatMap({ $0 }).count)' total objects across all Collections.")

/// back to your code
let syncError   = error as! SyncError
var fileURL: URL?

Hi @Reveel,

I’ll try to comment some of your statements inline: I won’t be exhaustive, though, just point out a couple of things. A foreword, however: Client Reset is an area in development, there are on the roadmap some modifications that would make life easier for developers, so this conversation will likely be outdated soon.

Of course: the sample code tried to stay simple, and assumed only one realm was opened. Interestingly, this may not be an issue for other SDKs, JavaScript for example has a per-realm error handler.

objectTypes isn’t mandatory, so it may not be set, and may not reflect what the actual content of the realm is: if you’ve the realm object, I’d rather go for realm.schema to detect the classes that are in the DB.

As it’s per-app, as you say, why not open it before opening all the Realms? This way, if an error occurs on, say, the first realm while you’re still opening the second, you’d still catch it.

No, I only do it once: at the beginning, there’s the condition:

	// Don't re-do it
	guard app.syncManager.errorHandler == nil else { return }

Hey @Paolo_Manna,

Thanks for getting back to me!

I’ll answer your points and give some background on my work for this. Then I’d like to ask your thoughts on some other related points - that I’d like to see the Realm team improve on.

A little background:
I had built my original EH (errorHandler) during legacy Realm, which had been modified with MongoDB. I am looking at it again right now since Flexible Sync can trigger more Client Resets. I feel I have tracked most of the changes from legacy to Mongo, that I found in docs. I am assuming some things may have improved with MongoDB, that I might not notice with my setup. As an example, back then I would sometimes receive an error that the recovery Realm file was not (yet) available to move. Hence, I built a device-folder monitoring service to notify me of changes in a given folder (e.g., “/recovered-realms”) … which may not be an issue currently. Additionally, I built processes to check for “error codes” and “error descriptions” trying to match both or one, and report on any discrepancies with docs, or even undocumented codes like error code ‘225’. This way I can track changes and compare them to docs’ error codes. This also helps in situations like error code ‘101’ which can have different descriptions, such as a more common one like: “sync is not enabled for this app”.

Reply to your points:
• Yes, you are correct '.objectTypes’ are optional and also that the Schema is a more solid approach (though a little bit more to translate that into an array for ‘.objectTypes:grin:). In my app, I ensure maintaining a value to pass for each Realm’s config and for each Realm later.
•• I found this as a benefit for many uses throughout my app. I do not know if it has improved recently, but previously, when restoring a Realm from the backup (i.e., for Client Reset) I was only able to open that Realm locally once I supplied the array to ‘.objectTypes’ (assumed reopen had to match initial open). I keep a static value for both the full ‘Array<ObjectBase.Type>’ and the ‘Array<Object.Type>’ for merging that exclude ‘EmbeddedObject’.
•• I think there is a bug or issue currently that I have noticed with Realms on a device that is syncing. Basically, it will show all the models in Realm Studio, even though I supplied a specific array to ‘.objectTypes’ (excluding certain models) and I have even declared ‘false’ for ‘shouldIncludeInDefaultSchema()’ in each class (e.g., Swift SDK).

• My original design did start the EH before opening Realms, I recently changed it to be right after opening the Realms. Since, beforehand, you’d be likely to see more of the 100-level error codes (which previously I needed to handle & report since I managed ROS too). Additionally, on the client side I handle these types of errors differently… since in most cases I have a wait, retry, etc. If there are some solid error cases prior to opening a Realm please let me know I’d will improve.

• D’oh - my bad. I do remember seeing that when I was initially reviewing your code, but I must have overlooked it when doing my comparison. I should have also copied your code into a project to test it out. Anyhow, thanks for pointing that out.

Some questions for you…
• The way Client Reset works, is that I read all the objects from the backed-up Realm and write them into the new (downloaded) Realm via ‘.create(: : update: .modified)’. That can make it prone for a long process (i.e., for larger Realms) which then carries a slew of potential issues. I always hoped for a way to query on a date (last sync’d) to reduce the processing before merging back in. Previously, to reduce tasking the client-side (and to be thorough), I built a way for the merging to occur on the server side (ROS/legacy) but cannot do that now. Additionally, even if I could, it would be hard without the support of any Realm SDKs available to use (e.g., a ‘.create()’ method) on a serverless Realm Function.
•• Do you know if that is something Realm might work on or could be submitted to be offered? -for both the ability to query on last synced and Realm SDK support on Realm Functions.

• Lastly, it would be nice if we could try to reconnect manually from the client-side, when there is ‘disconnected‘ state. For a Client Reset we would not have to ask the user to restart the app. Any thoughts on this and its possibility?

Hi @Reveel,

Some more considerations below

This is by design, for Partition-based Sync: if a MongoDB collection is under Sync, all its records that match the partition value are included. It would make things complicated to let the client dictate what should or shouldn’t be downloaded. Flexible Sync is another story, of course.

It’s not much about errors that can happen before opening a realm, but the fact that you open multiple realms: imagine this scenario

  • You open successfully realm 1
  • You start opening asynchronously realm 2
  • If in the meantime you get an error (say, a Client Reset) on realm 1, you’ll miss it if you set the error handler after all the realms are successfully opened.

As I wrote before, Client Reset handling is still in full development at this time: while I can’t comment on features that haven’t been released nor announced, you can look forward some changes coming in soon…

While the backend is proprietary technology, the Realm SDKs are still open source, you’re welcome to raise issues or provide suggestions on Github.

This certainly makes sense: a Github issue, or even a suggestion on our Feedback portal is always worth consideration.

That’s not necessary right now: if you use the .discardLocal mode, all happens rather automatically, and without the user realising a client reset happened (at the cost of discarding all local changes). Also, in .manual mode, you can close the realm (that’s tricky in Swift, as there’s the iOS memory handling involved), and re-opening it without quitting. My example does that, even though it’s not perfect.

1 Like

@Paolo_Manna,

• I believe the new change (by design) to show all Collections on local Realms is not good in my opinion. A user can now open that Realm from their local device, and then can see all Collections and their Schemas, which would include some that would not apply to them (e.g., maybe an employee or customer). This did not occur in legacy Realm.

• The way I start each Realm-App’s EH it does avoid that issue you mentioned with ‘asyncOpen’ - though thanks for being thorough, since pointing it out can help future readers.

• Happily awaiting to hear about the changes that are in the works! Hopefully not many (or any :crossed_fingers:) breaking changes :grimacing:.

• OK - I will write something up for Github soon, for both the SDK support on backend and getting a process to improve merging. Regarding the latter… I always felt that Realm could set a flag that is triggered on a Client-Reset to expose needed info on that back-up Realm, to allow us to query for recent sync’d (or even query a metadata file), in order to concisely query such backed-up Realm.

• Regarding a ‘reconnect’ to Sync, I will post on the feedback site too, though (IMHO) it seems like GitHub is more active.

• Yup - tricky for sure[!]… though I am hoping to know this trick! Yeah I do not want lose any of the User’s data and hence merge it; therefore I use ‘.manual’. I have tried many things; including what I saw in your code too. I had even included logging-out all the users (via ‘.logOut()’ catching potential ‘Error’ too), and have also tried other things too. Unfortunately, with the new Realm (w/ my focus on FS) I am not successful on many Client Resets for Swift. I have not been able to pin-point the nuances for a full-proof success nor identify all of the failures yet. I have even tried edge things like adding a ‘.suspend’ before an ‘.invalidate()’, ensuring a read on the re-loggged-in Realm (before merging), and also tried issuing a new user ‘tokens’, etc. (hoping something under the hood might trigger it). I had even played with utilizing ‘CFRunLoop’ – all without consistent success. I would really like to handle this for iOS (to avoid presenting the user a feel that resembles a Windows-like restart :joy: )… so I am looking for any advice or direction (to dig deeper) that you can provide - thank you in advance!

Can you clear up one last set of questions (assumptions) regarding Client Reset I have?
• Am I correct that both ‘beforeClientReset’ and ‘afterClientReset’ are only effective for ‘.discardLocal’? -and will not be offered for ‘.manual’ in the future.

• Am I correct that if I chose ‘.discardLocal’, since it does attempt to self-correct, that there is NO way to merge data if it fails? – of course, excluding some cases where ‘.discardLocal’ converts to ‘.manual’ (e.g., breaking or destructive schema changes).

Thank you for all your time!

Hi @Reveel,

I don’t think so: as you say, .manual is still the fallback, and is unlikely to change, if nothing else for compatibility reasons.

I don’t know enough to reliably comment on that, probably the best place to ask would be to open a Github issue (Realm Core would be the best - all SDKs rely on that for low-level behaviour)

Hope the discussion helps, even though I doubt its contents will still be relevant for more than a few months…

1 Like

@Paolo_Manna

OK. Thank you for your time.