Realm.asyncOpen doesn't call the callback

I had to terminate sync and re-enable it. During the login procedure I use asyncOpen for both a public and private realm to fully sync with the server before letting the user in the app.

Now Realm.asyncOpen on the private realm doesn’t execute its callback at all. I’m expecting it to execute the callback with an error so that I can handle it in the app.

MongoDB Realm log shows:

Error:

client file not found. The server has forgotten about the client-side file presented by the client. This is likely due to using a synchronized realm after terminating and re-enabling sync. Please wipe the file on the client to resume synchronization. { sessionIdent: 2, clientFileIdent: 29 } (ProtocolErrorCode=208)

How would I know to delete the file if the API doesn’t actually tell me to do so?
Also, the realm file is actually found on the device. FIleManager.default.fileExists(at: path) reports true

I don’t understand if this error can happen in production and how to mitigate it. How can you deal with schema migrations if you don’t have a migration block? you can’t just wipe user’s data because you added a field in an object can you?
Why can’t realm sync have a migration block in the first place?

4 Likes

can anybody from Realm respond to this? it’s been two months

1 Like

It’s frustrating when questions go unanswered but it may be due to the nature of the question.

It’s best to isolate questions to one per post - that way answers can be short and on point. Otherwise the tend to get very long and it’s hard to find the answer.

Also ensure the question body stays with the question topic; in this case the question about migrations is independent of why asyncOpen doesn’t function as expected.

Including your code that reproduces the issue is very important. For all we know you could be implementing the async call incorrectly which is why it’s not firing the closure code. Not saying it is, but we just don’t know.

Your question about migrations is very good. While the current documentation is good in some areas, it’s seriously lacking in others; especially in the areas that deal with how to handle the development process.

For example;

how does one add a property to an existing object in a sync’d realm. What is the step by step process to make that happen?**

and your question

How can you deal with schema migrations

I would ask separate questions for each topic and update this question with your code. Not making any promises as I not a Realmer but it may get better exposure.

**note that adding a property to a model in code will reflect in the realm console app schema section. However, the documentation is not clear on that. What about removing a property or changing the name?

1 Like

This might not be a definitive answer but it can lead you to the correct path.

Why can’t realm sync have a migration block in the first place?

The rule of thumb - sync’d realms don’t use migration block, local realms use migration blocks. This is not apparent at all from the docs and I’ve raised it in one of my other Forum post. In the interim, you can check out my Medium article on what all scenarios force you to restart sync while performing schema changes on a sync’d realm.

I’m expecting it to execute the callback with an error so that I can handle it in the app.

The other issue I sense from your question is how to perform client reset? - I’ve asked this question for iOS SDK and I found that answer to be helpful. You can find one client reset code example here.

If you want to listen to sync errors, you can set an error handler as follows.

// app is the instance of RealmSwift.App
app.syncManager.errorHandler = { (error, session)  in
     // Perform error handling
}

@siddharth_kamaria Super good info. Thanks for those links. Here’s another option for a client reset that’s not directly related to the original question.

Realm fixed the built in functionality for a client reset which doesn’t rely on the FileManager or storing information in User Defaults. It’s a matter of clearing the files stored of disk associated with each partition. Let me set this up (Swift).

Generally, an app will have a Constants file (per the documetation) which may store your Realm Partitions - like this

import RealmSwift

struct Constants {
    // Set this to your Realm App ID found in the Realm UI.
    static let REALM_APP_ID = "tasktracker-xyzab"
    static let REALM_PARTITION_VALUE = "Task Tracker"
    static let JAY_PARTITION_VALUE = "Jay Task"
    static let CINDY_PARTITION_VALUE = "Cindy Task"
}

So in this Task Tracker app, there are three partitions; a main Task, my task and Cindys tasks.

Then to do a client reset those three partitions need to be removed from disk;

func performDelete() {
    //let user = get the current user object.
    self.deletePartitionFilesFor(user: user, whichPartition: Constants.REALM_PARTITION_VALUE)
    self.deletePartitionFilesFor(user: user, whichPartition: Constants.JAY_PARTITION_VALUE)
    self.deletePartitionFilesFor(user: user, whichPartition: Constants.CINDY_PARTITION_VALUE)
}

which calls this function which relies on Realm.deleteFiles:for

func deletePartitionFilesFor(user: User, whichPartition: String) {
    do {
        let config = user.configuration(partitionValue: whichPartition)
        let result = try Realm.deleteFiles(for: config)

        if result == true {
            print("   all files have been deleted")
        } else {
            print("   deleting of files failed, it's probably open or the files were already deleted")
        }
        
    } catch let error as NSError {
        print(error.localizedDescription)
    }
}

This will not delete the .lock files but those don’t contain any sync’d data.

Also, and this is critical, for the above code to work the app cannot have already accessed Realm. e.g. once the app ‘touches’ Realm, those files cannot be deleted. So the code would need to be run before any Realm calls are made.

To tie this back to one part of the original question

you can’t just wipe user’s data because you added a field in an object can you?

As it stands, additive changes happen automatically. So if a property is added to an object, it will (eventually) show up in the console->realm->app name->schema section and when a new object is written, it will show up in the Collections area.

We’re not so lucky with destructive changes. The change will have to be made and the client will have to be ‘reset’ (wiped) per the above, and allow it to re-sync. There is no built in mechanism for this (HELLO REALM) so you’ll have to manually write the code to make that happen.

The documentation avoids this topic entirely in sync situations.

1 Like

@Jay That code example is really good and will help someone who is going through this thread. I find this Realm.deleteFiles approach to be much straightforward than FileManager approach.

The only issue with destructive schema changes is the fear of data loss. What I tried (and seem to be working decently) is taking a backup before deleting the realms and then copying all objects from the old realm to a new one.

newRealm.write {
    currentRealm.objects(type)
         .forEach { obj in
             newRealm.create(type, value: obj, update: updatePolicy)
         }
}

Hi

Thanks to everybody for the answers so far.

The reason I’ve been asking all these questions in the same post is that they’re all related. I can’t split them in multiple questions because they all stem from the same issue - a migration error that can’t be handled because asyncOpen doesn’t call its completion block. The migration error stems from the fact that I modified the schema and I had to terminate and reenable sync server side.

I didn’t include the code because it’s super basic and I thought it was inferred:

    Realm.asyncOpen(configuration: configuration, callbackQueue: .main) { result in
        switch result {
        case .failure(let error):
            print("Async open failed: \(error)")
                               
        case .success(let realm):
            print(“Async open success”)
        }
    }

I actually did strip it down to the above in my app while debugging this issue, stripping it of all handling and reducing it to the prints. In the case of an error the completion block is not called. It’s only called when the result is .success. As stated, there’s a migration error which you can only see if you dig through the Realm Sync logs in the MongoDB UI. A rather cryptic one at that which is un(or poorly)documented: “client file not found. The server has forgotten about the client-side file presented by the client. This is likely due to using a synchronized realm after terminating and re-enabling sync. Please wipe the file on the client to resume synchronization.”
The log directs me to delete the Realm client side but this poses several questions which my post is looking for an answer to:

  1. How would I know to delete this realm if AsyncOpen doesn’t report this error
  2. Is this error possible to happen while the app is live (it currently isn’t live) - by live I mean that there a version of it actively being used
  3. What would happen with the unsynced data on the device? If I’m deleting the realm then I’m wiping the user’s unsaved data
  4. Why is the log saying the client file is not found when FIleManager.default.fileExists(at: path) reports that the realm file indeed exists. Or is this client file something else than a realm file.

So all this just made me wonder why realm sync can’t actually have a migration block. Which I agree is a bit of a rant and a tad bit out of scope but still connected because having that would mitigate all those issues - when a destructive schema change is made then the developer can migrate the data to the new format and continue sync rather than delete everything the user has. I get the destructive changes to the schema are not handled automatically but it’s odd that I don’t have an option to handle them manually. The only option is to create a monster of an object that converts destructive changes into additive ones by just ignoring fields not used anymore. In the first few versions of an app this would be nothing much but if you keep making an app better in a year you won’t understand much from the objects you have in your program because you’ve been adding all sorts just to please realm sync. Couple that with the fact that Realm Sync doesn’t support a foreign keys equivalent (but rather having to transform previous object references to ids to link to) and your swift (my case) objects will look so odd and foreign from the language you use that you start questioning the reason you picked Realm in the first place. The main selling point of Realm was that programming for it was a matter of just making an object inherit from Object and that’s pretty much it. I didn’t really sign up for having arrays/lists of ids. Debugging anything and printing to console while you track an object is now 10 times harder just because of that. Rant over.

In my application any kind of hiccup like this would be unacceptable and I have to programatically cover all bases so I’m trying to understand all possible errors and how to gracefully recover. The MongoDB documentation is great in parts but sync is very poorly covered I’m afraid.

I’ll go through the links provided and see what I can learn from them and I’ll open some support tickets specific to my needs. I’ll report here everything that is generic enough to help other people

We see a different behavior across our apps and I am not sure what the difference is between our code and your code.

Our experience is when Atlas gets our of sync, it throws an error which can be seen in the console and our app receives that error as well.

I just tried it now with a destructive change; I added a property to a test object called “propertyToDelete” and then added some objects. After a moment it sync’d and the objects appeared in the console.

Then we went into the console->Realm->the app->schema and deleted that property. Upon saving, it generated an error in the console (as expected)

The changes you are trying to save are destructive and will require Sync to reinitialize.

Then we saved, let it reinitialize and ran the app. The console log shows this error

and when we run our code it shows a matching error in the Realm.asyncOpen function

2021-05-16 09:46:24.288551-0400 TestApp[95487:3157327] Sync: Connection[1]: Session[1]: Received: ERROR “Bad client file identifier (IDENT)” (error_code=208, try_again=0)
Failed to open realm: The server has forgotten about this client-side file (Bad client file identifier (IDENT)). Please wipe the file on the client to resume synchronization
2021-05-16 09:46:24.296675-0400 TaskTracker[95487:3157327] Sync: Connection[1]: Disconnected

Here’s our async open function for comparison, we pass in the partition string we want to access:

func configRealmSync(whichPartition: String) {
    guard let user = gTaskApp.currentUser else {
        fatalError("Logged out?")
    }
    
    let config = user.configuration(partitionValue: whichPartition)
    Realm.asyncOpen(configuration: config) { result in
        switch result {
        case .failure(let error):
            print("Failed to open realm: \(error.localizedDescription)")
        case .success(let realm):
            print("Successfully opened realm: \(realm)")
        }
    }
}

The only visible difference in the code is callbackQueue: .main in yours.

So I don’t have an explanation, just a different experience.

After further review the issue is that the companyID in state.user?.companyId is ‘nil’!
However in the database it exists:

{“_id":“60f455391eb0f390710628ef”,“_partition”:“user=60f455391eb0f390710628ef”,“name”:"jj@jj.com”,“companyId”:{“$oid”:“60f455391eb0f390710628fc”},“canReadPartitions”:[“user=60f455391eb0f390710628ef”],“canWritePartitions”:[“user=60f455391eb0f390710628ef”],“memberOfProject”:[{“_partition”:“project=60f455391eb0f390710628ef”,“name”:“Your First Project”,“hourlyRate”:{“$numberDecimal”:“0”},“order”:{“$numberLong”:“0”},“unbilledtotal”:{“$numberDecimal”:“0”}}]}

Is this a response to a different question? Or… what is the question you’re referring to?

it is the ObjectID which is similar. The issue was my “partitions.” Still trying to get them figured out.

Any luck on resolving why asyncOpen doesn’t call its callback? I’m having the same issue though the root cause is only because I terminated the sync and re-initialized (reason why I terminated the sync sounded like a one-time issue. More on this here: Failed to parse, or apply received changeset: ArrayErase: No such object: <ObjectClassName> in class · Issue #7478 · realm/realm-swift · GitHub)

Here’s what I’m doing in short:

  1. Download the latest Realm on app launch. Download succeeds. I kill the app
  2. On the Realm web UI, I terminate the sync and re-initialize
  3. Cold start the app. The app loads normally. After a second or two, Realm throws this error: ERROR "Bad client file identifier (IDENT) … "
  4. I catch this error via the SyncManager’s errorHandler. Then,
  5. Close (I think) all Realm instances. Then,
  6. Invoke SyncSession.immediatelyHandleError. Then,
  7. I used FileManager to move the backed up realm to be later restored (I also skipped this part after reading it’s not necessary. This didn’t fix the issue). Finally,
  8. With a one second delay (I tried without the delay too), I invoke Realm.asyncOpen to download the new realm and restore the backup. This callback never gets called

Here’s my code

let app = App(id: "gifts-ztmiy") //Constants.realmAppID)
self.app = app
app.syncManager.errorHandler = { [weak app] error, session in
    guard let app = app else { return }
    guard let user = session?.parentUser() else { return }
    guard let syncError = error as? SyncError else {
        fatalError("Unexpected error type passed to sync error handler! \(error)")
    }

    let realmConfigURL: URL? // = Documents/mongodb-realm/gifts-ztmiy/614a87d70569d4b7db2ce037/%2522Giftz%2522.realm
    if let partition = session?.configuration()?.partitionValue as? String {
        realmConfigURL = user.configuration(partitionValue: partition).fileURL
    } else {
        realmConfigURL = nil
    }

    switch syncError.code {
    case .clientResetError:
        guard let realmConfigURL = realmConfigURL, let (path, clientResetToken) = syncError.clientResetInfo() else {
            return
        } // file = Documents/mongodb-realm/gifts-ztmiy/recovered-realms/recovered_realm-20211014-114314-ubJPUDvC

        /// kill UI and show an alert
        injectNavigator().showWelcomeScreen()
        let dissmissAlert = injectAlertService().showLoadingAlert(title: "Housekeeping...")

        // Do not call SyncSession.immediatelyHandleError(_:) until you are sure that all references to the
        // Realm and managed objects belonging to the Realm have been nil’ed out, and that all autorelease
        // pools containing these references have been drained.
        Self.closeRealmSafely()

        // Immediately delete the local copy of the Realm
        SyncSession.immediatelyHandleError(clientResetToken, syncManager: app.syncManager)

        // Store the un-synced changes to be restored later TODO(move to background thread)
        Self.saveBackupRealmPath(source: path, destination: realmConfigURL)

        DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) {
            guard let user = session?.parentUser(), let partition = session?.configuration()?.partitionValue as? String else {
                return
            }

            /// show a loading screen while we reopen realm and restore from backup (housekeeping)
            let task = Realm.asyncOpen(
                configuration: user.configuration(partitionValue: partition /*"Giftz"*/ /*Constants.realmPartitionValue*/),
                callbackQueue: .main
            ) { result in
                dissmissAlert()

                switch result {
                case .success(let realm):
                    Self.restoreFromBackup(newRealm: realm)

                    /// restore UI after housekeeping has finished
                    injectNavigator().showMainScreenPossibly()

                case .failure(let error):
                    assertionFailure(error.localizedDescription)
                }
            }

            var progressAmount = 0
            task.addProgressNotification(queue: .main) { progress in
                if progress.isTransferComplete {
                    progressAmount    = 0
                    print("Transfer finished")
                } else {
                    guard progress.transferredBytes > progressAmount else { return }

                    let numFormatter = NumberFormatter()
                    numFormatter.numberStyle = .decimal
                    let transferredStr = numFormatter.string(from: NSNumber(value: progress.transferredBytes))
                    let transferrableStr = numFormatter.string(from: NSNumber(value: progress.transferrableBytes))

                    progressAmount = progress.transferredBytes
                    print("Transferred \(transferredStr ?? "??") of \(transferrableStr ?? "??")…")
                }
            }
        }

    default:
        // Handle other errors...
        break
    }
}

I tried something else and got this error

- failure : Error Domain=io.realm.unknown Code=211 "The client and server disagree about the history (Diverging histories (IDENT)). Please wipe the file on the client to resume synchronization" UserInfo={Category=realm::sync::ProtocolError, NSLocalizedDescription=The client and server disagree about the history (Diverging histories (IDENT)). Please wipe the file on the client to resume synchronization, Error Code=211}

Instead of Reopening the realm Async (step #7 of my last reply) I logged out the user and logged them back in. After logging in successfully, I invoke Realm.asycOpen and that’s when the error was thrown

Perhaps this alone isn’t enough to handle the clientResetError?