Realm Swift Tutorial failing offline-first usage

@Duncan_Groenewald - We worked with Realm Cloud (the legacy one) for a long time, actually since its inception and it was different for sure.

At a high level, the key difference is that back in the day, there were no partitions - all of the data was stored in a single Realm File locally (or multiple files if YOU choose to separate them) and the Cloud storage emulated that structure. e.g. all objects of all kinds were stored ‘in the same place’ both locally and in the cloud.

With MongoDB Realm, that had to be changed to make the client files more align with MongoDB cloud storage as their server structure was ‘different’.

With MongoDB Realm, each partition is it’s own Realm File (locally) and they are separated logically by partition, which we are finding is not an issue.

@Thomas_Goyne excellent post by the way. In general, I am starting to move away from asyncOpen() as much as possible. If the choice is stale data versus some unpredictable wait at the bringing of time, I find that stale data is the better alternative. The other problem with asyncOpen(), as I pointed out in a Medium article I wrote, is that it returns on the primary thread by default (although this can be changed). My recommendation would be to update the MongoDB Realm docs here

Currently, the docs suggest using asyncOpen() as the default way of opening a realm. My suggestion would be to recommend a sync Realm open, with asyncOpen() as the alternative. Currently, the issue is confusing and needs to be cleared up at the documentation level.

We use asyncOpen() the first time a client is initialised to ensure seed data gets downloaded. If you don’t require seed data then you might want to avoid it. Our clients get set up the first time by support staff as it is a complex desktop app and needs a network connection to complete the initial setup. We can’t use asynchronous open for subsequent use because user may be offline at any time and asyncOpen() would fail unless timeout parameters are set. For us it is not necessary to use asyncOpen after setup has been done.

1 Like

I have come to that conclusion. That unless you absolutely need data to get going, avoid asyncOpen(), regular sync open is better, as the wait time is too unpredictable.

@Richard_Krueger

We are not running into this issue, and we definitely start with asyncOpen. Is this a duplicatable issue or is there a Git request filed as we wan’t to avoid this issue if possible.

It is totally duplicatable. See this public GitHub repo I made

This is about a 20 line Realm program. It seems to hang on the first asyncOpen() eventually comes back about 5 minutes later.

Richard

I was able to clone, build, and go -

2021-02-09 12:08:16.899494-0700 HangingApp[91164:6730907] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed
2021-02-09 12:08:17.827458-0700 HangingApp[91164:6730911] Version 10.5.2 of Realm is now available: https://github.com/realm/realm-cocoa/blob/v10.5.2/CHANGELOG.md
2021-02-09 12:08:17.835052-0700 HangingApp[91164:6730958] Sync: Connection[1]: Session[1]: client_reset_config = false, Realm exists = true, async open = false, client reset = false
2021-02-09 12:08:17.942314-0700 HangingApp[91164:6730958] Sync: Connection[1]: Connected to endpoint '34.227.4.145:443' (from '10.1.131.38:58121')
2021-02-09 12:08:18.892592-0700 HangingApp[91164:6730958] Sync: Connection[2]: Session[2]: client_reset_config = false, Realm exists = true, async open = false, client reset = false
2021-02-09 12:08:18.999104-0700 HangingApp[91164:6730958] Sync: Connection[2]: Connected to endpoint '34.227.4.145:443' (from '10.1.131.38:58123')
here
2021-02-09 12:08:19.745929-0700 HangingApp[91164:6730787] Login success

@Richard_Krueger
@Ian_Ward

Me too. Not only did your project work. I plucked the code out of your app and made a macOS project and it connects over and over using the same code, no problem at all.

021-02-09 14:33:32.035666-0500 RealmHangTest[9821:1099884] Sync: Connection[1]: Session[1]: client_reset_config = false, Realm exists = true, async open = false, client reset = false

2021-02-09 14:33:32.086925-0500 RealmHangTest[9821:1099884] Sync: Connection[1]: Connected to endpoint ‘3.210.32.164:443’ (from ‘192.168.123.207:xxxxx’)

2021-02-09 14:33:32.655541-0500 RealmHangTest[9821:1099884] Sync: Connection[2]: Session[2]: client_reset_config = false, Realm exists = true, async open = false, client reset = false

2021-02-09 14:33:32.714108-0500 RealmHangTest[9821:1099884] Sync: Connection[2]: Connected to endpoint ‘3.210.32.164:443’ (from ‘192.168.123.207:52134’)

2021-02-09 14:33:33.141745-0500 RealmHangTest[9821:1099520] Login success

No perhaps a location issue related to global - like I was having ?

ah cool just wondering whether 12.4 support was coming. :+1:

It’s 20:57 EST, and right now it is working like a champ, i.e. no hanging whatsoever. As I said in a previous post, this issue seems to be intermittent. It started happening last Friday night and throughout the weekend. @Luccas_Clezar tested the same code and it hung for him too - that was two days ago. There are times when asyncOpen() just does not come back for a few minutes, which can be very disconcerting as it is usually the first thing that happens right after a login.

I ended changing my code to a regular sync open, which comes back immediately - so I will probably stick with that as a strategy moving forward.

Hi @Richard_Krueger,

I’ve done the same with my own refactoring, but this leaves me with another detail to contend with then.
Although the UI/UX continues onward, how can I force trigger the sync engine in the background to start its work?

Doesn’t it start automatically as soon as you open the realm ? If you have sync logging on you should see the sync connection

What if I open it on a background thread that doesn’t have a run loop event?
Or the app fluctuates between offline and online, which doesn’t always instantly trigger a sync (due to perhaps more multithreading).

There’s always a UI element (button or pull-to-refresh) in my app that gives the user control to trigger a sync anytime they want, instead of waiting on the app.

This is what I experienced when using the Realm App global deployment option - have you tried using the local option so see if you get a more consistent performance.

We always open the first Realm on the main thread - even if the app doesn’t use it. Then if we need some long task we open one or more realms on background threads. Say to produce reports or import data.

There is no ability to manually trigger a sync as far as I am aware - nor should it be necessary. Usually updates come through nearly instantly - fast enough for collaboration between two users in our experience.

Well that is with the current Realm Cloud - we haven’t done comprehensive testing on MongoDB Realm yet but I expect it should also be very quick.

I am not 100% sure about whether you need to open the Realm on a runloop thread for sync to be started - I don’t think so - at least I don’t recall running into this problem with our data migration app which does everything from a background thread and syncs just fine.

I should qualify that - it is macOS so may behave differently to iOS in this regards.

1 Like

@Duncan_Groenewald you can force a run loop on a background thread, I did this in my Storage example, because I need a background thread to service the upload request. You create the background thread as follows:

        if let uid = RealmManager.shared.currentUserId {
            if self.uploadThread==nil {
                self.uploadThread = Thread(target: self, selector: #selector(uploadThreadEntryPoint(uid:)), object: uid)
                self.uploadThread!.start()
            }
            
            if let uploadThread = self.uploadThread {
                perform(#selector(setupBackground), on: uploadThread, with: nil, waitUntilDone: false, modes: [RunLoop.Mode.common.rawValue])
            }
        }

Then you define the tread entry point like this

    @objc func uploadThreadEntryPoint(uid: String) {
        autoreleasepool {
            Thread.current.name = "CosyncUploadThread_\(uid)"
            let runLoop = RunLoop.current
            runLoop.add(NSMachPort(), forMode: RunLoop.Mode.default)
            runLoop.run()
        }
    }

This runloop is running on your background thread. Then your servicing function would like this:

    @objc func setupBackground() -> Void {
        if  let user = RealmManager.shared.app.currentUser,
            let uid = RealmManager.shared.currentUserId,
            let sessionId = AssetManager.shared.sessionId {
            
            self.userRealm = try! Realm(configuration: user.configuration(partitionValue: "user_id=\(uid)"))
            
            if let realm = self.userRealm {
                let results = realm.objects(CosyncAssetUpload.self)
                    .filter("uid == '\(uid)' && sessionId=='\(sessionId)' && status=='initialized'")
                
                self.notificationToken = results.observe { [self] (changes: RealmCollectionChange) in
            
                    switch changes {
                    case .initial:
                        for assetUpload in results {
                            self.uploadAsset(assetUpload: assetUpload)
                        }
                        
                    case .update( let results, _, _, _):
                        for assetUpload in results {
                            self.uploadAsset(assetUpload: assetUpload)
                        }
                        
                    case .error(let error):
                        // An error occurred while opening the Realm file on the background worker thread
                        fatalError("\(error)")
                    }
                }
            }
        }
    }

For more information, you can check out a Medium article I wrote about multi-threading in Realm.

The main rule to remember about threading in Realm, is that whatever happens on a thread Realm-wise must stay on the thread - it’s the Vegas rule of threading. Otherwise, MongoDB Realm is perfectly thread safe and performs flawlessly.

There is no ability to manually trigger a sync as far as I am aware - nor should it be necessary

I don’t want to be at the whim of when Realm decides to sync. As it can take a while for it to figure out that it’s back online and then try to sync. I’ve tested delays of over a minute.

As a user in the field that’s late for dinner and just got back to my car with a slow data connection, the last thing I want is to wait for my app to figure it out and sync. Machines should wait on us, not the other way around.

Looks like I’ll be making a request in the SDK forum.

See Thomas comment above (excerpt below) re using the network reachability observer - you might want to see why it is taking a couple of minutes to start syncing since it should start as soon as the network becomes available. Set sync logging to “all” on the client.

When connecting fails we do have automatic backoff to avoid killing the device’s battery, but we use a network reachability observer to immediately reconnect when the OS notifies us that the networking state has changed.

@Sebastian_Dwornik and @Duncan_Groenewald

Forgive me for asking but isn’t some of this discussion focusing on circumventing a built in functionality of MongoDB Realm?

The docs and developers recommend using asyncOpen() when first connecting so why would we want to circumvent it? (Given it’s not working for you, but that leads me to think there’s external influence involved)