Welcome to the MongoDB Community Forums @Philip_Su!
Thank you so much for taking the time to provide such detailed and constructive feedback – this is definitely appreciated and very helpful for the Realm product & documentation teams, as well as other users.
Congrats also on SparkEats – your app looks great. I hope there’s an Android version on the way soon ;-).
Regards,
Stennie
Definitely, @Stennie — working on Android as we speak. Reanimated2 requires Hermes while IIUC Realm can’t quite do that yet, and so I’m porting off of Reanimated2.
Realm has been a lifesaver. Totally loving it.
This is the sort of thing that frozen objects are supposed to address. Is there something which made them not work for you, or was this one of the docs problems where they failed to introduce the concept?
@Thomas_Goyne : I somehow missed this concept. In looking at the Realm React Native SDK guide right now, through the Quick Start, Fundamentals, and even the Reacting To Changes guides, I still don’t see any references to freezing. I think the concept should be covered quite early in the RN SDK in that nearly everyone’s going to hit this problem if they’re on RN.
Thanks for sharing the concept! Does freezing an object make it spreadable as well? Because if so, I can get rid of all the object copying that I do and instead just freeze Realm objects before handing them out to React components.
No, frozen objects still aren’t spreadable. They behave exactly like normal Realm objects except for that they always access an immutable snapshot of the data rather than updating to the latest version.
This thread worries me as I am about to enable sync for a large user base. When you “restart sync” and “delete client files”, what is the implication of that? Do you delete the data of all your clients? If that would happen for my app only once, I would be out of business 
@Simon_Persson : I believe the client files are merely the local cache of what’s in the cloud. So deleting them should be Mostly Harmless™. What you lose when you delete a client file are all the changes that haven’t already been synced to the server. So, for instance, if a customer:
… then they will lose all the changes they made in step 3. But no changes prior to step 2 (say, months of data) will be lost if you delete the local client file — those changes will instead be sync’d down the next time you launch the app and it connects to the internet.
So in general, the risk when you delete local client files is that you’ll lose unsync’d changes, IIUC. There are apparently ways to even recover from that, I believe (e.g. by loading those unsync’d changes prior to deleting the client file and copying them into the online Realm) — but I’m unfortunately no expert in that.
Thank you for your response.
So I guess this is the ClientResetError you are talking about? I was thinking it was more than that. Still, just loosing a day of work for a customer could cause bad reviews on AppStore/Google play, so I really need to get the error handling for client resets right. On a sidenote… I believe this was handled automatically at one point, but got removed with the transition to MongoDb Realm (MongoDb employees maybe know?).
Thanks @Philip_Su for your detailed report! Could you share more details of what Realm Sync can do that Firestore is lacking? Would be highly appreciated.
@M_S : the main benefits I found with Realm Sync vs. Firestore are the same as what I mentioned early in the OP. Full offline capability & fast local-first data access. These two benefits were big enough, in themselves, to justify the work I did to port my entire app.
Hello @Philip_Su, I’m having trouble with Firestore related to cache and access locally a very large collection of documents (around 10K) using the onSnapshot feature while offline. That problem is more evident on low-end devices. I’m using rn-firebase too. That was your problem when you were using Firestore? And if was can you tell me if using Realm improve that experience?
@William_Fernandes : when you say you’re “having trouble,” could you describe a bit more specifically what the problem was? I’m not sure I can answer whether I was having the same problem without a more precise description.
When using Firestore with enablePersistence on, I found that the app often still would hang on launch if there was poor / no connectivity. This is the main thing I moved to Realm in order to fix. Since Realm is offline-first with sync (instead of online-first with local persistence), it more reliably allows pure-offline use.
That said, I’ve never once had a server issue with Firestore, whereas with Realm, I still get client resets and server sync terminations/pauses every once in a while (usually for reasons which are still opaque to me). Remember that if you don’t author local client code that ports all changes to a new sync’d file whenever you get CLIENT_RESET, you’re going to lose all local changes for the period between when Realm Sync pauses/stops itself and when you next restart the client. Since that code was too hard for me to write/test correctly, I’ve essentially compromised by accepting a design where clients lose data for the time between when Realm Sync stops itself and when I’m able to reboot/restart it.
If the above paragraph doesn’t make sense to you, don’t worry. The TL;DR is basically that Realm Sync has solved my offline-app-launch problem completely, but introduced a variety of data loss scenarios which I fix instance-by-instance by hand.
Having trouble means the application is slow when it performs a query on top of the data that is cached like: and when the cached data is too big the app freezes.
Ex: collection('data').where('x', '==', 'x').onSnapshot(...), this will cache all those documents on the device, overtime when this data grown the app starts to become slow, and when I tried to update some document from this collection the app slows even more until it freezes completely. The bigger this collection the worse it gets.
I believe you local data on realm is big, did you test if it is slow on low-end devices, like with 1GB of RAM when you try to update the data or when it grows?
A variety of data-loss scenarios…. Now you are scaring me again.
How many active users do you have? How do you manuellt my fix these errors?
I have a big userbase and data loss is unacceptable 
Anyone from Realm want to comment? Is automatic client resets in the works?
Yes, we have improvements in progress there. We recently shipped an automatic client reset mode which discards any local unuploaded changes (which is an improvement over discarding any unuploaded changes and all new changes until the app is restarted), and we are currently working on a mode which will automatically recover unuploaded changes when possible.
My local data is small, so I can’t comment on Realm’s relative speed when data gets large. However, when it comes to your Firestore example, I think it’s expected behavior that onSnapshot can get slow as your data gets larger, because every time you launch your app, onSnapshot will play through all the data that meets the criteria as if they’re changes.
My app is tiny, and has nowhere near the data-loss sensitivity of many other types of apps. The most recent data-loss happened when Realm Sync paused itself, and then kept red-error’ing when I tried to restart it. It turns out that it red-error’d because there were four documents (out of perhaps a few thousand) that didn’t conform to the expected schema (specifically, they had an empty field when the schema expected a non-empty string). I fixed this by manually pulling up those four documents and manually setting non-empty strings to those fields.
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
The lesson there (for me) is to hop on Sync pauses right away. I was taking my sweet time because I assumed I could just unpause Sync and everyone would be caught up. I in no way expected Sync to refuse to be unpaused after a certain number of weeks because some sort of log data is flushed.
It turns out that it red-error’d because there were four documents (out of perhaps a few thousand) that didn’t conform to the expected schema (specifically, they had an empty field when the schema expected a non-empty string).
We have a fix for this to do this automatically for you coming in the next cloud release.
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
So this is because Sync depends on the Atlas changestream. If you continue to make writes to your Atlas cluster while Sync is not consuming the changestream then too many changes could accumulate and the resume token is lost. We do have an idea to automatically resume this for users but we want to ship the automatic Client Reset with recovery as Thomas mentioned above, as automatically recovering could trigger a client reset. See docs below -
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
So this let me worried. I´m actually making a system that generate invoices. The system is offline first. The realm sync data will used to run some queries for reporting in some website. Maybe some remote price changes but the invoices are going to be always generated on the devices.
Are you telling me that it could be a case of months of data loss (in this case, invoices) ?
When this occurs you get multiple alerts sent to you prompting the operator to respond. If you are in the Cloud UI you will also see a giant red banner alerting you to the issue.