I recently bit the bullet on SparkEatz and would like to share my experiences with the community in case it helps anyone dealing with one or both systems (Realm Sync + Firestore). I am a lot more familiar with Firestore, having used it on many projects, and so this may inform a lot of the opinions below.
First off, Realm Sync does an amazing job when it works — and it solves the key problem I was trying to address when porting from Firestore to Realm Sync in the first place: full offline capability as well as fast local-first data access. SparkEatz would not be where it is today without Realm Sync — thanks to the whole Realm team for a job very well done!
Overall, the biggest qualitative difference by far, at least from my perspective having launched only one production app on Realm, is that Realm Sync is far more fragile than Firestore. Perhaps this is just a reflection of my relative newness to the platform. But when I develop on Firestore, nothing ever seems to go wrong — sure, I can get schemas mixed up, occasionally try to read a Firestore Timestamp before it gets back down from the server, or mess up a trigger. But Firestore never takes down my app. Realm, however, frequently crashes the entire app if I do the wrong thing or if the wind blows the wrong way. Client resets, error logs on inability to connect to the network, etc etc. The times when I’ve had to Terminate Sync and Restart are always fraught with anxiety that it might all not reboot and that I might need to manually delete all client files in my code to recover. Note: I’m well aware that perhaps Realm Sync is rock solid and that I’m just ranting like a naive n00b — but I’m conveying my qualitative experience of developing on Realm Sync vs. Firestore, which is that for me, personally, I’m always afraid that Realm might crash the app if I cough wrong, whereas this has never been a fear of mine with Firestore (via RNFirebase).
The biggest functionality difference I’ve had to wrestle with is the semantics of permissioning: who can access which objects. Realm’s model is per-realm (i.e. all objects in a Realm have the same read/write permissions). Firestore’s model is per document path, which some pretty interesting deeper semantics (e.g. “you may only read this doc if you’re listed in the ‘subscribers’ field of the doc”). Realm’s monolithic permission model makes building multi-user apps quite tricky. For instance, consider an app where you and a friend can share some objects in a group – e.g. photos of cats. Now you want to add a new friend to the group as well. To do this in Firestore, you just have to author the initial permissions correctly: whoever’s in the group can read the object. To do this in Realm, you need to create a new Realm with all three people, copy all the objects from the original two-person Realm into the three-person, and delete the old Realm. Besides the trivial cat photos example, these sorts of permission semantics come up all the time in typical projects. So it ends up, at least for me, easiest to only use Realm as a per-user local store of their own objects. I keep all shared cross-user objects in Firestore. For instance, I want all users to read each other’s profiles, but only the user themselves may change their own profile. My understanding is that this isn’t possible with Realm permission semantics; this setup is trivial in Firestore.
The UI for managing Realm Sync on MongoDB feels a bit janky relative to the fully-integrated UI of Firebase/Firestore. There’s stuff you can see in parts of the UI that you shouldn’t modify because you should use the Sync tab instead (IIUC); there’s hopping between the Realm tab and the Atlas tab in order to check out some data; there are several changes you can make in the Schema or Permissions areas which, IIRC, lead to a bunch of website error popups that then require Sync Termination to recover from (and even an occasional realm-cli push). This might all be an artifact of Realm getting integrated into MongoDB; it feels a bit like “shipping the org chart.”
The documentation on Realm is worse than Firestore, making the learning curve steep (IMO). Understanding Realm is a lot more about searching the web and this forum for Q&A type solutions than it is about reading the docs and just getting set up and started.
Lemma: key topics required for any real production deployment should be covered in more depth, with more examples and explanation. Three that come to mind are: a) how really to handle CLIENT_RESET fully (including data porting, local file deletion, launching the new client file, etc), b) the subtle differences between opening a Realm synchronously and asynchronously, including the gotchas, and c) best practices around schema changes. More on that below.
When it comes to schema changes, I find the Realm Sync documents fall just a bit short of what you need to know as someone running a production service. For instance, the list of destructive changes finally made it from a forum post here into the docs – but not in all the places, just in one place IIRC that you happen to need to see. But furthermore, I think a clear explanation of how to deploy a schema change end-to-end in production should be covered somewhere. Some of the docs say to go back into Development Mode, but I can’t imagine that’s what’s really advised in the production DB (or please correct me if indeed it is the right thing!). So I end up hand-modifying the schema in the Web UI first, then updating the schema in my React Native Realm object, and finally deploying/launching the new code. I’m still not even sure if I’m doing the right thing here.
On the learning curve bit: there are some real oddities in Realm’s JS/TS implementation. It took me quite a bit of debugging and web sleuthing before figuring out that the JS objects that Realm returns cannot be spread. That is, no {...soup} for you. The Realm objects need you to access them by property name explicitly to fully instantiate / par-boil them. So to preserve reasonable semantics in the rest of my codebase, I implement a derealm function on all my Realm objects where I explicitly reference each property before returning it to the client code.
Similarly, one needs to be careful mixing Realm with React because of differences in expectation around data-consistency / data-currency. One common example is deleting an object from Realm – you better make sure none of your React objects are holding on to JS related to that object because you’ll get an error when they render, even if it’s the dying-gasp last render prior to React also figuring out the object is now deleted. Unlike most React work, you need to deal with Realm objects the way you deal with useRef – basically realize that the underlying object can change without your React code necessarily detecting it prior to a re-render. To make these semantics easier, I also derealm all objects (copy all Realm properties into a separate non-Realm JS object) before handing them to a React component so that React semantics can be maintained more easily.
Overall, adopting Realm Sync has been a great boon to my application. It lets me accomplish things that I never could have without it. And when it’s working, it really is pretty fabulous. But I find that I can’t move off Firestore because it offers multi-user permissioning that’s much more suitable for many common app scenarios. And I find that my general experience developing on Realm is that it’s hard to know what’s going on, the documentation is sparse, and there’s always a chance it’ll crash the app if I do something wrong — I’m basically always breathing lightly near it, tiptoeing by in order not to Wake the Giant™.
I hope this feedback is helpful to perhaps the team or to other people considering Realm Sync vs. Firestore. SparkEatz couldn’t be where it is today without Realm Sync — it’s an invaluable platform that has some amazing functionality for which I’m deeply grateful. My hope is that as the Realm community grows, the types of issues I mention above can be addressed over time.
Welcome to the MongoDB Community Forums @Philip_Su!
Thank you so much for taking the time to provide such detailed and constructive feedback – this is definitely appreciated and very helpful for the Realm product & documentation teams, as well as other users.
Congrats also on SparkEats – your app looks great. I hope there’s an Android version on the way soon ;-).
Definitely, @Stennie — working on Android as we speak. Reanimated2 requires Hermes while IIUC Realm can’t quite do that yet, and so I’m porting off of Reanimated2.
This is the sort of thing that frozen objects are supposed to address. Is there something which made them not work for you, or was this one of the docs problems where they failed to introduce the concept?
@Thomas_Goyne : I somehow missed this concept. In looking at the Realm React Native SDK guide right now, through the Quick Start, Fundamentals, and even the Reacting To Changes guides, I still don’t see any references to freezing. I think the concept should be covered quite early in the RN SDK in that nearly everyone’s going to hit this problem if they’re on RN.
Thanks for sharing the concept! Does freezing an object make it spreadable as well? Because if so, I can get rid of all the object copying that I do and instead just freeze Realm objects before handing them out to React components.
No, frozen objects still aren’t spreadable. They behave exactly like normal Realm objects except for that they always access an immutable snapshot of the data rather than updating to the latest version.
This thread worries me as I am about to enable sync for a large user base. When you “restart sync” and “delete client files”, what is the implication of that? Do you delete the data of all your clients? If that would happen for my app only once, I would be out of business
@Simon_Persson : I believe the client files are merely the local cache of what’s in the cloud. So deleting them should be Mostly Harmless™. What you lose when you delete a client file are all the changes that haven’t already been synced to the server. So, for instance, if a customer:
Launches your app
Turns on airplane mode
Makes a bunch of changes via your app to the local Realm DB
Deletes the client file (whether manually or through your code doing so)
Turns off airplane mode
… then they will lose all the changes they made in step 3. But no changes prior to step 2 (say, months of data) will be lost if you delete the local client file — those changes will instead be sync’d down the next time you launch the app and it connects to the internet.
So in general, the risk when you delete local client files is that you’ll lose unsync’d changes, IIUC. There are apparently ways to even recover from that, I believe (e.g. by loading those unsync’d changes prior to deleting the client file and copying them into the online Realm) — but I’m unfortunately no expert in that.
So I guess this is the ClientResetError you are talking about? I was thinking it was more than that. Still, just loosing a day of work for a customer could cause bad reviews on AppStore/Google play, so I really need to get the error handling for client resets right. On a sidenote… I believe this was handled automatically at one point, but got removed with the transition to MongoDb Realm (MongoDb employees maybe know?).
Thanks @Philip_Su for your detailed report! Could you share more details of what Realm Sync can do that Firestore is lacking? Would be highly appreciated.
@M_S : the main benefits I found with Realm Sync vs. Firestore are the same as what I mentioned early in the OP. Full offline capability & fast local-first data access. These two benefits were big enough, in themselves, to justify the work I did to port my entire app.
Hello @Philip_Su, I’m having trouble with Firestore related to cache and access locally a very large collection of documents (around 10K) using the onSnapshot feature while offline. That problem is more evident on low-end devices. I’m using rn-firebase too. That was your problem when you were using Firestore? And if was can you tell me if using Realm improve that experience?
@William_Fernandes : when you say you’re “having trouble,” could you describe a bit more specifically what the problem was? I’m not sure I can answer whether I was having the same problem without a more precise description.
When using Firestore with enablePersistence on, I found that the app often still would hang on launch if there was poor / no connectivity. This is the main thing I moved to Realm in order to fix. Since Realm is offline-first with sync (instead of online-first with local persistence), it more reliably allows pure-offline use.
That said, I’ve never once had a server issue with Firestore, whereas with Realm, I still get client resets and server sync terminations/pauses every once in a while (usually for reasons which are still opaque to me). Remember that if you don’t author local client code that ports all changes to a new sync’d file whenever you get CLIENT_RESET, you’re going to lose all local changes for the period between when Realm Sync pauses/stops itself and when you next restart the client. Since that code was too hard for me to write/test correctly, I’ve essentially compromised by accepting a design where clients lose data for the time between when Realm Sync stops itself and when I’m able to reboot/restart it.
If the above paragraph doesn’t make sense to you, don’t worry. The TL;DR is basically that Realm Sync has solved my offline-app-launch problem completely, but introduced a variety of data loss scenarios which I fix instance-by-instance by hand.
Having trouble means the application is slow when it performs a query on top of the data that is cached like: and when the cached data is too big the app freezes.
Ex: collection('data').where('x', '==', 'x').onSnapshot(...), this will cache all those documents on the device, overtime when this data grown the app starts to become slow, and when I tried to update some document from this collection the app slows even more until it freezes completely. The bigger this collection the worse it gets.
I believe you local data on realm is big, did you test if it is slow on low-end devices, like with 1GB of RAM when you try to update the data or when it grows?
Yes, we have improvements in progress there. We recently shipped an automatic client reset mode which discards any local unuploaded changes (which is an improvement over discarding any unuploaded changes and all new changes until the app is restarted), and we are currently working on a mode which will automatically recover unuploaded changes when possible.
My local data is small, so I can’t comment on Realm’s relative speed when data gets large. However, when it comes to your Firestore example, I think it’s expected behavior that onSnapshot can get slow as your data gets larger, because every time you launch your app, onSnapshot will play through all the data that meets the criteria as if they’re changes.
My app is tiny, and has nowhere near the data-loss sensitivity of many other types of apps. The most recent data-loss happened when Realm Sync paused itself, and then kept red-error’ing when I tried to restart it. It turns out that it red-error’d because there were four documents (out of perhaps a few thousand) that didn’t conform to the expected schema (specifically, they had an empty field when the schema expected a non-empty string). I fixed this by manually pulling up those four documents and manually setting non-empty strings to those fields.
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
The lesson there (for me) is to hop on Sync pauses right away. I was taking my sweet time because I assumed I could just unpause Sync and everyone would be caught up. I in no way expected Sync to refuse to be unpaused after a certain number of weeks because some sort of log data is flushed.
It turns out that it red-error’d because there were four documents (out of perhaps a few thousand) that didn’t conform to the expected schema (specifically, they had an empty field when the schema expected a non-empty string).
We have a fix for this to do this automatically for you coming in the next cloud release.
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
So this is because Sync depends on the Atlas changestream. If you continue to make writes to your Atlas cluster while Sync is not consuming the changestream then too many changes could accumulate and the resume token is lost. We do have an idea to automatically resume this for users but we want to ship the automatic Client Reset with recovery as Thomas mentioned above, as automatically recovering could trigger a client reset. See docs below -
Upon then un-pausing Realm Sync, it told me that it couldn’t unpause. Something about too many days having elapsed, about some sort of logs being lost in the meantime. So my only choice at that point was to stop the service and to restart Sync, which causes Client Reset™ which then means everyone on my app lost perhaps two months of their data.
So this let me worried. I´m actually making a system that generate invoices. The system is offline first. The realm sync data will used to run some queries for reporting in some website. Maybe some remote price changes but the invoices are going to be always generated on the devices.
Are you telling me that it could be a case of months of data loss (in this case, invoices) ?
Welcome to the MongoDB Community Forums 
Anyone from Realm want to comment? Is automatic client resets in the works?