We use partition-based sync for our mobile app. In our public realm, we have an Event (ID, name, start time, end time, etc). In our user realm, we have EventRegistration (EventID, etc). We have a requirement to show a page with all Events grouped by whether or not the user is registered for it. I would love to be able to use GroupedRealmCollection because we could rely completely upon the Realm library to sort the Events into groups. However, it doesn’t seem possible because Event and EventRegistration are in different realms. Am I missing something?
I also realize we could use backlinks for this use case. But again: do they work across realms?
Probably what I need is some best-practices guidance for designing realms to simplify querying across them.
Short no, long answer yes.
You can build and put in an infinite number of Realm apps provided they all use different keys, yada yada, but they don’t talk to each other.
So if you want them to interact with each other, it wouldn’t be directly with each other. What you need to do is build packages to take the information Realm is bringing in, and convert/modify accordingly to new JSON documents, with a third Realm to ingest the documents and project the changes your service is creating.
So far this is the only means I’ve successfully created results like you’re asking for. (Mind you this was in April of 2022) In a lot of cases this can be a hindrance, while some it can be a major security feature as some compliance requirements even if the app holds the same information, that information has to be segregated somehow.
How I did it:
I synced MongoDB Realm instances to two separate Core Data’s, and then synced the two core data’s so they were “one” core data but both Realms could see the information. This is how I made two Realm instances in the same app see each others collections and from there, I used Swift packages I built to transform results as necessary which then were ingested by a third RealmDB, and then whatever data wasn’t necessary from the other two collections were deleted.
Now thinking about it, you could do this without the third, but the third just made it easier to organize.
Also, you can do the above I mentioned to make two Atlas DBs communicate with one another via Realm in the same mobile application. You take the Realm App from one Atlas cluster, and a Realm App from another, implement device sync, implement both Realm apps locally into the app and then connect both Realms to the Core Data in iOS or Rooms in Android, and then each Atlas DB can be synced to each other + the app.
You can also do this to make a failover. So if Realm in one Atlas has a Bad changeset, you won’t care so much because the other Realm app will keep you running, and CoreData will maintain integrity between both Realms, so when you term and resync the failed Realm, you won’t lose unsynced data.
Whoa. Seems complicated just to achieve grouping data in a front-end UI.
Something I didn’t explain in my original question: We currently have both Events and EventRegistrations in the “user” realm, but we don’t like this configuration because it means the size of the data in Mongo grows so fast as you add more users. Because the Event data is duplicated for each user. So, the idea is to put Events into a “public” realm - this way, we can just store one copy in the back end, and it will sync down to all devices. Problem is, it seems we can’t group the Events using out-of-the-box Realm functionality (e.g. GroupedRealmCollection or backlinks). So, we may end up with a custom solution for this very common problem, which seems messed up. Realm should support this, no?
So, this is the Mongo/Realm design issue that we are up against.
Oh, I would just use one collection app for users and one for event, and just use app logic to confirm if a user has paid for, checked into, etc an event as the second to just display the events.
To be honest with you, there’s actually a lot with an app like that you need to consider, you also should implement another collection for the user demographic data, another for user and event metrics like how often a user attends, how often an event is viewed in the app etc as well.
You actually need separate apps anyway that can’t talk to each other, because you’re going to need a separate collection/realm app for transactions for a myriad of compliance regulations, and making accounting easier.
Such as referencing the user object ID and tying transactions to the user, as you can’t have credit card information etc. be exposed to other user data, should a breach occur on say a users private info, that’s going to be different from a breach occurring and exposing a users credit card and having it also identify the user and the users other private info.
Keeping these separated can save you potentially millions in fines by the FCRA and FDIC when they turn around and go after the source of a breach that caused CC Fraud. You do want to make sure that you set this up if you plan to store anything credit card related. Otherwise you’d want to make sure you use a service like Stripe, or WooCommerce, Apple Pay, etc.
Another reason is you want to have an easy way to see user financial transactions and make things easier for your bookkeeper for tax withholdings and so on. you’ll want another collection with aggregations in Atlas to compute everything for you. As you’ll also need to make sure all federal and state taxes are observed on the users, too which vary state by state that you’ll need to make sure are in place. This is all easier to organize with separate collections and applicable Realm apps to just route it all where needed, but you can of course keep that within the user category but it would mean more computing needed, more aggregations and indexes to separate the information from the rest of the user information and event information.
In recap, have each user get assigned a specific user ID, and have that User ID be what ties a user to transactions in a separate collection, separate realm app.
Transaction data must be kept for 7 years for compliance, and will always be auditable by the IRS and any state revenue department. Planning your Realm Apps for this app accordingly will actually save you a lot of time and prevent a lot of problems if you play this smart. And it’ll keep all of your realm data small respectively.
Hi there! Apologies if I’m missing something, but it’s still not clear to me from this why you’re splitting the data across two separate Realms. Nothing you’ve described requires it, I think, and as you’ve pointed out grouping the Events would be much easier if all the data was in the same Realm.
As Brock suggests, you can avoid duplication of Events if you had separate Tables for Events and Users, instead of duplicating each Event under each user.
Could you elaborate why they are separated?
I think what they are trying to do is shorten the size of the collections themselves, which splitting events from users would be ideal, as then the event you just tie in via transactions who attended the event by the unique user IDs.
But realistically though, at least for financial data, they would want that on another collection for accounting and regulatory simplicity. A lot of aspects in an app like that if it involves commercial sales, particularly for the metrics and user activity. What events a user views more, how long do they spend reading an event description etc.
I can see where multiple realm apps in the same mobile app would be ideal like that, as organically on the cluster each Realm App is a collection of its own, and just a single app can balloon in size which is what the OP is realizing.
Thanks all, we really appreciate all the feedback. By the way I’m on the same team as Philip Hadley and what we are really looking for here is to make sure our understanding is correct and possibly get some recommendations.
To recap: the business application simply deals with events (in that any user can see all current and upcoming events) as well as registrations (a user can be registered to 0 or many events and only they can see their registration info).
As Philip explain in the first iteration in the app we design our documents to contain both the event info, the user contact info, and the registration if the user was registered for an event. Note that we still created a document event when a user was not registered (just without the registration Id). This approach was to ensure that every user of the app fetch all events via the user partition-based sync. As Brock mentioned this meant that the size of the collection on the server would quickly grow and become a burden to manage is that updating 1 event end date for example meant we needed to update the entire set of documents for all users.
For this reason, we are changing the implementation now such that events will be in a global partition while registrations would remain as a user partition. While this is solving the initial problem of data proliferation on the mongo side, we came to understand that it is not possible to directly query the data against multiple realms even when there is “loose relationship” (a user registration contains the event code/ID). From the information we have seen this far it would seem that we need to perform 2 queries and relate the data within the app with some custom code. While this is possible it does feel like the sync created a problem that Realm was design to solve (in that Realm can maintain relationships within a data model).
Is this conclusion correct? Meaning does the partition sync take away the data relationships that previously could be define in realm? Are there any work arounds? The current example is somewhat simple, however as the app grows, for future use cases we are thinking of migrating to the flexible sync which would create more partitions and further “break” the data model relationships.
This is why I was suggesting having Events separate from users, and you use app logic to then tie the users to the event so then you only have one event for everyone, and the event itself keeps a list of users who signed up to attend.
Personally would go Flexible Sync outright from the beginning as it’s simpler and solves a lot of problems you can experience otherwise.
Thanks for raising the concern. We have not heard from you in a while, could you please confirm this issue has been resolved for you?
Partition Sync does not take away relationships. It’s defined differently on the cloud i.e. in MongoDB. You can read more information on Realm Relationships here.
