EventGet 50% off your ticket to MongoDB.local NYC on May 2. Use code Web50!Learn more >>
MongoDB Developer
Realm
plus
Sign in to follow topics
MongoDB Developer Centerchevron-right
Developer Topicschevron-right
Productschevron-right
Realmchevron-right

Realm Data and Partitioning Strategy Behind the WildAid O-FISH Mobile Apps

Andrew Morgan12 min read • Published Feb 14, 2022 • Updated Jun 12, 2023
Realm
Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
In 2020, MongoDB partnered with the WildAid Marine Protection Program to create a mobile app for officers to use while out at sea patrolling Marine Protected Areas (MPAs) worldwide. We implemented apps for iOS, Android, and web, where they all share the same Realm back end, schema, and sync strategy. This article explains the data architecture, schema, and partitioning strategy we used. If you're developing a mobile app with Realm, this post will help you design and implement your data architecture.
MPAs—like national parks on land—set aside dedicated coastal and marine environments for conservation. WildAid helps enable local agencies to protect their MPAs. We developed the O-FISH app for enforcement officers to search and create boarding reports while they're out at sea, patrolling the MPAs and boarding vessels for inspection.
O-FISH needed to be a true offline-first application as officers will typically be without network access when they're searching and creating boarding reports. It's a perfect use case for the Realm mobile database and MongoDB Realm Sync.
This video gives a great overview of the WildAid Marine Program, the requirements for the O-FISH app, and the technologies behind the app:
This article is broken down into these sections:

The O-FISH Application

There are three frontend applications.
The two mobile apps (iOS and Android) provide the same functionality. An officer logs in and can search existing boarding reports, for example, to check on past reports for a vessel before boarding it. After boarding the boat, the officer uses the app to create a new boarding report. The report contains information about the vessel, equipment, crew, catch, and any laws they're violating.
Crucially, the mobile apps need to allow users to view and create reports even when there is no network coverage (which is the norm while at sea). Data is synchronized with other app instances and the backend database when it regains network access.
iOS O-FISH App in Action
iOS O-FISH App in Action
The web app also allows reports to be viewed and edited. It provides dashboards to visualize the data, including plotting boardings on a map. User accounts are created and managed through the web app.
All three frontend apps share a common backend Realm application. The Realm app is responsible for authenticating users, controlling what data gets synced to each mobile app instance, and persisting the data to MongoDB Atlas. Multiple "agencies" share the same frontend and backend apps. An officer should have access to the reports belonging to their agency. An agency is an authority responsible for enforcing the rules for one or more regional MPAs. Agencies are often named after the country they operate in. Examples of agencies would be Galapogas or Tanzania.

System Architecture

The iOS and Android mobile apps both contain an embedded Realm mobile database. The app reads and writes data to that Realm database-whether the device is connected to the network or not. Whenever the device has network coverage, Realm synchronizes the data with other devices via the Realm backend service.
O-FISH System Architecture
O-FISH System Architecture
The Realm database is embedded within the mobile apps, each instance storing a partition of the O-FISH data. We also need a consolidated view of all of the data that the O-FISH web app can access, and we use MongoDB Atlas for that. MongoDB Realm is also responsible for synchronizing the data with the MongoDB Atlas database.
The web app is stateless, accessing data from Atlas as needed via the Realm SDK.
MongoDB Charts dashboards are embedded in the web app to provide richer, aggregated views of the data.

Data Partitioning

MongoDB Realm Sync uses partitions to control what data it syncs to instances of a mobile app. You typically partition data to limit the amount of space used on the device and prevent users from accessing information they're not entitled to see or change.
When a mobile app opens a synced Realm, it can provide a partition value to specify what data should be synced to the device.
As a developer, you must specify an attribute to use as your partition key. The rules for partition keys have some restrictions:
  • All synced collections use the same attribute name and type for the partition key.
  • The key can be a string, objectId, or a long.
  • When the app provides a partition key, only documents that have an exact match will be synced. For example, the app can't specify a set or range of partition key values.
A common use case would be to use a string named "username" as the partition key. The mobile app would then open a Realm by setting the partition to the current user's name, ensuring that the user's data is available (but no data for other users).
If you want to see an example of creating a sophisticated partitioning strategy, then Building a Mobile Chat App Using Realm – Data Architecture describes RChat's approach (RChat is a reference mobile chat app built on Realm and MongoDB Realm). O-FISH's method is straightforward in comparison.
WildAid works with different agencies around the world. Each officer within an agency needs access to any boarding report created by other officers in the same agency. Photos added to the app by one officer should be visible to the other officers. Officers should be offered menu options tailored to their agency—an agency operating in the North Sea would want cod to be in the list of selectable species, but including clownfish would clutter the menu.
We use a string attribute named agency as the partitioning key to meet those requirements.
As an extra level of security, we want to ensure that an app doesn't open a Realm for the wrong partition. This could result from a coding error or because someone hacks a version of the app. When enabling Realm Sync, we can provide expressions to define whether the requesting user should be able to access a partition or not.
Expression to Limit Sync Access to Partitions
Expression to Limit Sync Access to Partitions
For O-FISH, the rule is straightforward. We compare the logged-in user's agency name with the partition they're requesting to access. The Realm will be synced if and only if they're the same:

Data Schema

At the highest level, the O-FISH schema is straightforward with four Realms (each with an associated MongoDB Atlas collection):
  • DutyChange records an officer going on-duty or back off-duty.
  • Report contains all of the details associated with the inspection of a vessel.
  • Photo represents a picture (either of one of the users or a photo that was taken to attach to a boarding report).
  • MenuData contains the agency-specific values that officers can select through the app's menus.
O-FISH Data Model
You might want to right-click that diagram so that you can open it in a new tab!
Let's take a look at each of those four objects.

DutyChange

The app creates a DutyChange object when a user toggles a switch to flag that they are going on or off duty (at sea or not).
O-FISH Duty Change
These are the Swift and Kotlin versions of the DutyChange class:
Swift
Kotlin
On iOS, DutyChange inherits from the Realm Object class, and the attributes need to be made accessible to the Realm SDK by making them dynamic and adding the @objc annotation. The Kotlin app uses the @RealmClass annotation and inheritance from RealmObject.
Note that there is no need to include the partition key as an attribute.
In addition to primitive attributes, DutyChange contains user which is of type User:
Swift
Kotlin
User objects are always embedded in higher-level objects rather than being top-level Realm objects. So, the class inherits from EmbeddedObject rather than Object in Swift. The Kotlin app extends the @RealmClass annotation to include (embedded = true).
Whether created in the iOS or Android app, the DutyChange object is synced to MongoDB Atlas as a single DutyChange document that contains a user sub-document:
There's a Realm schema associated with each collection that's synced with Realm Sync. The schema can be viewed and managed through the Realm UI:
O-FISH Duty Change Schema

Report

The Report object is at the heart of the O-FISH app. A report is what an officer reviews for relevant data before boarding a boat. A report is where the officer records all of the details when they've boarded a vessel for an inspection.
In spite of appearances, it pretty straightforward. It looks complex because there's a lot of information that an officer may need to include in their report.
Starting with the top-level object - Report:
Swift
Kotlin
The Report class contains Realm List s (RealmList in Kotlin) to store lists of instances of classes such as CrewMember.
Some of the classes embedded in Report contain further embedded classes. There are 19 classes in total that make up a Report. You can view all of the component classes in the iOS and Android repos.
Once synced to Atlas, the Report is represented as a single BoardingReports document (the name change is part of the schema definition):
O-FIsh Boarding Reports Document
Note that Realm lists are mapped to JSON/BSON arrays.

Photo

A single boarding report could contain many large photographs, and so we don't want to embed those within the Report object (as an object could grow very large and even exceed MongoDB's 16 MB document limit). Instead, the Report object (and its embedded objects) store references to Photo objects. Each photo is represented by a top-level Photo Realm object. As an example, Attachments contains a Realm List of strings, each of which identifies a Photo object. Handling images will step through how we implemented this.
Swift
Kotlin
The general rule is that it isn't the best practice to store images in a database as they consume a lot of valuable storage space. A typical solution is to keep the image in some store with plentiful, cheap capacity (e.g., a block store such as cloud storage - Amazon S3 of Microsoft Blob Storage.) The O-FISH app's issue is that it's probable that the officer's phone has no internet connectivity when they create the boarding report and attach photos, so uploading them to cloud object storage can't be done at that time. As a compromise, O-FISH stores the image in the Photo object, but when the device has internet access, that image is uploaded to cloud object storage, removed from the Photo object and replaced with the S3 link. This is why the Photo includes both an optional binary picture attribute and a pictureURL field for the S3 link:
Swift
Kotlin
Note that we include the referencingReportID attribute to make it easy to delete all Photo objects associated with a Report.
The officer also needs to review past boarding reports (and attached photos), and so the Photo object also includes a thumbnail image for off-line use.
Each agency needs the ability to customize what options are added in the app's menus. For example, agencies operating in different countries will need to define the list of locally applicable laws. Each agency has a MenuData instance with a list of strings for each of the customizable menus:
Swift
Kotlin

Handling images

When MongoDB Realm Sync writes a new Photo document to Atlas, it contains the full-sized image in the picture attribute. It consumes space that we want to free up by moving that image to Amazon S3 and storing the resulting S3 location in pictureURL. Those changes are then synced back to the mobile apps, which can then decide how to get an image to render using this algorithm:
  1. If picture contains an image, use it.
  2. Else, if pictureURL is set and the device is connected to the internet, then fetch the image from cloud object storage and use the returned image.
  3. Else, use the thumbNail.
O-FISH photo handling
When the Photo document is written to Atlas, the newPhoto database trigger fires, which invokes a function named newPhoto function.
O-FISH newPhoto trigger
The trigger passes the newPhoto Realm function the changeEvent, which contains the new Photo document. The function invokes the uploadImageToS3 Realm function and then updates the Photo document by removing the image and setting the URL:
uploadImageToS3 uses Realm's AWS integration to upload the image:

Summary

We've covered the common data model used across the iOS, Android, and backend Realm apps. (The web app also uses it, but that's beyond the scope of this article.)
The data model is deceptively simple. There's a lot of nested information that can be captured in each boarding report, resulting in 20+ classes, but there are only four top-level classes in the app, with the rest accounted for by embedding. The only other type of relationship is the references to instances of the Photo class from other classes (required to prevent the Report objects from growing too large).
The partitioning strategy is straightforward. Partitioning for every class is based on the name of the user's agency. That pattern is going to appear in many apps—just substitute "agency" with "department," "team," "user," "country," ...
Suppose you determine that your app needs a different partitioning strategy for different classes. In that case, you can implement a more sophisticated partitioning strategy by encoding a key-value pair in a string partition key.
For example, if we'd wanted to partition the reports by username (each officer can only access reports they created) and the menu items by agency, then you could partition on a string attribute named partition. For the Report objects, it would be set to pairs such as partition = "user=bill@some-domain.com" whereas for a MenuData object it might be set to partition = "agency=Galapagos". Building a Mobile Chat App Using Realm – Data Architecture steps through designing these more sophisticated strategies.

Resources

If you have questions, please head to our developer community website where the MongoDB engineers and the MongoDB community will help you build your next big idea with MongoDB.

Facebook Icontwitter iconlinkedin icon
Rate this tutorial
star-empty
star-empty
star-empty
star-empty
star-empty
Related
Code Example

Building a Mobile Chat App Using Realm – The New and Easier Way


Jul 18, 2023 | 22 min read
Tutorial

Saving Data in Unity3D Using PlayerPrefs


Sep 07, 2022 | 11 min read
Tutorial

Saving Data in Unity3D Using BinaryReader and BinaryWriter


Sep 07, 2022 | 12 min read
Article

Making SwiftUI Previews Work For You


Mar 06, 2023 | 13 min read
Table of Contents