Hello everyone,
I am pretty close to going in production with a mobile application that uses Realm.
While testing different scenarios however I found a problem, I did not really think of before.
I created my own CI/CD that does the following:
Build Android application
Build iOS application
Deploy both applications to the store
Promote Realm vom Dev to Prod
Now however arises the following problem:
The moment I promote Realm from Dev to Prod, the applications in the stores are not yet live (this can take up to a few days). And even if they are, I can not ensure that the users will even update the application on time. That means, that my Realm is always “ahead” of my application. So if I - for example - rename a field in Realm, the application of all users will break in the second, I promote my schema from Dev to Prod.
All the migration logic that I could put into my application to compensate this will thus have no effect, as they dont have the application yet.
I am kind of lost now, how the procedure should look like. I thought of following scenario and wanted to ask for feedback first:
Build iOS + Android application
Deploy them to the store with a fixed release date (e.g. 5 days in the future) in the future
Create a CRON job that runs in 5 days and promotes Realm Dev to Prod
Block the users in the application until they update (problem)
Don’t touch Dev for 5 days (problem)
I hope I made my problem clear and there are already some best practices.
Hello @Thomas_Anderl : Previously when I have encountered similar situation, we always mark urgent release for iOS and keep the Android App ready in pipeline and once both are ready we force update all the previous build user. (Didn’t impact us, we had 400K DAU)
And not sure if you meant dev branch by “Don’t douch Dev for 5 days”, we normally create a release candidate once code is ready for production from Dev branch, which unblocks future release work.
As for breaking schema changes, the best advice I can offer is to not do them. If at all possible it is best to just perform additive schema changes in production.
We are trying to think about how to make this a better experience for users though, so I am curious if you can go into more detail about what kinds of changes you expect to make in production and why?
Hey, only doing additive changes is already a good suggestion.
I plan on adding additional features (e.g. profile verification). Most of them could be added by making the fields optional and nothing should break.
However when a field of a future feature becomes mandatory, it will get tricky.
So far I usually change datatypes often. As there is no enum support, for example, I sometimes switch between saving keys of enums as strings vs integers.
Another thing I might change in the future is references. As it is not possible to query links with sync directly, there is a chance that I need to change links to ObjectIds to be able to query them.
To load all messages for a chat, for example, I need to store the Chat in the Message as ObjectId to sync them. If I ever decide to add different queries for performance reasons, I will need to make breaking changes to the schems.
This is great. Thank you for your feedback. I have added it to the doc as we scope out the work on how to make these changes more palatable when using sync.
For background, the reason these are difficult are that many mobile developers do not control when people update their apps, so if you change a field from string to int you will have some % of your mobile clients think a field is a string and some % think it is an integer and it can be incredible unclear what the server should send to older clients once the field has been updated.
Out of curiosity, what would you expect in this situation?
I can imagine, that different versions make extra effort when designing such systems.
My first, intuitive thought would be to have versioned schemas. As the application itself comes with a specifc version (e.g. 1.2.4) this could be passed when initializing the realm. This would of course require to also define that application version, when the schema is deployed.
Realm could then sync with the corresponding schema version to send the data in the correct format. Once all users are on the current version (e.g. 1.2.5), all users receive the current schema, and the version 1.2.4 of the schema could be deleted.
So lets assume in version 1.2.5 the datatype changes from int to an optional string:
Schema version 1.2.5 is deployed
App version 1.2.5 is deployed to the store (but not yet distributed)
Users still receive version 1.2.4 of the schema (still an integer)
As people update the application from the store, some people will get the data as strings now
At some point, everybody is on version 1.2.5 (maybe also by forcing them), and schema version 1.2.4 can be discarded
There must however be a strategy defined in code on how the data should be parsed to the correct type and what should happen if it cannot be parse (e.g. in the database is already saved a string, and it couldn’t parse it to integer). The most suffisticated version would be, that breaking changes require a parser, that basically is a funcion taking the old type and returning the new type. This could look like this:
/*
* Version 1.2.5 is making an integer to an optional string and assumes new clients send it as string already.
* This function is required and attached to version 1.2.5 to send data correctly to mobile phones with version 1.2.4.
* If the type in the database is still in the type of 1.2.4, just return it as it is.
* Otherwise this function is called. There should probably also be a smilar function with
* reversed parameter/return-value, for old devices still uploading an integer and parsing it to an optional string. This function becomes useless, once all users migrated.
*/
function educatonLevelRead(string? typeInDatabase): int{
if(typeInDatabase == null)
return 0;
switch(typeInDatabase){
case "elementary school":
return 1;
case "bachelors":
return 2;
default:
return 0;
}
}
I would whatsoever recommend/encourage developers to force updating the application once a new version is available in the store, so there is a maximum of two parallel schema versions available (one being the newest version, and one being the version that users who didn’t upgrade yet, still have).
This is just a rather spontanous idea I got. I assume you already considered this, giving my input whatsoever.
Appreciate your thoughts. It is a tough issue to solve and we are just trying to take in some opinions to ensure we ultimately build things the way people want to use them.
It sounds like you have identified an issue with your current deployment process. You can add versioning to your Realm schema and application. This way, you can ensure that the application will only work with a specific version of the Realm schema. When you make changes to your Realm schema, you can update the version number and include a migration strategy to handle the update. If it doesn’t work, look for alternatives. Some reliable experts, such as those from https://smartengines.com/ may help. The best approach depends on your specific needs and circumstances. However, implementing versioning, feature flags, and releasing updates together can help ensure that users do not experience issues.
