Error applying instruction to object in table - when doing anonymous login, for an Object with a List<EmbeddedObject> property (ProtocolErrorCode=212)

Madalin_Sava · May 18, 2023, 10:17am

My setup for flexible sync in an iOS app, using RealmSwift (schema seems fine, it was created automatically by using development mode):

final class User: Object {
    @Persisted var params: List<UserParam>
}

final class UserParam: EmbeddedObject {
    @Persisted var key: UserParamKey
    @Persisted var value: String
}

enum UserParamKey: Int, PersistableEnum {
    case firstName, lastName, businessName [etc]
}

User schema:

{
  "title": "User",
  "type": "object",
  "required": [
    "_id",
    "lastOnboardingStep"
  ],
  "properties": {
    "_id": {
      "bsonType": "objectId"
    },
    "calendarId": {
      "bsonType": "string"
    },
    "lastOnboardingStep": {
      "bsonType": "long"
    },
    "ownerId": {
      "bsonType": "string"
    },
    "params": {
      "bsonType": "array",
      "items": {
        "title": "UserParam",
        "type": "object",
        "required": [
          "key",
          "value"
        ],
        "properties": {
          "key": {
            "bsonType": "long"
          },
          "value": {
            "bsonType": "string"
          }
        }
      }
    }
  }
}

When I’m using a local Realm everything works fine. When I do anonymous login, I’m getting these errors:

{
   "logs":[
      {
         "_id":"6465f6ed0a9d994f8d714314",
         "co_id":"6465f6eb0a9d994f8d7142c9",
         "type":"SYNC_SESSION_END",
         "user_id":"6465f6ebeb4f073b5d5f2ca9",
         "domain_id":"645915213a82d1d7fbafefca",
         "app_id":"645915213a82d1d7fbafefc9",
         "group_id":"64591410ffb83f492c3916c7",
         "request_url":"/api/client/v2.0/app/billy-jgeoz/realm-sync",
         "request_method":"GET",
         "remote_ip_address":"5.14.130.67",
         "started":"2023-05-18T09:59:09.083Z",
         "completed":"2023-05-18T09:59:09.126Z",
         "function_call_location":"DE-FF",
         "function_call_provider_region":"aws-eu-central-1",
         "error":"ending session with error: failed to generate history batches: error generating object modifications: error generating post image: image generator encountered error applying instruction to state: error applying instruction to object in table 'User' with primary key '6464e5cc20c418867833b418' at field path 'params.0': ArrayInsert.prior_size was 0 but built-up array was only of length 14 (ProtocolErrorCode=212)",
         "error_code":"BadChangeset",
         "messages":[
            "Session was active for: 0s"
         ],
         "platform":"unknown",
         "platform_version":"Version 16.4 (Build 20E247)",
         "sdk_name":"Realm Swift",
         "sdk_version":"10.39.1",
         "sync_query":{
            "Settings":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "User":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Appointment":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Client":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Invoice":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "RecurrenceStatus":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Service":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")"
         },
         "sync_session_metrics":{
            "uploads":2,
            "downloads":4,
            "downloaded_changesets":3,
            "downloaded_changesets_size":920,
            "changesets":2
         }
      },
      {
         "_id":"6465f6ed0a9d994f8d714313",
         "co_id":"6465f6eb0a9d994f8d7142c9",
         "type":"SYNC_CLIENT_WRITE",
         "user_id":"6465f6ebeb4f073b5d5f2ca9",
         "domain_id":"645915213a82d1d7fbafefca",
         "app_id":"645915213a82d1d7fbafefc9",
         "group_id":"64591410ffb83f492c3916c7",
         "request_url":"/api/client/v2.0/app/billy-jgeoz/realm-sync",
         "request_method":"GET",
         "remote_ip_address":"5.14.130.67",
         "started":"2023-05-18T09:59:09.083Z",
         "completed":"2023-05-18T09:59:09.126Z",
         "function_call_location":"DE-FF",
         "function_call_provider_region":"aws-eu-central-1",
         "error":"failed to generate history batches: error generating object modifications: error generating post image: image generator encountered error applying instruction to state: error applying instruction to object in table 'User' with primary key '6464e5cc20c418867833b418' at field path 'params.0': ArrayInsert.prior_size was 0 but built-up array was only of length 14 (ProtocolErrorCode=212)",
         "error_code":"BadChangeset",
         "messages":[
            "Upload message contains 1 changesets (total size 1.4 kB) to be integrated"
         ],
         "platform":"unknown",
         "platform_version":"Version 16.4 (Build 20E247)",
         "sdk_name":"Realm Swift",
         "sdk_version":"10.39.1",
         "sync_query":{
            "Service":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Settings":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "User":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Appointment":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Client":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "Invoice":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")",
            "RecurrenceStatus":"(ownerId == \"6465f6ebeb4f073b5d5f2ca9\")"
         },
         "sync_write_summary":{
            "Service":{
               "inserted":[
                  "6464ce7583f82645c6f9bc2d"
               ]
            },
            "Settings":{
               "inserted":[
                  "6464e5cc20c418867833b419"
               ]
            },
            "User":{
               "inserted":[
                  "6464e5cc20c418867833b418"
               ]
            }
         }
      }
   ]
}

It should be possible to have a list of embedded objects, right?

Madalin_Sava · May 18, 2023, 12:35pm

I’m getting the same errors with development mode on or off.
Also, I know the issue is with the list property because of the count (14) of items in the logs.

Kiro_Morkos · May 18, 2023, 4:01pm

Hi @Madalin_Sava,

Your schema looks correct, but the error indicates that your sync history has been corrupted somehow. We’ll need some additional information to figure out the root cause of this issue. Can you provide the following if possible in a DM?

Realm debug logs from the user encountering the issue. You can follow the instructions here to set the log level from your swift app.
Optionally, a dump of the __realm_sync_645915213a82d1d7fbafefc9 database in your Atlas cluster. This contains a copy of your synced data, so only share this if this is a dev app without sensitive data.

Kiro_Morkos · May 22, 2023, 2:48pm

Thanks for sharing those!

Given the logs and history you shared, it appears that the issue is that the user is trying to update a User object that is not in their query view. i.e. the user is trying to modify the object with primary key ObjectId('6464e5cc20c418867833b418'), which has ownerId = 6465f1adab0d491cdd8353f0. However, the user’s active query on the User table is ownerId == "646788c12ed19c14cc6e4966". Normally this would trigger a compensating write, but there is a known issue being investigated now that triggers the error you’re seeing instead when lists are involved.

In the meantime, I don’t have enough information to know how your app is able to modify an object that it cannot see. Generally this is only possible if you’re not using unique primary keys, and try to create an object that already exists outside of the user’s active query view.

Madalin_Sava · May 25, 2023, 9:27am

I did some more debugging and got stuck, can you help me out?
What I’m doing:

using a local realm
let user =app.login(credentials: .anonymous)

let configuration = user.flexibleSyncConfiguration { subs in
subs.append(
            QuerySubscription<User>(name: User.className()) {
                $0.ownerId == user.id
            }
        )
// similar subscriptions for the other objects
}

Realm(configuration: configuration, downloadBeforeOpen: .never)
create a copy for every existing object, set ownerId to currentUser.id
call create(type, value: newObject, update: .modified) on the new realm

I checked several times the User._id and ownerId and I didn’t see any mismatch like you mentioned. ownerId is always the right one and it’s the same when I check the logs in AppServices. Maybe you looked at an older subscription query?
I also tried to look at the files I sent you but didn’t find the subscription queries, maybe you can show me how to check them and debug on my own.

How can I find the instruction that causes the illegal write? I never get any errors for a Realm.write call or in the logger. Also, is there a way to log the subscription query as it’s being executed? In the AppServices Logs, the subscriptions look ok, always with the logged in User.id.

Also, I checked and there is no other write besides the create call, and the object I pass has the right ownerId.

Sometimes I’m getting an exception in RealmDatabase:network.hpp.do_recycle_and_execute:2770 (the single throw; instruction) when debugging or at startup (every time, in this case) if I try to open a synced realm from the cached anonymous user, not sure if it’s related.

Kiro_Morkos · May 25, 2023, 2:26pm

create a copy for every existing object, set ownerId to currentUser.id

It sounds like this may be your issue. If an object already exists on the server with the same primary key (_id value), then this will be considered “updating an object outside your view”, which is not allowed. When copying the object, try setting a new, unique _id value. Let me know if that works!

Madalin_Sava · May 26, 2023, 11:49am

That’s right, but then I’m not sure how to solve my use case. Let’s say I have an Object subclass called “Preferences” and I need exactly one instance of it for every user.
If a user starts with a local realm (doesn’t login), I’ll have a Preferences instance with _id of “1” (for simplicity).
Then, the user logs in with Apple and I copy the object to the synced realm and change the _id to “2”.
If the user logs out, I go back to another local realm and I’ll have the instance with _id “3”.
User logs in with Google, _id will be “4”. What happens if the user decides to also log in with Apple (link identities)? Is there a way to manually merge the realms or what is the best practice?

Kiro_Morkos · May 26, 2023, 2:26pm

If I’m understanding correctly, when the user logs back in with an existing account, the Preferences object with ownerId set to the user id will be synced back down to the device (because of the subscription). If applicable, you could then merge the “local” preferences into the synced one. Does that answer your question?

You also mentioned linking identities, but I’m not sure how that’s applicable here. Linking an additional identity provider to an existing user does not create a new user object, so the ownerId should not change in that case.

Madalin_Sava · May 30, 2023, 10:14am

Makes sense, I think at this point my uncertainties are rather coming from the business logic requirements. The Realm usage seems clear for now.

Thank you for your help!