Object Lock; prevent deleting while in use

I am going to throw this out as a general question: is there a best practice to handle any kind of object (record) locking - optimistic, pessimistic or otherwise?

Here’s the scenario

Suppose theres a multi user, multi location app for selling gears and pulleys. Users can enter orders for those items - the order can be abandoned (user clicks Cancel) or saved (user clicks OK)

Imagine a UI where the user clicks ‘New Order’ and can then enter some line items on the order - those exist in memory only while the order is being created and then persisted when the order is saved.

A customer comes in and orders a gear and a pulley. The user adds a gear and a pulley to the order.

While they are entering the customers order, another user at another workstation decides the company no longer carry gears and deletes the gear object.

But wait - that object is already on the order (not saved yet, just in memory)

Ideally, what should happen is that the user whose attempting to delete the gear should be told “That gear is in use and cannot be deleted at this time”

But what happens is; they delete it, which invalidates the object - the user’s app where the order is being entered likely crashes or perhaps the item just disappears off the screen (depending on how it’s coded)

One solution is to keep a “in use” property flag on the object, and when it’s added to an order in process, set the flag to true. It’s a simple solution; before allowing an object to be deleted only of that property is set to false. If it’s “in use” stop the user from deleting it.

class Gear: Object {
   @Persisted var in_use = false
}

However. What if the “in use” flag is set and the app crashes, or the power goes out, or the user force-quts the app or any number of things. They flag would remain set, even if the item was not in use.

Is there a strategy here?

How can objects be flagged as in use so they are not inadvertently deleted while they are being used?

SQL offers record locking as a generally built in option. Other NoSQL databases offer user persistence so in the above example, an app is disconnected from the server, a server function can run - in this case upon user/app disconnect it could reset any in-use properties to false.

Realm/Mongo doesn’t seem to offer anything like that so how are other handling it?

@Jay They do actually… I’ve written a lot about this stuff already in my book I’ve been writing, but here’s some free excerpts from it:

One way to handle object locking in Realm is by using transactions. Transactions provide atomicity and isolation, which can help ensure data integrity in multi-user scenarios. Here’s an example of how you could implement optimistic locking in Realm:

  1. When a user starts editing an order, create a new transaction and retrieve the objects that the user will be modifying:
let realm = await Realm.open(config);
let orderId = ...; // ID of the order being edited
let order = realm.objectForPrimaryKey("Order", orderId);
let items = order.items;
  1. Before the user saves the changes, check whether any of the objects have been modified by another user:
let modifiedItems = items.filtered("modificationDate > $0", user.lastSyncDate);
if (modifiedItems.length > 0) {
  // Notify the user that the data has been modified and they need to refresh
  return;
}
  1. If no objects have been modified, update the objects and commit the transaction:
realm.write(() => {
  // Update the order and its items
  order.customerName = ...;
  items[0].quantity = ...;
  
  // Set the modification date on the objects
  let now = new Date();
  order.modificationDate = now;
  items.forEach((item) => item.modificationDate = now);
});

By setting the modification date on the objects, you can detect whether they have been modified by another user since the current user started editing them. If any objects have been modified, you can notify the user that they need to refresh the data before saving their changes.

This is just one example of how you could handle object locking in Realm. The approach you choose will depend on the specific needs of your application and the types of modifications you need to support.

Another approach to object locking in Realm is to use transaction versions. When a transaction is started, Realm increments the transaction version, and any changes made in that transaction are tagged with the current version. You can then use this version to track changes and prevent conflicts.

For example, when a user begins editing an object, you could store the current transaction version. When the user saves their changes, you can check the transaction version to see if any changes have been made since the user began editing the object. If so, you can prompt the user to review the changes and decide whether to merge or discard their changes.

Here’s an example of how you could implement this approach in code:

// Get a reference to the Realm instance
let realm = try! Realm()

// Get the object to edit
let gear = realm.object(ofType: Gear.self, forPrimaryKey: gearId)

// Store the current transaction version
let version = realm.configuration.currentTransactionVersion

// Begin a write transaction
realm.beginWrite()

// Make changes to the object
gear.name = "New Gear Name"

// Check if the transaction version has changed
if version != realm.configuration.currentTransactionVersion {
   // Transaction version has changed - handle conflict
   // Display a message to the user and prompt them to review changes
   // The user can then choose to merge or discard their changes
} else {
   // Transaction version has not changed - commit changes
   try! realm.commitWrite()
}

This approach allows you to track changes to objects and prevent conflicts when multiple users are editing the same objects simultaneously. However, it does require additional code to handle conflicts and merge changes, so it may not be the best approach for all applications.

Ultimately, the best approach for handling object locking in Realm will depend on the specific needs of your application and the types of modifications you need to support. It’s important to carefully consider the potential risks and benefits of each approach before choosing the one that’s right for your application.

In addition to the strategies mentioned above, there are a few other things you can do to mitigate the risk of data conflicts and object deletion in Realm:

  1. Implement proper error handling: Whenever an object is deleted or modified, make sure to handle any errors that may occur. For example, if a user tries to delete an object that is currently in use by another user, make sure to catch the error and provide an appropriate message to the user.

  2. Use transactions: Transactions provide a way to group multiple write operations together into a single atomic unit. This can help to ensure that modifications are applied consistently and reliably, even in the face of conflicts or errors.

  3. Use versioning: By including a version number or timestamp with each object, you can ensure that conflicting modifications are detected and resolved appropriately. For example, if two users attempt to modify the same object simultaneously, you can compare the version numbers or timestamps to determine which modification should take precedence.

Overall, the key to handling object locking in Realm is to design a system that is both robust and flexible. By carefully considering the specific needs of your application and the potential risks and benefits of different approaches, you can create a solution that meets the needs of your users while minimizing the risk of data loss or corruption.

Here are a few more coding examples for handling object locking in Realm:

  1. Using a dedicated locking table:

One approach to handling object locking in Realm is to create a dedicated locking table that keeps track of which objects are currently being modified. This table could include information such as the object ID, the user ID of the user who is currently modifying the object, and the time when the modification started.

Here’s an example of what the schema for such a table might look like:

class ObjectLock: Object {
   @Persisted(primaryKey: true) var objectId: String
   @Persisted var lockedByUser: String
   @Persisted var lockStartTime: Date
}

To lock an object, you would create a new ObjectLock object with the appropriate values and add it to the Realm. To check if an object is currently locked, you would query the ObjectLock table for any locks that are currently in place for that object.

  1. Using transactions:

Another approach to handling object locking in Realm is to use transactions to ensure that modifications are made atomically. Transactions provide a way to group multiple modifications together into a single, atomic operation, which can help to ensure that no other users are modifying the same objects at the same time.

Here’s an example of how you might use transactions to modify an object in Realm:

let realm = try! Realm()
let objectToModify = realm.object(ofType: MyObjectType.self, forPrimaryKey: objectId)

// Perform the modification inside a write transaction
try! realm.write {
    objectToModify.propertyToModify = newValue
}

By wrapping the modification inside a write transaction, you can ensure that no other users are modifying the same object at the same time.

  1. Using optimistic locking:

Finally, another approach to handling object locking in Realm is to use optimistic locking. With optimistic locking, you assume that no other users are modifying the same objects at the same time, but you include a version number or timestamp with each object. When a user saves changes to an object, you check the version number or timestamp to make sure that no other changes have been made in the meantime.

Here’s an example of how you might use optimistic locking in Realm:

class MyObjectType: Object {
   @Persisted(primaryKey: true) var id: String
   @Persisted var propertyToModify: String
   @Persisted var version: Int // optimistic locking version number
}

let realm = try! Realm()
let objectToModify = realm.object(ofType: MyObjectType.self, forPrimaryKey: objectId)

// Modify the object's properties
objectToModify.propertyToModify = newValue

// Increment the object's version number
try! realm.write {
   objectToModify.version += 1
}

// Save the changes to the object, checking the version number in the process
try! realm.write {
   realm.add(objectToModify, update: .modified)
}

By checking the object’s version number before saving changes, you can detect if any other changes have been made in the meantime and handle the conflict appropriately.

And this is another example for transactional logic for this and some more explanation about it, and how to handle it.

Another approach to handling object locking in Realm is to use transaction observers. Transaction observers are a powerful feature in Realm that allow you to listen for changes to specific objects or collections and take action in response to those changes.

Here’s an example of how you could use transaction observers to handle object locking in Realm:

let gear = realm.object(ofType: Gear.self, forPrimaryKey: gearId)

// Add a transaction observer to the gear object
let observerToken = gear?.observe { change in
    switch change {
    case .change(let properties):
        // The gear object has changed, handle the change
        // This could involve updating the UI or taking some other action
    case .deleted:
        // The gear object has been deleted, handle the deletion
        // This could involve removing the object from the UI or taking some other action
    case .error(let error):
        // There was an error observing the transaction, handle the error
        // This could involve displaying an error message or taking some other action
    }
}

// Make changes to the gear object
try! realm.write {
    gear?.inUse = true
}

// Save the changes and release the observer
try! realm.commitWrite(withoutNotifying: [observerToken])

In this example, we add a transaction observer to the Gear object and listen for changes to the object. When the observer is notified of a change, we can handle the change appropriately, such as updating the UI or taking some other action.

Before making changes to the Gear object, we start a write transaction and obtain a reference to the transaction observer token. We then make our changes to the object and save the changes using the commitWrite(withoutNotifying:) method. By passing in the observer token to the withoutNotifying parameter, we ensure that the observer is not notified of the changes we just made.

This approach ensures that we are notified of any changes made to the Gear object, regardless of whether they were made by the current user or another user. By handling these changes appropriately, we can ensure that the data remains consistent and that users are not able to inadvertently modify data that is already in use.

- DevOps Databases and Mobile Apps - A guide for everyone. -

The actual repo itself is mostly empty for public viewing, but there will be free chapters put into the repo for it later on, but this stuff above should answer your questions quite cohesively. Let me know if you have any questions.

@Brock

Thanks for the awesome reply! Super great information.

I’ve read it over several times and I can’t see where it addresses the core issue though - perhaps my question was too vague. For example, suppose this happens

  1. When a user starts editing an order, create a new transaction and retrieve the objects that the user will be modifying:

and before the user saves the order

  1. Before the user saves the changes, check whether any of the objects have been modified by another user:

Say the user takes 5 minutes to edit the order and during that 5 minute time period, another user simply deletes the objects the editing user has read in? There’s nothing preventing those read in objects from being deleted while “in use”, hence the title of the post: Object Lock; prevent deleting while in use

That being said, this part is more toward the question

  1. Using a dedicated locking table:

And is similar to the solution I mentioned in my question, but it has a flaw:

However. What if the “in use” flag is set and the app crashes, or the power goes out, or the user force-quts the app or any number of things. They flag would remain set, even if the item was not in use.

In SQL, records read in can be locked - that prevents them from being altered or deleted but they are still available for use as ‘read-only’ records. This is obviously a very clean and simple solution.

NoSQL databases are a bit more challenging and generally don’t offer SQL record/document locking like SQL. However, many offer a presence system where the server “know” about client connection status and when that status changes, can perform an action.

So if a client connects and an order is created and 5 OrderItems are added, the linked Gear objects could be set to “in-use” (via a locking table as mentioned in your post). If there’s a d/c, the server knows that and can simply clear that table, which resets those items.

I hope that clarify’s the question a bit further.

@Brock

I have some followup questions if you don’t mind.

In your post, the first two presented options for ‘locking’ are

One way to handle object locking in Realm is by using transactions

and

Another approach to object locking in Realm is to use transaction versions

Both of those options seem to lean toward just notifying a user the object they are editing has been changed - it doesn’t prevent a change (e.g. lock) - just lets them know the object has been changed.

Am I understanding these are more of a reactive approach than proactively prevent deleting in the first place?

Wouldn’t simply attaching an observer (as mentioned) to each object do the same thing? If another user modifies the object, that will fire an event and the code/user would know the object was changed?

The other question is about the word ‘transaction’. Realm does not have read transactions at all and the only transactions are within a write, and write transactions are first come-first-served. So if I open a write transaction to modify an object

try! realm.write {
    //take some time and do some stuff with the gear
     gear.name = "top gear"
}

and before that transaction completes another user does this

try! realm.write {
    realm.delete(gear)
}

…And the gear object is now deleted; believe it or not, the first transaction will complete without an error but there will be no gear object. This is what I am trying to prevent.

oh… on this. I am not sure commitWrite will work here as the first section of code will commit when the closure ends.

// Make changes to the gear object
try! realm.write {
    gear?.inUse = true
    ---> write is committed here <---
}

// Save the changes and release the observer
try! realm.commitWrite(withoutNotifying: [observerToken])

I think you may want this

try! realm.write(withoutNotifying: [token]) {
    // ... write to realm
}

Any tips or thoughts would be appreciated as this is a big issue for our project and so far, we have been unable to find a complete solution to prevent an object from being deleted while in use.

Yes, you are correct that the first two options presented for locking in Realm are reactive and not proactive. They notify the user that the object has been changed, but they do not prevent other users from modifying the object while it is being edited by another user.

Attaching an observer to each object can also achieve the same result of notifying the user when the object has been changed. However, it would require more code to implement compared to using transactions or transaction versions.

Regarding transactions in Realm, you are correct that Realm does not have read transactions and that write transactions are first come, first served. In the scenario you described, where one user is modifying an object while another user deletes it, the first transaction will complete without an error, but the object will no longer exist in the Realm.

To prevent this, you can implement a lock mechanism using transactions or transaction versions, where the object is locked when a user starts editing it and unlocked when the user finishes editing it. This will prevent other users from modifying or deleting the object while it is being edited.

As for the commitWrite method, you are correct that it will commit the changes to the Realm when the closure ends. If you want to write to the Realm without notifying the observer, you can use the write method with the withoutNotifying parameter, as you suggested. This will write to the Realm without notifying the observer, which can be useful in scenarios where you want to prevent notifications from being sent while a user is editing an object.

As far as Transactions go, you would construct the logic for the transactions with a blocking measure to prevent it occurring logically. The main issue with Realm or ANY NoSQL solution, is that none of them possess locking like you’re looking for organically to them. So you have to create the logic.

Everything that is done from this point forward is largely going to have to be custom solutions, above are examples of solutions you can develop and execute.

Thanks @Brock super good info and really helpful.

We’ve been doing some testing where we have a table of objects that are 'in use" and before an object is deleted, we check for it in the table using some versioning per your posts above.

However, we’re run into additional issues because while Realm Writes are Atomic, the Reads are not (is there such a thing?). In other words, the delays in reading and writing can get things out of sync quickly and if there are 50 users all working on data… it just becomes a mess.

One thing I did come across is findAndModify which is atomic; a document is found and modified concurrently. This would be a help.

However, it doesn’t appear (to my knowledge) that’s a Realm ‘thing’. It feels like an Upsert would have the same functionality but it’s not clear from the documentation if it’s a guaranteed atomic call.

You’re welcome! I’m glad the information was helpful.

Regarding your issue with syncing reads and writes in Realm, you’re correct that Realm does not currently support atomic reads. However, there are some strategies you can use to mitigate this issue:

  1. Minimize the duration of reads: You can try to minimize the duration of reads by only fetching the data that you need and using indexes to speed up queries. Additionally, you can consider caching frequently accessed data in memory to avoid unnecessary reads.

  2. Use optimistic locking: Optimistic locking is a technique where you add a version number or timestamp to each document and use this to detect conflicts. When you write to a document, you include the version or timestamp, and if the document has been updated since you last read it, the write will fail. You can then retry the write with the updated data. This approach can help reduce conflicts and ensure that writes are consistent.

  3. Use transactions: Transactions allow you to perform a series of writes atomically, ensuring that all writes are either committed or rolled back. You can use transactions to update multiple documents at once, which can help ensure that reads and writes are consistent.

Regarding the findAndModify function, it is not currently supported in Realm. However, as you mentioned, the upsert function can be used to achieve similar functionality. According to the documentation, upsert operations are implied atomic and will either update an existing document or insert a new document if it doesn’t already exist. If multiple threads are trying to update the same document, the last write will win. However, if you’re using optimistic locking, conflicts can be detected and resolved appropriately.

However, this is assuming the way it’s written in Kotlin’s SDK documentation isn’t written in error as it implies to me that upsert is atomic.