Concurrent read and write

Hey,
I have interesting problem that I believe can be solved with mongo atomic updates.

Let’s say I need to create pairs, given [a,b,c] and [1,2,3] my code can get any item from both array and create a pair to any item on the other array. This code run concurrently.

I would like use Mongo to ensure each pair is unique - e.g a valid results is a-1,b-2,c-3 but not a-1,b-1,c-1.

The main challenge here is reading which items already have matches and find the first that doesn’t and I am not sure if I can do it in an atomic operation. If I can, this will solve the entire problem

See MongoDB Transactions

I’m familiar with transactions and I actually looked at the docs before posting, reading data inside transaction does not guarantee anything so I am not sure how it is helpful unless I am missing something

Is this a simulation of a physical system?

Yes, right now I am thinking to solve it with another collection - so when getting a line, say a and a potential pairs - [1,2,3], I can do the following:

  • Read mongo to find the first that doesn’t have a match, let’s say 2
  • Write 2 to the lock collection
  • Write a-2 to the pairs collection.

If another thread tries to match 2 to, say, c in the same time one of them will fail (the lock collection has unique key index)

Where do you look for the already existing match? In the pairs collection?

Yes. Given read is not atomic, even inside transaction, I had to implement the lock

Why not perform an update with a filter on the condition and check for updated records to see if it actually completed and then deal with the case where it did not?

I’m not sure how. Basically I need to insert the first pair that does not exist and it’s a bit hard query to write

Can you give examples of documents? You could tag a document you’re working on to avoid another process updating it, that obviously creates a bottleneck in the system though.

That eventually what I did just with another collection.

The problem is, the pair collection is just a collection of connections, it has documents like
{
origin: ‘a’
target: ‘b’
}

Now if 2 process gets a. [1,2,3] and b, [1,2,3] the collection is empty - so I had to use the lock collection to add document like
key: 1
So the other process will fail