How to update first empty object in array?

Yes, I want to pre-allocate specific number of elements in array, and keep number of elements static from there on. I was thinking of having element like "maxNumberOfObjects": 4, but counting array elements in many places felt also pretty convenient, so defaulted to that.

Actually, document I would be updating, I know, so I have _id of it at hand. So I can target specific document, and update empty object in arrayOfObjects. Like you said more common way would be to use $push, this is kind of what I want to do, though instead of pushing new object in, updating first empty object, and ideally failing if there are no empty objects. If “nice” failing isn’t possible, I could first check that there is empty object before updating.

More traditional database/programming way I would most likely go finding [index] in first query, then make another query which does update. However that update method seems to have arrayFilters which I have feeling could get this done in one query. Didn’t find anyone trying to filter empty object, so trying to figure out how to write such query.

So I believe it would be something like:

db.collection.update(
    { "_id": "abc123" },
    { $set: { "objectName": "example2", "objectData": "newdata" } },
    {
    arrayFilters: $[ how_to_filter_empty_object ]
    }
)

But I’m a bit lost what should be that actual filter there, and a bit unsure how correct is that $set line.
Do you catch my idea here, and have some pointers how to make that update as working one @Asya_Kamsky?
Also if you see some big drawbacks (other than being uncommon way to have empty objects), I’m eager to hear. My idea with this document structure has been that I need less queries and document structure contains meaningful information itself too.

Would it help to know that you can compare something to {} to match an empty object?

I can show example of how to do this with all three methods, just wanted to check if you want to see the answer or prefer to just be pointed towards the answer :slight_smile:

I tried this using the aggregation for the update (MongoDB version 4.2). The query looks for the first empty object in the array and updates it with the new object.

NEW_OBJ = { "objectName": "example2", "objectData": "newdata" }

db.collection.updateOne(
    { 
        _id: 123,  
        arrayOfObjects: { }
    },
    [
        { 
            $addFields: {
                // ix is index of the first element which is an empty object
                ix: { $indexOfArray: [ "$arrayOfObjects", { } ] },
                ixs: { $range: [ 0, { $size: "$arrayOfObjects" } ] } 
            } 
        },
        { 
            $project: { 
                arrayOfObjects: { 
                    $map: {
                        input: "$ixs",
                        in: { 
                            $cond: [ 
                                { $eq: [ "$ix", "$$this" ] },
                                 NEW_OBJ,
                                { $arrayElemAt: [ "$arrayOfObjects", "$$this" ] }
                             ] 
                        }
                   }
               }
           }
       }
    ]      
)

This works fine from the Mongo Shell.

1 Like

Thank you @Prasad_Saya, that looks interesting approach. I’ll give it a go. It looks more complex than I had expected it to be, as I was fiddling around something like this:

db.test.update( 
   { "_id": "abc123" }, 
   { $set: { "objectName": "example1", "objectData": "newdata"}},
   { arrayFilters: $arrayOfObjects[ {} ]});

But I was running into error that $arrayOfObjects wasn’t defined, and was just coming to ask a bit of example from @Asya_Kamsky how that filter works, and am I even in right path.

I think you were looking for something more like

db.test.update( 
   { "_id": "abc123" }, 
   { $set: { "arrayOfObjects.$[i]": {"objectName": "example1", "objectData": "newdata"}}},
   { arrayFilters: [ { i: {$eq: {}} } ] }
);

ps edited to fix syntax

1 Like

You can also do it using positional operator:

db.coll.update(
   {_id:x, arrayOfObjects:{}},
   {$set: {“arrayOfObjects.$”:{new:”whatever”}}})

Thank you very much @Asya_Kamsky, both are nice and clean solutions. Tried both, and I think there is small typo on that arrayFilter example, missing { there maybe? What worked for me is like this (for future references when someone tries same solutions)

db.test.update( 
   { "_id": "abc123" }, 
   { $set: { "arrayOfObjects.$[i]": {"objectName": "example1", "objectData": "newdata"}}},
   { arrayFilters: [ { i: { $eq: {} }} ] });

Both seem to work similarly when there are empty objects which can be updated, giving following results:

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

However when document doesn’t have empty objects, first one still gives Matched: 1, but doesn’t of course modify anything. Second one gives zeroes to both Matched and Modified.

So I think I’ll go with first one, and see if I can get that result checked in code, and hop into different usecase if there ain’t empty objects anymore.

Thank you once more Asya, this allows me to continue onward in my project. :slight_smile:

1 Like

If you add the check for presence of empty object to the query in the first query then the result would be the same - it says I didn’t find a matching record.

I must have overlooked how these when I was testing them in mongo shell. Now that I tried implementing in nodejs, I noticed that arrayFilter way actually doesn’t work as intented. Instead of updating only first empty object, it updates all empty objects in that array in given document.

As first one seems to be updating only first object, I can work with that. However is there some modification for arrayFilter to match only one/first, or is it actually meant to match all elements in array always?

To overcome this issue of updating all the empty objects with the update operation, the aggregation solution is implemented. The aggregation first checks if there is an empty object in the array, then, gets the index of the first empty object, and, replaces that empty object with the supplied new object.

The query does look complex, but it is a solution for the requirement.

Instead of updating only first empty object, it updates all empty objects in that array in given document

Ah, I see what you’re saying - you don’t want to update all the matching entries, just the first one. Positional update (first syntax) is the one to use then - while aggregation in update works as well (as long as you’re on 4.2+) unfortunately there is no syntax to easily short-circuit from $map iteration over the array - with a small array it’s not a big deal, but for larger arrays it might be noticeably slower.

It can actually be done with $concatArrays rather than $map by concatenating using the index (location of the first {} element in the array rather than for iteration but like this:

[ {$set:{ arrayOfObjects:{$let: {
     vars: { ix: { $indexOfArray: [ "$arrayOfObjects", { } ] } },
     in: {$concatArrays: [
         {$slice:[ "$arrayOfObjects", 0, "$$ix"] },
         [ { newObjectHere } ],
         {$slice:[ "$arrayOfObjects", {$add:[1, "$$ix"]}, 4] }
     ]
}}}}]

This would go as the second argument to update. I didn’t run this so I’m sure there’s a paren missing somewhere or an off-by-one error… :slight_smile:

1 Like

I’ll give that a testing in few days, once I return to that part of project. If I don’t see performance difference between first working one and this arrayFilter one, I might continue using first one. It is much shorter and easier to read :smiley: I’m fairly sure that if I need to return that query at later date, or someone new tries to start understanding it, it will be appreciated. :slight_smile:

That’s however fine example of how powerfull things can be done directly in query, and I might need that at later time. Thank you very much @Asya_Kamsky for taking time with this. :slight_smile:

1 Like

This is the same update query I had posted earlier with some refinement:

db.collection.updateOne(
    { 
        _id: 123,  
        arrayOfObjects: { }
    },
    [
        { 
            $set: { 
                arrayOfObjects: { 
                    $map: {
                        input: { $range: [ 0, { $size: "$arrayOfObjects" } ] },
                        in: { 
                            $cond: [ 
                                { $eq: [ { $indexOfArray: [ "$arrayOfObjects", { } ] }, "$$this" ] },
                                { "objectName": "example2", "objectData": "newdata" }, // this is the new object to be inserted
                                { $arrayElemAt: [ "$arrayOfObjects", "$$this" ] }
                             ] 
                        }
                   }
               }
           }
       }
    ] 
)
1 Like

Thank you @Prasad_Saya, that query starts to look like understandable now. I’ll try to give it a go as well… Though at a glance it looks like having similar elements as Asya’s previous query:

That one seemed to work without need to do any $map, updating only first empty object it found, so is it necessary to have this more complex query Prasad? As I haven’t tested yet, I don’t know do they behave differently in edge situations… like when there is no arrayOfObjects{} in document for some reason, or if it doesn’t have any empty objects. Or performance wise big differences? In my current use case arrayOfObjects is usually around 10 elements, most likely at maximum rare cases 30-50 elemenets, so quite small. But concurrent operations by different users to different documents can be significant, if this product takes off properly. But I’ll test them out…

Note that in all examples, you should include the test for {arrayOfObjects:{}} so that it’s guaranteed that there is an empty object in the array.

1 Like

@Prasad_Saya you are correct that it’s the same approach with aggregation update, I was simply pointing out that $map is less efficient in the case where you want to replace a single element in an array.

Yes, iterating over the entire array (i.e., using $map) is less efficient just to update one array element (especially if the array has a lot of elements). Also, noted that the update is for one document only (as it is queried by its _id).

I’m not really sure of the business and technical logic behind “pre-allocating” empty array elements.

Thanks so much

Bob

I’m also curious about the justification here. @kerbe, can you expand on why pre-allocation is necessary?

I highly recommend using supplementary fields instead. Having a field called currentNumberOfElements (or simply count) helps here. Instead of finding an empty object to update in an array of potentially one or more empty objects, I recommend using query predicates to decide if an update needs to occur or not.

For example:
{ arrayOfObjects: [ { a: 1, b: 1 } ], count: 1 }

The update statement can look like this:
db.collection.updateOne({ count: { $lt: 4 } }, { $push: { arrayOfObjects: { a: 2, b: 2 } }, $inc: { count: 1 } })

Only documents which have an “open” array position would be updated, removing the need to “find” an empty element. Does that work for your use case?

1 Like

It isn’t absolutely necessary, as I said before. It has felt convenient, and felt that it could reduce some queries and also some code in the way I have been doing it this far.

Maybe it wasn’t really clear in my original question, or hasn’t been clearly stated in later time, but I don’t try to find document which has empty object in array. Update is going to be for one specific document, which _id is known. So those examples @Justin gave, are unnecessary, or wouldn’t be used like that.

My document design comes from thought, that document has certain number of badges, which these objects represent. In application empty objects are rendered empty, and when they get content, they automatically render that content. I was thinking that I would have something like count, and first print outline, and then fill that with how many badges there is, but it felt unnecessary to store extra information to different field, and operate that, when structure of document itself already provides needed information.

I have also been thinking grouping these badges, and having extra variables for how they are wanted to be grouped felt really difficult. But thought now is that I can create multi dimensional array, which holds objects as they are planned to be rendered, and document structure itself then is used for rendering decisions.

And why it goes like that? Because there could be different layouts in documents. Some could be simple having 10 badges, that are dumbed as is. Others could have like 20 badges, but they are wanted to be grouped in four groups, each holding five badges. Or there could be document which has grouping where first row has only three badges, second one four, third one five… or something other funky groupings.

So, to get those funky groupings work, I came up that empty objects can show what is desired layout. At least currently it feels easier to start thinking how to render those, than having some other structure storing layout information.

And this query I was trying to figure out, was query to add one new badge to specific one document. I was thinking of just manipulating arrayOfObjects in nodejs code, and then push whole array replacing previous, but it felt wrong, as it would potentially change layout. So wanted to have update to target first empty object in that document, so structure remains always unaltered, and there shouldn’t potentially be race conditions if there would be multiple update operations directed to same document ~same time. This way update succees, if there is empty object, or should fail if there isn’t empty one anymore when MongoDB starts doing update (or well, not fail, but not update anything, which I can then detect in nodejs code, and work around that accordingly).

Of course, I am open for ideas, and curious to hear am I creating something that will shoot me in the leg in long run. Does my explanation give more insight of my design/thought process @Justin & @Robert_Cochran? Any comments or doubts?