$setDifference fails to match objects with a different order of fields

Working with Data
1

Consider the below query

Query

db.collection.aggregate([
    {
        $project : {
            "result" : {$setDifference : [
                
                [{"names" : "a", "value": "something"}, {"names" : "x"}, {"names" : "y"}, {"names" : "z"}, {"names" : "h"}, {"names" : "r"}, {"names" : "s"}, {"names" : "t"}],
                [{"value": "something", "names" : "a"}, {"names" : "b"}, {"names" : "c"}, {"names" : "d"}, {"names" : "t"}]
                
                ]},
        }
    },
    {
        $limit:1
    }
    ])

Result

{
	"_id" : ObjectId("5e4a5521deb23e00017b6162"),
	"result" : [
		{
			"names" : "a",
			"value" : "something"
		},
		{
			"names" : "x"
		},
		{
			"names" : "y"
		},
		{
			"names" : "z"
		},
		{
			"names" : "h"
		},
		{
			"names" : "r"
		},
		{
			"names" : "s"
		}
	]
}

Now the expectation is to have only [x y z h r s] is the final result. But mongodb assumes {"names" : "a", "value": "something"} and {"value": "something", "names" : "a"} are two different values even though they are identical.

Is this a bug? Are there any workarounds for this? (My actual data has 5 fields so being in a different order is more likely imo)

2

No it is not a bug. Operations like $setDifference and $setIntersection are based on value equality. Two objects are equals if the have same fields and same values in the same order.

Normalize your data so that fields are in the same order. Since objects are usually updated using an application or an API, having the fields in the same order is usually not an issue. The issue arise when human update the values or when code is broken and objects are updated directly rather than a data access layer.

3

I see.

The issue arise when human update the values or when code is broken and objects are updated directly rather than a data access layer.

This makes sense to me. The documents I checked seems to have the same order.

Is there a way for me to check if theres any out of order?

4

The concept of aggregation pipeline is so powerful that there is

To illustrate the logic lets start with the collection:

{ _id: 0, a: { foo: 1, bar: 2 } }
{ _id: 3, a: { bar: 4, foo: 5 } }

The goal is to find out of any of the a: have the wrong order. The correct order is to have foo: first and then bar:. In the collection above _id:0 has the correct order and _id:3 has not.

The first stage will set the field _a: using $objectToArray on a:

_set = { "$set" : { _a : { $objectToArray : "$a" } } }

Then a simple $match stage will ensure that foo is first and bar is second. But since

_wrong_order = {
    "_a.0.k" : { "$ne" : "foo" } ,
    "_a.1.k" : { "$ne" : "bar" }
}
_match = { $match : _wrong_order }

And the final pipeline would be:

pipeline = [ _set , _match ]