Mongodb duplicate Unique index value

Working with Data
11

Hi @Divakar_V

I suspect an invalid document may have been inserted into the collection, the quickest way to confirm this is to do a full validate on the collection.

e.g. db.runCommand( { validate: "myCollection", full: true } )

Stennie’s post is relevant to this issue, it is worth a read as well as the linked documentation:

The first duplicate field is getting indexed but the last one is displayed in queries, making it look like the unique index is validated. Below is an example of how this can occur, the output of a validate with this issue and the logs that identify the _id of the impacted documents.

# create the unique index
mongosh --quiet --eval 'db.getSiblingDB("test")["dup"].createIndex({e:1,d:1},{unique:true})'

# create the test data
mongoimport -d test -c dup << EOF
{"d":300,"e":"P"}
{"d":300,"e":"D"}
{"d":300,"e":"X","e":"P"}
{"d":300,"e":"U"}
{"d":300,"e":"Q","e":"X","e":"P"}
EOF 

# do a find that looks like there are duplicates
mongosh --eval 'db.getSiblingDB("test")["dup"].find({d:300})'
[
  { _id: ObjectId('6880657da3ad720dac095156'), d: 300, e: 'P' },
  { _id: ObjectId('6880657da3ad720dac095157'), d: 300, e: 'P' },
  { _id: ObjectId('6880657da3ad720dac095158'), d: 300, e: 'D' },
  { _id: ObjectId('6880657da3ad720dac095159'), d: 300, e: 'U' },
  { _id: ObjectId('6880657da3ad720dac09515a'), d: 300, e: 'P' }
]

# another sample query
mongosh --eval 'db.getSiblingDB("test")["dup"].find({e:"X"})'
[ { _id: ObjectId('6880657da3ad720dac09515a'), d: 300, e: 'P' } ]

# perform full validation of collection
mongosh test --eval 'db.getSiblingDB("test").runCommand({validate:"dup",full:true})'
{
  ns: 'test.dup',
  uuid: UUID('a7305a36-4670-4749-84b8-8a98fa094076'),
  nInvalidDocuments: 0,
  nNonCompliantDocuments: 2,
  nrecords: 5,
  nIndexes: 2,
  keysPerIndex: { _id_: 5, e_1_d_1: 5 },
  indexDetails: { _id_: { valid: true }, e_1_d_1: { valid: true } },
  valid: true,
  repaired: false,
  warnings: [
    'Detected one or more documents in this collection not conformant to BSON specifications. For more info, see logs with log id 6825900'
  ],
  errors: [],
  extraIndexEntries: [],
  missingIndexEntries: [],
  corruptRecords: [],
  ok: 1
}

# process the log for documents with duplicate fields
jq '.|select(.id==6825900)' mongod.log
{
  "t": {
    "$date": "2025-07-23T04:33:45.833+00:00"
  },
  "s": "W",
  "c": "STORAGE",
  "id": 6825900,
  "ctx": "conn86",
  "msg": "Document is not conformant to BSON specifications",
  "attr": {
    "recordId": "17",
    "reason": {
      "code": 378,
      "codeName": "NonConformantBSON",
      "errmsg": "A BSON document contains a duplicate field name : e in element with field name 'e' in object with _id: ObjectId('6880657da3ad720dac095157')"
    }
  }
}
{
  "t": {
    "$date": "2025-07-23T04:33:45.833+00:00"
  },
  "s": "W",
  "c": "STORAGE",
  "id": 6825900,
  "ctx": "conn86",
  "msg": "Document is not conformant to BSON specifications",
  "attr": {
    "recordId": "20",
    "reason": {
      "code": 378,
      "codeName": "NonConformantBSON",
      "errmsg": "A BSON document contains a duplicate field name : e in element with field name 'e' in object with _id: ObjectId('6880657da3ad720dac09515a')"
    }
  }
}
2 Likes