Mongodb duplicate Unique index value

[Divakar_V]

I created unique index on mongodb key on text field but I’m able to insert the duplicate keys so how is it possible in mongodb

[Michael_Nwuju (Michael)]

Hi @Divakar_V, can you please share your schema design so we can help you better?

Also, if you are using MongoDB Atlas, you can verify if your unique index is working by taking the following steps -

• Browse Collections
• Choose the collection where you added the unique index
• Look out for this menu under your collection name, usually in this format - database.collection

• Click on Indexes and you should see if your unique index is there.

[Divakar_V]

you can see here last two documents have same value for mappedEnvironment and date

[steevej (Steeve Juneau)]

It might sound stupid as a question but I see no other explication.

Are you sure you are connected to the same cluster and using the same database and viewing the same collection?

[Divakar_V]

Im using mongodb installed in linux vm and using same database and viewing same collection

[steevej (Steeve Juneau)]

Having this failure in such a trivial use case is really surprising.

Can you redo the screen shoot where we see the duplicate documents but make sure we see the same first line as the screenshot of the indexes? The first screenshot cuts out the long path of the database and collection name. And since you have complicated names.

My linux Compass is a bit old and I am not sure from your screenshot if the database is API and the collection is Insights.Dashboard.Traffic or the database is API.Insights and the collection is Dashboard.Traffic, …

[Michael_Nwuju (Michael)]

I also wanted to ask if the index was created after the duplicates were created or before

[Divakar_V]

No first the index is created then only we added the data

[aneroid (Anirudh Dutt)]

@Divakar_V

you can see here last two documents have same value for mappedEnvironment and date

Okay, instead of screenshots/images of code, hover over the two ‘duplicate’ documents and click ‘copy’ using the icon on the right side. And then paste them as text using three backticks like this (or use the “pre-formatted text” icon in formatting bar).

\```

As for the correctness the examples:

@steevej

Are you sure you are connected to the same cluster and using the same database and viewing the same collection?

If documents, database, collection, index, etc. all being the same as in the screenshot, then I suspect it’s a unicode issue and the characters only appear similar in a screenshot but are actually different. Either the string values or the field names are different in the documents or in the index definition.

For the index definition, use the shell command db.<collection name>.getIndexes() and paste that output, also as text, not as an image.

[chris (Chris Dellaway)]

Hi @Divakar_V

I suspect an invalid document may have been inserted into the collection, the quickest way to confirm this is to do a full validate on the collection.

e.g. db.runCommand( { validate: "myCollection", full: true } )

Stennie’s post is relevant to this issue, it is worth a read as well as the linked documentation:

Mongodb allows multiple keys with the same name

The underlying BSON storage format in MongoDB uses an ordered object representation rather than a dictionary, so technically duplicate fields are allowed (but not commonly used). This is noted in the documentation on field names: BSON documents may have more than one field with the same name. Most MongoDB interfaces, however, represent MongoDB with a structure (e.g. a hash table) that does not support duplicate field names. If you need to manipulate documents that have more than one field wit…

The first duplicate field is getting indexed but the last one is displayed in queries, making it look like the unique index is validated. Below is an example of how this can occur, the output of a validate with this issue and the logs that identify the _id of the impacted documents.

---

# create the unique index
mongosh --quiet --eval 'db.getSiblingDB("test")["dup"].createIndex({e:1,d:1},{unique:true})'

# create the test data
mongoimport -d test -c dup << EOF
{"d":300,"e":"P"}
{"d":300,"e":"D"}
{"d":300,"e":"X","e":"P"}
{"d":300,"e":"U"}
{"d":300,"e":"Q","e":"X","e":"P"}
EOF 

# do a find that looks like there are duplicates
mongosh --eval 'db.getSiblingDB("test")["dup"].find({d:300})'
[
  { _id: ObjectId('6880657da3ad720dac095156'), d: 300, e: 'P' },
  { _id: ObjectId('6880657da3ad720dac095157'), d: 300, e: 'P' },
  { _id: ObjectId('6880657da3ad720dac095158'), d: 300, e: 'D' },
  { _id: ObjectId('6880657da3ad720dac095159'), d: 300, e: 'U' },
  { _id: ObjectId('6880657da3ad720dac09515a'), d: 300, e: 'P' }
]

# another sample query
mongosh --eval 'db.getSiblingDB("test")["dup"].find({e:"X"})'
[ { _id: ObjectId('6880657da3ad720dac09515a'), d: 300, e: 'P' } ]

# perform full validation of collection
mongosh test --eval 'db.getSiblingDB("test").runCommand({validate:"dup",full:true})'
{
  ns: 'test.dup',
  uuid: UUID('a7305a36-4670-4749-84b8-8a98fa094076'),
  nInvalidDocuments: 0,
  nNonCompliantDocuments: 2,
  nrecords: 5,
  nIndexes: 2,
  keysPerIndex: { _id_: 5, e_1_d_1: 5 },
  indexDetails: { _id_: { valid: true }, e_1_d_1: { valid: true } },
  valid: true,
  repaired: false,
  warnings: [
    'Detected one or more documents in this collection not conformant to BSON specifications. For more info, see logs with log id 6825900'
  ],
  errors: [],
  extraIndexEntries: [],
  missingIndexEntries: [],
  corruptRecords: [],
  ok: 1
}

# process the log for documents with duplicate fields
jq '.|select(.id==6825900)' mongod.log
{
  "t": {
    "$date": "2025-07-23T04:33:45.833+00:00"
  },
  "s": "W",
  "c": "STORAGE",
  "id": 6825900,
  "ctx": "conn86",
  "msg": "Document is not conformant to BSON specifications",
  "attr": {
    "recordId": "17",
    "reason": {
      "code": 378,
      "codeName": "NonConformantBSON",
      "errmsg": "A BSON document contains a duplicate field name : e in element with field name 'e' in object with _id: ObjectId('6880657da3ad720dac095157')"
    }
  }
}
{
  "t": {
    "$date": "2025-07-23T04:33:45.833+00:00"
  },
  "s": "W",
  "c": "STORAGE",
  "id": 6825900,
  "ctx": "conn86",
  "msg": "Document is not conformant to BSON specifications",
  "attr": {
    "recordId": "20",
    "reason": {
      "code": 378,
      "codeName": "NonConformantBSON",
      "errmsg": "A BSON document contains a duplicate field name : e in element with field name 'e' in object with _id: ObjectId('6880657da3ad720dac09515a')"
    }
  }
}

[steevej (Steeve Juneau)]

WOW

Almost like a “Deer in headlights” .

[chris (Chris Dellaway)]

When i wish there was more than the simple like button @steevej

This might not be the cause, its the only thing that makes sense to me.

[aneroid (Anirudh Dutt)]

@chris

The first duplicate field is getting indexed but the last one is displayed in queries, making it look like the unique index is validated.

ten

[steevej (Steeve Juneau)]

It makes a lot of sense. But in mongosh and js, I never encounter this issue because the JSON parser only keeps the last value:

node> d = {"d":300,"e":"Q","e":"X","e":"P"}
{ d: 300, e: 'P' }

Which is a node feature I use a lot to update a copy of an object:

original = { foo : 1 , bar : 2 }
updated_copy = { ... original , bar : 3 }

[chris (Chris Dellaway)]

Correct, as Stennies post mentions most interfaces(drivers etc) don’t allow this, and though some tools provide builders that can create documents with duplicate field they are not supported.

In the wild I have only seen this with mongoimport and using a custom codec with a driver.

[aneroid (Anirudh Dutt)]

@chris

In the wild I have only seen this with mongoimport and using a custom codec with a driver.

For me, I’ve only seen this with someone who manually entered data and had unicode enabled on their keyboard or was purposefully asking a bad-faith question (which I experienced on StackOverflow) like 'e' != 'ḛ' and the bottom ligature wasn’t visible in the screenshot.

TBH, it doesn’t make sense that the BSON spec allows multiple fields with the same name. In a URL, like example.com/?p=1&p=2&p=3 that makes p an array-equivalent with the value p = [1, 2, 3].

[DEBASISH_NAYAK]

Hello Divakar,

Thank you for sharing your observation.

Basically a unique index enforces that indexed field should not have duplicate value.

I acknowledged your concern, however help to understand following points for further validation:

• Share code snippet or output to validate the behaviours.
• Ensure to share database version.
• Current database deployment architecture.( standalone/ replica set / sharded cluster)

These information will help to understand the true behaviour that you want us to validate.

Best Regards,
Debasish Nayak
MongoDB (TSE)