E11000 duplicate key error collection with period in the text

Working with Data
1

I am trying to insert some documents into a collection and I get a duplicate key error, I have checked the documents that I am inserting multiple times, and removing the key seems to enable the insert to happen.

Here is two examples of the document I am trying to insert, with the unique key index on ldhName unless I’m mistaking somthing, the key would be unique.

{
  "ldhName": "exampleone.org.uk",
  "unicodeName": "storyinstone.org.uk",
  "tld": "org.uk",
  "lastUpdated": "2024-08-23T01:33:12"
}

{
  "ldhName": "exampletwo.co.uk",
  "unicodeName": "eurovision2023.co.uk",
  "tld": "co.uk",
  "lastUpdated": "2024-08-23T01:38:47"
}

The error it’s throwing up is:

Exception('Unable to insert document', DuplicateKeyError('E11000 duplicate key error collection: namechase.domains index: ldhName_text dup key: { _fts: "uk", _ftsx: 0.6666666666666666 }, full error: 
{\'index\': 0, \'code\': 11000, \'errmsg\': \'E11000 duplicate key error collection: namechase.domains index: ldhName_text dup key: { _fts: "uk", _ftsx: 0.6666666666666666 }\', \'keyPattern\': {\'_fts\': \'text\', \'_ftsx\': 1}, \'keyValue\': {\'_fts\': \'uk\', \'_ftsx\': 0.6666666666666666}}'))))

From what I can see, it creates the index on the last part of the , and not the entire string as I would expect, and removing the periods seems to solve the issue too, is this a missunderstanding of how the unique key works, or am I missing somthing?

2

Please share the exact index you have created.

3

The issue with text index is that it works at the word level. In the default locale the dot is a word separator. Since you defined the index as unique, then each word can occur only once.

I think you need 2 indexes.

1 - a text index that is not unique
2 - a normal index that is unique

This way you could add many ldhName that contains .uk because uk does not have to be unique anymore. However the second index will prevent the insertion of duplicated full domain.

4

Other than what’s in the error I’m not sure what you’re asking for, the index was created using MongoDB Compas, so I have included a screenshot of the index creation dialogue.

namechase-index-issue
namechase-index-issue617×997 31.1 KB

5

What you shared is fine.

Have you tried

6

@DanielRCoates, it has been more that a week that I provided replies to your issue.

Please provide some followup in order to keep this forum useful.

Thanks

7

Oh sorry, I didn’t realise you had replied, had issues with my emails recently, so I moved provider.

I’m not 100% certain what you mean by two indexes, I have one setup for the DocumentID using its default settings, and one for the ldhDomain as a unique text field.

The issue only seems to be with the . in the domain, if I remove all the . and just add examplecouk it works fine.

8

Having re read this, I will try using two indexes, one for ldhIndex that contains examplecouk, and then a non unique one on ldhName of example.co.uk

1 Like
9

This solved the issue, but doesn’t really fit what I was trying to do, I’ll have to have another think about how I’m storing the data.

Thank You @steevej

1 Like
10

After talking this over with another developer mate of mine, they suggested that I was using the wrong index type, and that I should take a look at a Single Field Index, this solved the issue.

11

I am not too sure what you mean by Single Field Index. Could you please provide more details on the exact index you created. I am pretty sure I am not the only who do not understand. Because, as far as I understand, the following are already 2 indexes on a single field:

and

Both are single field index because only one field is specified.

Yes because the . is a word separator for text index. That is why when the index is unique you can only have 1 document with the work uk. Which would not make sense for a field named ldhDomain. That is why you do not want the index to be unique.

By 2 indexes I mean

1 - a text index on ldhDomain that is not unique so that you can easily (without regex) search for .uk or .co.uk
2 - a normal (not text) unique index again on ldhDomain to prevent multiple documents with example.co.uk

12

Hi @steevej al I was after was the MongoDB equivalent of a no-duplicates, the issue came from a personal misunderstanding of how the index works, and I don’t know how to explain what I’ve ended up using exactly.

I’m not interested in searching for .uk or .co.uk in this field as there is also a tld field for that purpose.

I’m gonna delete this thread so not to confuse anyone else, thank you for trying to help, I don’t think I explained what I was after clearly, and that caused more issues.

The index I ended up using was: 'Single field Index, 1 (ASC), unique’ instead of ‘single field index, text, unique’

1 Like
13

Your last post is clear enough.

Please do not delete the thread.

14

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.