Remove/Insert is outputting constant error

Working with Data
1

Hi there, I have a code that goes like this:

async function addDOC(){

	await DOCDBCHECK.remove({ });
	console.log("Collection being updated!");
	console.log("3");
	console.log("2");
	console.log("1");
	await DOCDBCHECK.insertOne(doc);
	DOCDBCHECKBUP.insertOne(doc);
	console.log("Collection successfully updated!");

}

And it constantly outputs error saying the ID is not unique? Strange that, functions are timed to wait for one another, so there shouldn’t be a problem… whats happening?

Error outputted is:

                return callback(new error_1.MongoServerError(res.writeErrors[0]));
                                ^

MongoServerError: E11000 duplicate key error collection: data.TFTbcBUP index: _id_ dup key: { _id: ObjectId('637f9f14d839980dd5e4da91') }
    at C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\operations\insert.js:53:33
    at C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\cmap\connection_pool.js:292:25
    at C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\sdam\server.js:215:17
    at handleOperationResult (C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\sdam\server.js:287:20)
    at Connection.onMessage (C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\cmap\connection.js:219:9)
    at MessageStream.<anonymous> (C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\cmap\connection.js:60:60)
    at MessageStream.emit (node:events:513:28)
    at processIncomingData (C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\cmap\message_stream.js:132:20)
    at MessageStream._write (C:\Users\FT\Downloads\FoobaseFinal\node_modules\mongodb\lib\cmap\message_stream.js:33:9)
    at writeOrBuffer (node:internal/streams/writable:391:12) {
  index: 0,
  code: 11000,
  keyPattern: { _id: 1 },
  keyValue: {
    _id: ObjectId {
      [Symbol(id)]: Buffer(12) [Uint8Array] [
         99, 127, 159,  20, 216,
         57, 152,  13, 213, 228,
        218, 145
      ]
    }
  },
  [Symbol(errorLabels)]: Set(0) {}
}

What am I doing wrong?
I thought initially that it was the timing of the functions, given them async await properties and still this happening, can someone guide me through how to overcome this error?

(do you require any other information to assist me on this? Just let me know…)

EDIT:

I then have this on a timer:

setInterval(addDOC, oneMinute*6);
2

You are inserting the same document twice.

Once with

and another time with line righ after:

3

I am inserting it in different collections, what is wrong with that? One the document first gets removed from collection, then the updated one goes in, and the other goes into a back up collection with all the many times it restarts so a record is kept for safekeeping and development usage… I’m confused

What should I do?

4

I think whats happening is with the BUP function…
Here, look…

on second line of error… Its only complaining about the BUP one, and on the BUP one saying that there is a repeated _id…? which is automatically added by Mongo…?

5

Your choice of variable names fooled me. It has been a long since I did FORTRAN with all upper case variables.

I see one of 2 things:

  1. Both variables are referring to the same collection. Share the code where variables are initialized.
  2. Since you insert in 2 collections, you should remove from both.
6

ahh, worry not it happens even to the best.

The variables are pointing to different collections, check bellow:

var TFTDBCHECK = client.db("data").collection("TFTbc");
var TFTDBCHECKBUP = client.db("data").collection("TFTbcBUP");

Heres what I did, and as far as I’ve went I’m having no errors…

I turned the BUP one to a function, thus separating the logic processing of the insertion into two different processes, I think, at least in my head it goes like that, either that or it was a timing issue, as I see no problem in uploading the file multiple times in different collections, simply perhaps a _id issue was in the difference in milliseconds at which the insertions were being made via the Mongodb framework… whatcha recon?

Heres the working code, fyi… =)

	await TFTDBCHECK.insertOne(DOC);
	console.log("Collection successfully updated!");
	async function addBUP(){
		await TFTDBCHECKBUP.insertOne(DOC);
	}setTimeout(addBUP, 5000);

Cheers

1 Like
7

@steevej It occasionally still outputs error, I dont know how to proceed other than doing something I find inadmissible… removing the BUP insert to keep track of events…

Any ideas?

8

If mongod complains that you have a duplicate _id then it is because your code insert it twice in the collection. There is no implicit insert done by any part of the system. If the error only occurs from time to time, then most likely the initial states of your collections is not what you think.

But the real question why to you duplicate your document in 2 collections?

Also, why in the original code, do you remove the document in only 1 collection but not in both?

9

One was the back up, and the remove was meant to completelly clean collection then readding same entry but further elaborated…

I’ve moved on from that tho, changed the logic now I don’t need to remove and insert it reads if exists one before doing anything…

Thank you for the help

1 Like