Dropping admin.system.new_users collection

Working with Data Developer Tools
,
1

Hi there,

I have been using MongoDB for over ten years, but today I am facing a problem and I don’t know how to best deal with.
So I have a large production database that started out with MongoDB 2.x (not sure exactly which, perhaps 2.4), and which has been updated all the way to 4.2 and very recently to 5.0. I thought everything was ok until I discovered, today, that mongodump no longers perform my daily backups.

Failure message is:

mongodump --oplog -u admin -p <my password here> --authenticationDatabase admin --out /data/backup/
2023-03-27T08:11:27.198+0200    Failed: error creating intents to dump: error creating intents for database admin: error counting admin.system.new_users: (Unauthorized) not authorized on admin to execute command { count: "system.new_users", lsid: { id: UUID("11218ac0-f185-4035-ac82-e06b31273527") }, $clusterTime: { clusterTime: Timestamp(1679897486, 4), signature: { hash: BinData(0, 83B632FBABBB17FECFA209B38BD559DD413C8E97), keyId: 7158403034257555467 } }, $db: "admin", $readPreference: { mode: "primaryPreferred" } }

After some googling/reading it seems that this admin.system.new_users collection was (automatically) created during upgrade to 2.6, but never cleaned up. Its content is obsolete and could be safely deleted, and that would fix the mongodump issue, except that, for some reason, I can’t.

Every attempt to delete it fails with “can’t drop system collection” (probably because it is in the system namespace), and it can’t be renamed either, any attempt to do so ends with “Invalid system namespace”.

Looking at mongodb source code, there seem to be no easy way to drop that old unwanted collection which prevents me from backing up my database.

Actually I managed to get my backups working again by adding a specific role to my admin user privileges: [ { resource : { "db" : "admin", "collection" : "system.new_users" }, actions: [ "find" ] }], but this isn’t fully satisfying.
Of course a full re-creation of the database without the offending collection would work, but that would result in an unacceptable downtime for my customers.

So my question: is there a way to have mongodb drop that collection, bypassing the system namespace checks ? It seems that crafting a specific oplog entry could actually do that, but I not very comfortable with playing with such dark magic on a production database!

Any idea ?
Thanks

2

Check this jira ticket
You have to give explicit drop privileges on that collection
https://jira.mongodb.org/browse/SERVER-20793

3

Thanks for your reply, but unfortunately it doesn’t help.

Without permission, the result is:

db.system.new_users.drop()
uncaught exception: Error: drop failed: {
        "ok" : 0,
        "errmsg" : "not authorized on admin to execute command { drop: \"system.new_users\", lsid: { id: UUID(\"39647226-1638-45dc-ba17-0edffeab4f2c\") }, $clusterTime: { clusterTime: Timestamp(1679982428, 1), signature: { hash: BinData(0, 29228B39097FD3021D97D56D945171720FEDF25C), keyId: 7158403034257555467 } }, $db: \"admin\" }",
        "code" : 13,
        "codeName" : "Unauthorized",
        "operationTime" : Timestamp(1679982429, 5)
}

With the appropriate permission, it goes further but still won’t do the job:

pc:PRIMARY> db.updateRole("cleanup",{privileges: [ { resource : { "db" : "admin", "collection" : "system.new_users" }, actions: [ "find", "dropCollection" ] }], roles: [ "root" ]})
pc:PRIMARY> db.system.new_users.drop()
uncaught exception: Error: drop failed: {
        "ok" : 0,
        "errmsg" : "can't drop system collection admin.system.new_users",
        "code" : 20,
        "codeName" : "IllegalOperation",
        "operationTime" : Timestamp(1679982508, 5)
}

That would have been too easy…

4

Did you try rename method?
https://jira.mongodb.org/browse/SERVER-5972

5

Unfortunately yes, the collection can’t be renamed either:

pc:PRIMARY> db.updateRole("cleanup",{privileges: [ { resource : { anyResource: true }, actions: [ "renameCollectionSameDB", "find", "insert", "dropCollection", "anyAction" ] }], roles: [ "root" ]})
pc:PRIMARY> db.system.new_users.renameCollection("oldusers")
{
        "ok" : 0,
        "errmsg" : "error with source namespace: Invalid system namespace: admin.system.new_users",
        "code" : 20,
        "codeName" : "IllegalOperation",
        "operationTime" : Timestamp(1679993389, 10)
}
6

Hi @Nicolas_Bouquet,

Can you try to use this role to drop the collection db.system.new_users:

P.S. I have never try to used it :scream:
BR

7

After some hesitation, I finally tried… But it doesn’t work either, still the same ‘IllegalOperation’ result. I feel like it cannot be solved by permissions at all.

1 Like
8

I tested this in test db
Created a collection system.xyz when I drop it says illegal operation
Was able to rename it and drop
That jira ticket user also says he can drop the collection after rename
Can you show output of
db
Show collections when you tried rename

9

@Ramachandra_Tummala,

Here is the list of collection in that admin database:

pc:PRIMARY> show collections
system.backup_users
system.keys
system.new_users
system.roles
system.users
system.version

Not sure why the rename method doesn’t work for me, it could because it was ‘fixed’ in the mongodb version I am using (5.0.14), or because or doesn’t work with ‘admin’ database.