MongoDB Data Purging Automation

Ops and Admin Installation & Upgrades
,
1

Hi All,

Currently we have MongoDB Cluster with 1 Primary Node & 1 Secondary node. Every month we need to do data purging to get better performance. Currently we are doing MongoDB data purging manually. We need to automate this task.

Can someone guide use how to automate “Data Purging of MongoDB”?

2

You should look into TTL indexes. Time to Live indexes allow MongoDB to automatically delete a document after a n number of seconds.

db.colname.createIndex( { "date_field_in_document": 1 }, { expireAfterSeconds: 3600 } )
3

Thanks @tapiocaPENGUIN ,

@tapiocaPENGUIN

  1. Can we use some kind of shell/programming language script for mongodb data purging?

  2. If we enable TTL then how to ensure backup? Does it apply to data present in mongodb which is there before we creation of TTL?

4

Welcome to the MongoDB Community Forums @Phonon_Infrastructur!

You can use shell/programming scripts to interact with your MongoDB deployment or environment.

To help understand your use case, please share more details about:

  • the current manual data purging process that you would like to automate.
  • your O/S and MongoDB server versions
  • any automation/orchestration tooling you would be using to automate tasks (if applicable)

TTL indexes do not do anything special in terms of document removal: expired documents are removed from the primary and those deletions replicate to secondaries via the oplog.

Your backup strategy would remain the same: choose a supported backup method and schedule that meets your requirements for disaster recovery considerations like Recovery Point Objective (RPO), Recovery Time Objective (RTO), performance impact, etc.

TTL indexes are based on an indexed date field. The TTL background thread runs every 60 seconds and removes all documents with an indexed value that is older than the TTL index expireAfterSeconds value.

Any documents matching the TTL expiry will be removed, even if they were created before the TTL index.

Regards,
Stennie

1 Like