MongoDB files and collection Purge

Hi there,

We are using MongoDB files and chunks for storing images and pdf files that are larger than 16 MB in size. We have a requirement to purge this data at the end of the day.

We found that MongoDB provides an inbuilt feature to delete the docs using a TTL index. So defined that TTL index using the uploadDate column of the files collection. It is working perfectly and deleting the records from files collection. But the related records in the chunk collection remain undeleted which is problematic.

Another option we thought of adding uploadDate field in chunk collection and define TTL index there as well but could not find a way to achieve it. I see that we can add a metadata field in files collection but did not find a way to do it in chunk collection.

Another option we discussed is - since we have spring boot microservice , we can have a spring scheduler to invoke at defined interval and call a service method which will invoke the delete() gridFS API for deleting the expired documents. But since we can have multiple instances of this microservice with common database then each instance will end up running their own scheduler service for the same thing which does not seem to be right.

If you come across such scenario and know of achieving this then that will help me.

Thanks,
GG

Hi @GDG,
You can try using Change Streams to watch creations of documents of the fs.chunks collection and set it’s ttl on creation.

You can calculate the “end of day” using the _id field. (with getTimestamp)

Goodluck,
Rafael,