I have a requirement where i need to delete documents which are of older version once a new newer documents are inserted for a given date. I have two ways to do this, one if soft delete where i set some flag like isDeleted to true and delete documents through a cleaning job. Other way is to hard delete the documents of older versions.
My only requirement is execution time as i have other operations to do along with the delete inside a transaction. I want to know whether updating a flag to delete is faster or deleting the records happens fastly. I also have indexes on the records, so want to know the performance impact on both operations considering the index updates required after both operation.
The records that are available to deletion will be around 10k and can grow to max 50k. Can someone please help me in understanding the performance impact of both operations ? please let me know if any additional details needs to be mentioned.
It all depends of your data size, traffic and usage. Only you can determine the best model for your use-cases by testing both solution using your expected data size, traffic and usage.
There is too many variables involved. Any solution is a trade off. You, as a software writer, need to determine which set of trade off is best suited for your application.
Few things to consider:
soft delete does not remove keys for indexes so it might slow down queries
soft delete might involve more write than hard delete since an updated document is written back to disk
soft delete complicate each queries because they have to include (isDeleted:{$ne:true}).
hard delete might involve a bigger short time impact since indexes are updated
you might easily hide your delete implementation via a well written API
the flexible nature of mongodb allows you to start with 1 implementation and switch later to a more performant model
continuous improvement is better than delayed perfection (Mark Twain, I think)
Happy testing. It would be nice if you later share any insight you discover.