I have a web application split into multiple services which all use the same database. All services query and alter mostly the same collections. This arises the problem that bugs detected in production are very hard to debug as you don’t really know which services updated a given record and how. Given so, we are thinking to implement history tracking mechanism that would let us to track the last N versions of all records in a given collection alongside some metadata (like which service did the update and other relevant info).
For this, I was thinking that Change Streams are a good fit, but unfortunately there does not seem to be a way to append metadata to an update/insert operation without actually storing it in the record itself. I saw in the docs that there is a
comment option (https://docs.mongodb.com/manual/reference/command/update/), but it won’t be available in the Change Stream.
What approaches would you suggest to implement this functionality without hurting performance? Those are the properties/constraints of the system:
- the database is a MongoDB Replica Set
- the history of changes will only be queried manually so it shouldn’t be highly performant
- the number of stored versions should be limited so we don’t run out of storage space
- the performance of existing queries should not be impacted
- the performance of insert/update operations should be impacted as little as possible
- preferably the history of changes won’t be stored on the same database as the records themselves, so there won’t be additional pressure on the production database