We are using mongo for storing transient data which needs to be deleted periodically. every time data is accessed we need to update the value of expiry field this expiry field is indexed which get’s used to query data for bulk delete. this expiry field earlier was of the type TTL Index but the we dropped it and created normal index on it.
following query was getting used when this field was ttl type index.
db.log_events.createIndex( { “expireAt”: 1 }, { expireAfterSeconds: 0 } )
We are using mongo db 4.4 and our db size is 20GB
our index size is also coming at GB (we are not sure why such case maybe it’s because if change in ttl value on every access)
Most of the time we see that just 30 minutes before BIG ttl delete operation (more than 1000 ops per second) the connections spike and write latencies spike in db, and then after 30 minute we always see TTL delete operation being triggered.
We would like to know what operations are running which are triggering these ttls. and what should be done for the same.
Sounds exactly like:
From this description an impulse of connections and writes and inherintly(or explicily) sessions. Then the sessions are then cleaned per TTL.
1 Like