Timeseries TTL not working as expected

Ops and Admin
1

We are running mongodb 8 with a timeseries collection of about 4TB storagesize. We’ve setup an expiration date of 30 days but it seems like expiration is not happening at all. How would we diagnose this? We are seeing no issues in the logs but data from more than a month a go is not expiring.

2

hello,

you should be able to diagnose this by

  1. checking to see if its even running
  2. check if its incorrectly defined or maybe its a disabled retention setting
  3. check to see if the docs you are referring to are in a bucket that never qualifies for deletion

you can run

db.getCollectionInfos({ name: "whateverYourCollectionNameIsHere" })[0].options

it should say that the expire after seconds is 30 days (2592000) in the timeseries/auto gen id time field index definition. the field named in the timeField is bson so its a ISODate in UTC. its good to know that the TTL compares wall-clock time (so any drift/wrong type stops deletions)

after that you can also run

db.serverStatus().metrics.ttl

the passes field should increase every 60 sec

or you can just check too:

db.adminCommand({ getParameter: 1, ttlMonitorEnabled: 1 })

i dont know what your comfort level is with logs but you can do this:

db.setLogComponentVerbosity({ ttl: 1 })
tail -f /var/log/mongodb/mongod.log | grep TTL

you should see logs popping up on buckets/docs deleted

if all of that is good then its likely the last point. this one trips a lot of people up since it doesnt work like non-timeseries.

the gist is:

  • the TTL applies at bucket level not doc level
  • if something gets put into the bucket that doesnt fall in the TTL then the whole bucket will not get deleted until everything falls in the TTL.

you can tweak this aging out by modifying the: bucketMaxSpanSeconds

1 Like