TTL deleting thread isn't working?

Is it possible that TTL deleting thread is not running at all for some reason?

I have a collection db.expiringdownloadlinks with such records:

{ "_id" : ..., "expires" : ISODate("2021-12-26T23:32:42.190Z"), "linkId" : "1z0hkcj/tutorial.zip"}

I created an index on expires, here’s how db.expiringdownloadlinks.getIndexes() shows it:

{
  "v" : 2,
  "key" : {
    "expires" : 1
  },
  "name" : "expires_1",
  "ns" : "js_ru.expiringdownloadlinks",
  "expireAfterSeconds" : 0,
  "background" : true
}

Is it correct that a record with expires field like above (2021 year) must be deleted?

For some reason, nothing happens to it.

P.S. I tried dropping and re-creating the index, and also tried waiting for 1 hour.

You scenario works fine for me.

You must have a typo somewhere.

Check database name? collection name? field name?

Thanks.

Solved. TTL wasn’t running due to a replication issue.

Welcome to the MongoDB Community @Ilya_Kantor !

I see found a solution before I posted this draft, but it may still be a useful reference for troubleshooting.

By default the TTL monitor thread runs every 60 seconds. Each invocation should increment the ttl.passes metric in db.serverStatus():

test> db.serverStatus().metrics.ttl
{ deletedDocuments: Long("0"), passes: Long("6") }

test> db.serverStatus().metrics.ttl
{ deletedDocuments: Long("0"), passes: Long("7") }

If the ttl.passes count isn’t changing every minute or so, it is possible the TTL thread is not running or an admin has configured a longer interval.

If the number of passes is increasing but there are no new deletedDocuments, I would try running an equivalent query to see if there are documents that should have expired.

For example:

db.expiringdownloadlinks.insertMany([
   {
       "expires" : ISODate("2021-12-26T23:32:42.190Z"), 
       "note": "Expires immediately -- valid BSON Date in the past" 
   },
   {
       "expires" : ISODate("2025-12-26T23:32:42.190Z"), 
       "note": "Expires in the future -- valid BSON Date" 
   },
   {
       "expires" : "2021-12-26T23:32:42.190Z", 
       "note": "Won't expire -- not a valid BSON Date" 
   },
   {
       "note": "Won't expire -- missing the `expires` field" 
   }
])

Checking expected results before adding a TTL index:

// Documents with past expiry date
db.expiringdownloadlinks.find({
   "expires" : {
      $lte: new Date()
   }
})

// Documents with future expiry date
db.expiringdownloadlinks.find({
   "expires" : {
      $gt: new Date()
   }
})

// Documents that will never expire
db.expiringdownloadlinks.find({
   "expires" : {
      $not: {
         $type: "date",
      }
   }
})

After adding a TTL index on expires any documents with past expiry date should be removed within 60 seconds:

// TTL expiry per-document
// https://www.mongodb.com/docs/manual/tutorial/expire-data/
db.expiringdownloadlinks.createIndex( 
   { "expires": 1 }, 
   { expireAfterSeconds: 0 }
)

If the TTL expiry isn’t working for you as above, please provide some further details including:

  • Your specific version of MongoDB server
  • Output of db.serverStatus().metrics.ttl

Regards,
Stennie

1 Like

Thank you @Stennie for the in-depth answer!

I wish I knew this TTL metrics stuff before asking.
Surely it’ll be useful for future searchers!

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.