TTL deleting thread isn't working?

[Ilya_Kantor]

Is it possible that TTL deleting thread is not running at all for some reason?

I have a collection db.expiringdownloadlinks with such records:

{ "_id" : ..., "expires" : ISODate("2021-12-26T23:32:42.190Z"), "linkId" : "1z0hkcj/tutorial.zip"}

I created an index on expires, here’s how db.expiringdownloadlinks.getIndexes() shows it:

{
  "v" : 2,
  "key" : {
    "expires" : 1
  },
  "name" : "expires_1",
  "ns" : "js_ru.expiringdownloadlinks",
  "expireAfterSeconds" : 0,
  "background" : true
}

Is it correct that a record with expires field like above (2021 year) must be deleted?

For some reason, nothing happens to it.

P.S. I tried dropping and re-creating the index, and also tried waiting for 1 hour.

[steevej (Steeve Juneau)]

You scenario works fine for me.

You must have a typo somewhere.

Check database name? collection name? field name?

[Ilya_Kantor]

Thanks.

Solved. TTL wasn’t running due to a replication issue.

[Stennie_X (Stennie)]

Welcome to the MongoDB Community @Ilya_Kantor !

I see found a solution before I posted this draft, but it may still be a useful reference for troubleshooting.

@Ilya_Kantor

Is it possible that TTL deleting thread is not running at all for some reason?

By default the TTL monitor thread runs every 60 seconds. Each invocation should increment the ttl.passes metric in db.serverStatus():

test> db.serverStatus().metrics.ttl
{ deletedDocuments: Long("0"), passes: Long("6") }

test> db.serverStatus().metrics.ttl
{ deletedDocuments: Long("0"), passes: Long("7") }

If the ttl.passes count isn’t changing every minute or so, it is possible the TTL thread is not running or an admin has configured a longer interval.

If the number of passes is increasing but there are no new deletedDocuments, I would try running an equivalent query to see if there are documents that should have expired.

For example:

db.expiringdownloadlinks.insertMany([
   {
       "expires" : ISODate("2021-12-26T23:32:42.190Z"), 
       "note": "Expires immediately -- valid BSON Date in the past" 
   },
   {
       "expires" : ISODate("2025-12-26T23:32:42.190Z"), 
       "note": "Expires in the future -- valid BSON Date" 
   },
   {
       "expires" : "2021-12-26T23:32:42.190Z", 
       "note": "Won't expire -- not a valid BSON Date" 
   },
   {
       "note": "Won't expire -- missing the `expires` field" 
   }
])

Checking expected results before adding a TTL index:

// Documents with past expiry date
db.expiringdownloadlinks.find({
   "expires" : {
      $lte: new Date()
   }
})

// Documents with future expiry date
db.expiringdownloadlinks.find({
   "expires" : {
      $gt: new Date()
   }
})

// Documents that will never expire
db.expiringdownloadlinks.find({
   "expires" : {
      $not: {
         $type: "date",
      }
   }
})

After adding a TTL index on expires any documents with past expiry date should be removed within 60 seconds:

// TTL expiry per-document
// https://www.mongodb.com/docs/manual/tutorial/expire-data/
db.expiringdownloadlinks.createIndex( 
   { "expires": 1 }, 
   { expireAfterSeconds: 0 }
)

If the TTL expiry isn’t working for you as above, please provide some further details including:

• Your specific version of MongoDB server
• Output of db.serverStatus().metrics.ttl

Regards,
Stennie

[Ilya_Kantor]

Thank you @Stennie_X for the in-depth answer!

I wish I knew this TTL metrics stuff before asking.
Surely it’ll be useful for future searchers!