Mongod service doesn't stop and restart after fatal exception

OS: Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1099-aws x86_64)
MongoDB: v4.4.5

MongoDB experiences fatal exceptions such this one logged in mongod.log

{"t":{"$date":"2023-03-30T00:03:11.041+00:00"},"s":"F",  "c":"CONTROL",  "id":4757800, "ctx":"ftdc","msg":"Writing fatal message","attr":{"message":"terminate() called. An exception is active; attempting to gather more information"}}

After that, the mongod.service log shows systemd indicating that it is stopping mongod (output from journalctl -u mongod.service)

Mar 30 00:03:11 ip-xxx-xxx-xxx-xxx mongod[27818]: {"t":{"$date":"2023-03-30T00:03:11.933+00:00"},"s":"F",  "c":"CONTROL",Mar 30 14:30:05 ip-xxx-xxx-xxx-xxx systemd[1]: Stopping MongoDB Database Server...

But checking the status of the service with systemctl status mongod.service shows that it is still active (output from sudo journalctl -u mongod.service) but there are not further entries in mongod.log.

● mongod.service - MongoDB Database Server
   Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2023-03-27 11:14:27 UTC; 72h ago
 Main PID: 29624 (mongod)
   CGroup: /system.slice/mongod.service
           └─29624 /usr/bin/mongod --config /etc/mongod.conf

My mongod.conf is pretty cookie cutter, as far as I can tell (will include it below).

Is there anything I can do get mongod to shutdown and restart properly? Or anything I can do to help mongod recovery from the exception?

This is my first post in this community so, if I haven’t included something that would be helpful, please just let me know and I will add it as quickly as I can.

Thank you!

# mongod.conf
  dbPath: /var/lib/mongodb
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

  port: 27017

  timeZoneInfo: /usr/share/zoneinfo

## Enterprise-Only Options:

Try with latest mongodb version
Check this thread

1 Like

Ah yes. I do always see those same FTDC log lines. I will start working on updating mongo to v6 in a test env, but would like to try disabling FTDC in production and see if that helps. Do you think that’s a good thing to try?

Yes, you could try that. I guess you will get rid of these log lines (because FTDC is creating them), but the Mongo will be still crashing. You should check ulimit (if you are on Linux), antivirus (if any) activity for Mongo directories, and sure, disk space and RAM memory utilization. Keep in mind that disk space and memory utilization could be temporarily high, which means you should have more space than it is consuming at the load.

1 Like

Great! I will carry on updating mongodb. Hopefully that will do the trick :crossed_fingers:
Thank you for helping.

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.