Mongod service failing to start

Hi,
I upgrade mongod from 3.4 to 4.2 but the service is failing to start. below are the logs. Can anyone please help me how to fix the issue? Cheers

systemctl status mongod

● mongod.service - MongoDB Database Server
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sat 2022-07-16 21:43:42 UTC; 8s ago
Docs: https://docs.mongodb.org/manual
Process: 925479 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=100)
Process: 925477 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 925475 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 925473 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
Main PID: 919663 (code=exited, status=0/SUCCESS)

Jul 16 21:43:40 NMS systemd[1]: Starting MongoDB Database Server…
Jul 16 21:43:40 NMS mongod[925479]: about to fork child process, waiting until server is ready for connections.
Jul 16 21:43:40 NMS mongod[925479]: forked process: 925481
Jul 16 21:43:42 NMS mongod[925479]: ERROR: child process failed, exited with error number 100
Jul 16 21:43:42 NMS mongod[925479]: To see additional information in this output, start without the “–fork” option.
Jul 16 21:43:42 NMS systemd[1]: mongod.service: Control process exited, code=exited status=100
Jul 16 21:43:42 NMS systemd[1]: mongod.service: Failed with result ‘exit-code’.
Jul 16 21:43:42 NMS systemd[1]: Failed to start MongoDB Database Server.
[root@NMS yum.repos.d]# mongod --config=/etc/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 925613
ERROR: child process failed, exited with error number 100
To see additional information in this output, start without the “–fork” option.

Thanks

Hi,
Can you explain what upgrade method you used? Did you upgrade the mongod binaries?
It will require upgrading from one major version to the next.

you would have to upgrade in this order:

3.4 -> 3.6, 3.6 -> 4.0, 4.0 -> 4.2
2 Likes

Thanks for your response. I followed the steps shown in below link.
LayerStack Tutorials - LayerStack - How to upgrade MongoDB on Linux Cloud Servers.

if you upgrade the mongod binaries you can not upgrade directly from 3.4 to 4.2. it is always better to follow official MongoDB documentation, also there was a similar topic - please read:

2 Likes

Assuming upgrade went fine i see issue could be due to permissions
Error 100 means permissions issue on your datafiles
Check ownership of your dbpath/logpath dirs
You should start service with systemctl which calls mongod to start mongodb
I see you are starting mongod as root user?You should not use root user

1 Like

Hi Ramachandra,
Please see the below permission for dbpath/logpath dirs

Directory rights
storage:
dbPath: /var/lib/mongo
drwxr-xr-x. 4 mongod mongod 16K Jul 16 21:45 mongo

System Log:
path: /var/log/mongodb/mongod.log
drwxr-xr-x. 2 mongod mongod 24 Jun 15 20:01 mongodb
-rw-r-----. 1 mongod mongod 29M Jul 20 03:40 mongod.log

and which user should i use?

Thanks

Use normal user and use sudo when you need root privileges
Your mongod command output asks you to run the command without fork.Did you try this?
Also check mongod.log
It will give more details on why it is failing to start

Try to spin up your own mongod using a different port,dbpath,logpath like below

mongod --port 29000 --dbpath your_home_dir --logpath your_home_dir --fork

If it is working you can check/troubleshoot why it is failing with default config file(/etc/mongod.conf)

Hi, I did ran the command but still no success
$ mongod
2022-07-23T17:13:01.129+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols ‘none’
2022-07-23T17:13:01.132+0000 W ASIO [main] No TransportLayer configured during NetworkInterface startup
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] MongoDB starting : pid=340778 port=27017 dbpath=/data/db 64-bit host=NMS
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] db version v4.2.21
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] git version: b0aeed9445ff41af07449fa757e1f231bce990b3
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.1k FIPS 25 Mar 2021
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] allocator: tcmalloc
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] modules: none
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] build environment:
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] distmod: rhel80
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] distarch: x86_64
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] target_arch: x86_64
2022-07-23T17:13:01.133+0000 I CONTROL [initandlisten] options: {}
2022-07-23T17:13:01.133+0000 E NETWORK [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2022-07-23T17:13:01.133+0000 F - [initandlisten] Fatal Assertion 40486 at src/mongo/transport/transport_layer_asio.cpp 684
2022-07-23T17:13:01.133+0000 F - [initandlisten] \n\n***aborting after fassert() failure\n\n

[Avall@NMS etc]$ mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/mongod.log --port 29000 --fork
about to fork child process, waiting until server is ready for connections.
forked process: 340936
ERROR: child process failed, exited with error number 1
To see additional information in this output, start without the “–fork” option.
[Avall@NMS etc]$ mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/mongod.log --port 29000
2022-07-23T17:16:52.088+0000 F CONTROL [main] Failed global initialization: FileRenameFailed: Could not rename preexisting log file “/var/log/mongodb/mongod.log” to “/var/log/mongodb/mongod.log.2022-07-23T17-16-52”; run with --logappend or manually remove file: Permission denied
[Avall@NMS etc]$ sudo mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/mongod.log --port 29000
2022-07-23T17:17:44.695+0000 I CONTROL [main] log file “/var/log/mongodb/mongod.log” exists; moved to “/var/log/mongodb/mongod.log.2022-07-23T17-17-44”.
[Avall@NMS etc]$ mongo --port 29000
MongoDB shell version v4.2.21
connecting to: mongodb://127.0.0.1:29000/?compressors=disabled&gssapiServiceName=mongodb
2022-07-23T17:18:15.038+0000 E QUERY [js] Error: couldn’t connect to server 127.0.0.1:29000, connection attempt failed: SocketException: Error connecting to 127.0.0.1:29000 :: caused by :: Connection refused :
connect@src/mongo/shell/mongo.js:353:17
@(connect):2:6
2022-07-23T17:18:15.041+0000 F - [main] exception: connect failed
2022-07-23T17:18:15.041+0000 E - [main] exiting with code 1

Check permissions/ownership of this file
ls -lrt /tmp/mongodb-27017.sock
Most likely it is owned by root when you tried to start mongod as root
You may have to remove this file and try again.Before removing make sure no mongod is running on port 27017

Most of the issues are due to permissions
Your second command would have succeeded on port 29000 if you had given a different dirpath which i have mentioned clearly in my reply to use “your home dir” but you have given the same path that was used before.

Was your third command successful?I dont see waiting for connections message.Check mongod.log
You can also check by ps -ef|grep mongod
You should not start mongod as sudo
This will create the files owned by root and when you try to start mongod as normal user it cannot read/write the files as they are owned by root
Show the contents of /var/lib/mongo and /var/log/mongodb/
Best thing is empty these dirs and start fresh

1 Like