Mongodb Replica issues - SocketException: Error connecting to 127.0.0.1:27017

Ops and Admin Replication
,
1

Hello everyone,

I am working with Mongodb Replica clusters.
I keep getting the following error. I got the same error 3-4 times in 2 weeks. I restarted the server via AWS and it worked for 4 5 days without any problems. But I need a complete solution.

Most of the things I’ve researched haven’t helped.
I did the installation from the mongodb documentation.

Mongodb is working fine now. Because I rebooted. But after 3 4 days it will give the same error again and when I type “mongo” I will not be able to connect.

Servers: AWS - Ubuntu 20.04 2CPU-1GB Ram (1 primary 2 secondary)
Mongodb version: v5.0.5

SSH connection error (mongo)

ubuntu@ip-17x-3x-3x-110:~$ mongo
MongoDB shell version v5.0.5
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
**Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :**
connect@src/mongo/shell/mongo.js:372:17
@(connect):2:6
exception: connect failed
exiting with code 1
ubuntu@ip-17x-3x-3x-11x:~$


ubuntu@ip-17x-3x-3x-11x:/var/log/mongodb$ sudo systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: failed (Result: signal) since Thu 2021-12-30 05:52:01 UTC; 16h ago
       Docs: https://docs.mongodb.org/manual
    Process: 458 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=killed, signal=KILL)
   Main PID: 458 (code=killed, signal=KILL)

Dec 19 19:26:39 ip-17x-3x-3x-11x systemd[1]: Started MongoDB Database Server.
Dec 30 05:52:01 ip-17x-3x-3x-11x systemd[1]: mongod.service: **Main process exited, code=killed,** status=9/KILL
Dec 30 05:52:01 ip-17x-3x-3x-11x systemd[1]: mongod.service: **Failed with result 'signal'.**

**Sorry for typos and translation.

Thanks.

2

Can you comment as to why the prompt is

when you run mongo command and why it is

when you run systemctl? You try to connect to localhost, shouldn’t be the same machine? Did you redacted something for security reasons? Sometimes redacted logs or output fools us and make analysis difficult.

Mongod received the kill signal so somebody or something is terminating it. What else run on this machine?

1 Like
3

Hi,

“Mongod received the kill signal so somebody or something is terminating it. What else run on this machine?”

  • Only mongodb (replicasets) is running on the machine.

“when you run systemctl? You try to connect to localhost, shouldn’t be the same machine? Did you redacted something for security reasons? Sometimes redacted logs or output fools us and make analysis difficult.”

  • Nothing has been changed for security reasons. I never changed the AWS Security group. 3 Servers are in the same security group.

I restarted 3 servers yesterday. It was working flawlessly.
Today I realized that I could not connect with Mongodb Compass and I made an SSH connection to the replica sets (my primary server) through the servers and these errors appeared again.

In addition;
The contents of the file /etc/mongod.conf

Screenshot_23
Screenshot_23757×789 13.9 KB

4 
when you run systemctl? You try to connect to localhost,

I didn’t understand what you said. I am trying to connect to the server via SSH connection.

5

Unless your mongod is up and running you will not be able to connect by mongo shell or compass
Are the servers where you ran sudo systemctl and mongo command are same?
steevej was asking why the prompts look different
ubuntu@ip-17x-3x-3x-110 vs ubuntu@ip-17x-3x-3x-11x
Could be a typo while redacting?

1 Like
6

@steevej @Ramachandra_Tummala

I edited it that way to hide the ip address. :sweat_smile:

Mongod is running. But as “steevej” said something is killing the system.
I haven’t found the solution yet…

I get the connection when the servers are restarted.
after a while (2-3 days) i can’t connect with compass or shell.

7

You will need to check your security logs to see if there are root or mongod logins, sudos? You may have third party monitoring tools that automatically kills big memory or CPU users.

1 Like
8

Do you think this kill is related to “ulimit settings”?

You will need to check your security logs to see if there are root or mongod logins, sudos? You may have third party monitoring tools that automatically kills big memory or CPU users.

I will review. Thanks. @steevej

9

Quite possible. There should be trace in the logs.

What are the specs of the machine? What is the data size?

You wrote

and

Does that mean that the 3 mongod servers of your replica set is running on the same machine?

1 Like
10

Hi,

Mongodb is running on 3 different servers. (separate servers)
Machine specs; 1GB RAM - 2CPU - 12GB Disk (Aws ec2 servers)

Quite possible. There should be trace in the logs.

I increased the ulimit values, but when I restart the machine, the ulimit keeps returning to its old values.

Thanks. @steevej

11

That is why in the link you provided it is written to do it in the started up script.

Insufficient for any serious work.

I would consider using Atlas instead.

1 Like
12

I’m testing right now. I’m not very good with mongodb.
I will increase server features in production environment. @steevej

That is why in the link you provided it is written to do it in the started up script.

I have reviewed this and researched it on different sites. None of them worked.

14

Make sure that you have added replica set nodes in the host machine in etc/hosts file.
Just like the below example -

127.0.0.1	s1 s2 s3

Note - 127.0.0.1 is your host machine and s1, s2 and s3 are the nodes (members) of the replicaset.