Unable to connect to DB using remote server

Hello everyone,
I am relatively new to mongo-verse and just testing out mongodb in my dev environment. I have a standalone installation which has a custom data and log path, nothing fancy. When I try to connect to the DB the following things happen,

  1. Connection from the same host using the hostname works as expected
  2. Connection from any other host from the same domain fails on multiple hosts.

Information about the server running mongodb,

  1. OS
bash-4.4$  cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.2"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.2"
  1. CPU
bash-4.4$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           4
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz
Stepping:            7
CPU MHz:             2194.843
BogoMIPS:            4389.68
Hypervisor vendor:   VMware
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            25344K
NUMA node0 CPU(s):   0-3
  1. DB Startup Command

mongod --auth --tlsAllowInvalidCertificates --dbpath /custompath/mongo/data --logpath /custompath/mongo/log/mongod.log --fork --bind_ip_all --verbose

  1. mongo connection

mongosh “mongodb://user:crypticpassword@dbhostname”

  1. Error
C:\mongosh>mongosh "mongodb://user:crypticpassword@dbhostname"
Current Mongosh Log ID: 6123dc021d2e9d2c9fdff608
Connecting to:          mongodb://<credentials>@dbhostname/?directConnection=true
MongoServerSelectionError: connection <monitor> to x.x.x.x:27017 closed
  1. Successfull connection log
{
    "t": {
        "$date": "2021-08-23T13:30:35.139-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn1",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "hello": true,
            "maxAwaitTimeMS": 10000,
            "topologyVersion": {
                "processId": {
                    "$oid": "6123d8d8ca6db228ca7a1f29"
                },
                "counter": 0
            },
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 313,
        "locks": {},
        "remote": "127.0.0.1:37224",
        "protocol": "op_msg",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.506-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22943,
    "ctx": "listener",
    "msg": "Connection accepted",
    "attr": {
        "remote": "x.x.x.x:36652",
        "uuid": "44ca9d62-bfcf-4757-8014-fa1918fb2320",
        "connectionId": 52,
        "connectionCount": 2
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.510-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 51800,
    "ctx": "conn52",
    "msg": "client metadata",
    "attr": {
        "remote": "x.x.x.x:36652",
        "client": "conn52",
        "doc": {
            "driver": {
                "name": "nodejs",
                "version": "4.1.0"
            },
            "os": {
                "type": "Linux",
                "name": "linux",
                "architecture": "x64",
                "version": "4.18.0-193.60.2.el8_2.x86_64"
            },
            "platform": "Node.js v14.17.4, LE (unified)|Node.js v14.17.4, LE (unified)",
            "application": {
                "name": "mongosh 1.0.5"
            }
        }
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.510-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn52",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "ismaster": true,
            "helloOk": true,
            "client": {
                "driver": {
                    "name": "nodejs",
                    "version": "4.1.0"
                },
                "os": {
                    "type": "Linux",
                    "name": "linux",
                    "architecture": "x64",
                    "version": "4.18.0-193.60.2.el8_2.x86_64"
                },
                "platform": "Node.js v14.17.4, LE (unified)|Node.js v14.17.4, LE (unified)",
                "application": {
                    "name": "mongosh 1.0.5"
                }
            },
            "compression": ["none"],
            "loadBalanced": false,
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 329,
        "locks": {},
        "remote": "x.x.x.x:36652",
        "protocol": "op_query",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.541-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22943,
    "ctx": "listener",
    "msg": "Connection accepted",
    "attr": {
        "remote": "x.x.x.x:36654",
        "uuid": "d7c6255a-ec1c-4cff-b485-7bc18045f67d",
        "connectionId": 53,
        "connectionCount": 3
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.542-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 51800,
    "ctx": "conn53",
    "msg": "client metadata",
    "attr": {
        "remote": "x.x.x.x:36654",
        "client": "conn53",
        "doc": {
            "driver": {
                "name": "nodejs",
                "version": "4.1.0"
            },
            "os": {
                "type": "Linux",
                "name": "linux",
                "architecture": "x64",
                "version": "4.18.0-193.60.2.el8_2.x86_64"
            },
            "platform": "Node.js v14.17.4, LE (unified)|Node.js v14.17.4, LE (unified)",
            "application": {
                "name": "mongosh 1.0.5"
            }
        }
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.542-04:00"
    },
    "s": "D1",
    "c": "ACCESS",
    "id": 20226,
    "ctx": "conn53",
    "msg": "Returning user from cache",
    "attr": {
        "user": {
            "user": "admin",
            "db": "admin"
        }
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.542-04:00"
    },
    "s": "D1",
    "c": "ACCESS",
    "id": 20226,
    "ctx": "conn53",
    "msg": "Returning user from cache",
    "attr": {
        "user": {
            "user": "admin",
            "db": "admin"
        }
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.542-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn53",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "ismaster": true,
            "helloOk": true,
            "client": {
                "driver": {
                    "name": "nodejs",
                    "version": "4.1.0"
                },
                "os": {
                    "type": "Linux",
                    "name": "linux",
                    "architecture": "x64",
                    "version": "4.18.0-193.60.2.el8_2.x86_64"
                },
                "platform": "Node.js v14.17.4, LE (unified)|Node.js v14.17.4, LE (unified)",
                "application": {
                    "name": "mongosh 1.0.5"
                }
            },
            "compression": ["none"],
            "loadBalanced": false,
            "saslSupportedMechs": "admin.admin",
            "speculativeAuthenticate": {
                "saslStart": 1,
                "mechanism": "SCRAM-SHA-256",
                "payload": {
                    "$binary": {
                        "base64": "biwsbj1hZG1pbixyPVpUTFloZm5tS2I1cHkwWmdkVFlRRC9HYjY4c1pqeDZM",
                        "subType": "0"
                    }
                },
                "autoAuthorize": 1,
                "options": {
                    "skipEmptyExchange": true
                },
                "db": "admin"
            },
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 582,
        "locks": {},
        "authorization": {
            "startedUserCacheAcquisitionAttempts": 2,
            "completedUserCacheAcquisitionAttempts": 2,
            "userCacheWaitTimeMicros": 1
        },
        "remote": "x.x.x.x:36654",
        "protocol": "op_query",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.555-04:00"
    },
    "s": "D1",
    "c": "ACCESS",
    "id": 20226,
    "ctx": "conn53",
    "msg": "Returning user from cache",
    "attr": {
        "user": {
            "user": "admin",
            "db": "admin"
        }
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.555-04:00"
    },
    "s": "I",
    "c": "ACCESS",
    "id": 20250,
    "ctx": "conn53",
    "msg": "Authentication succeeded",
    "attr": {
        "mechanism": "SCRAM-SHA-256",
        "speculative": true,
        "principalName": "admin",
        "authenticationDatabase": "admin",
        "remote": "x.x.x.x:36654",
        "extraInfo": {}
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.555-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn53",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "saslContinue": 1,
            "conversationId": 1,
            "payload": "xxx",
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 125,
        "locks": {},
        "authorization": {
            "startedUserCacheAcquisitionAttempts": 1,
            "completedUserCacheAcquisitionAttempts": 1,
            "userCacheWaitTimeMicros": 3
        },
        "remote": "x.x.x.x:36654",
        "protocol": "op_msg",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.556-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn53",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "ping": 1,
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 38,
        "locks": {},
        "remote": "x.x.x.x:36654",
        "protocol": "op_msg",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:30:39.563-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn53",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "buildInfo": 1,
            "lsid": {
                "id": {
                    "$uuid": "4ae2d08f-1165-4b32-9969-4078722fc8f1"
                }
            },
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 1923,
        "locks": {},
        "remote": "x.x.x.x:36654",
        "protocol": "op_msg",
        "durationMillis": 0
    }
}
  1. Failed connection log
{
    "t": {
        "$date": "2021-08-23T13:32:15.220-04:00"
    },
    "s": "I",
    "c": "COMMAND",
    "id": 51803,
    "ctx": "conn1",
    "msg": "Slow query",
    "attr": {
        "type": "command",
        "ns": "admin.$cmd",
        "appName": "mongosh 1.0.5",
        "command": {
            "hello": true,
            "maxAwaitTimeMS": 10000,
            "topologyVersion": {
                "processId": {
                    "$oid": "6123d8d8ca6db228ca7a1f29"
                },
                "counter": 0
            },
            "$db": "admin"
        },
        "numYields": 0,
        "reslen": 313,
        "locks": {},
        "remote": "127.0.0.1:37224",
        "protocol": "op_msg",
        "durationMillis": 0
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:15.905-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22943,
    "ctx": "listener",
    "msg": "Connection accepted",
    "attr": {
        "remote": "x.x.x.x:57011",
        "uuid": "895126b1-0456-4f65-8571-4828dc85205b",
        "connectionId": 55,
        "connectionCount": 2
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:15.911-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn55",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 394
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:15.911-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn55",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 536
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:15.911-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22944,
    "ctx": "conn55",
    "msg": "Connection ended",
    "attr": {
        "remote": "x.x.x.x:57011",
        "uuid": "895126b1-0456-4f65-8571-4828dc85205b",
        "connectionId": 55,
        "connectionCount": 1
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:16.588-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22943,
    "ctx": "listener",
    "msg": "Connection accepted",
    "attr": {
        "remote": "x.x.x.x:57012",
        "uuid": "37fea4df-d53d-47d9-95a8-cde1d44cc3e4",
        "connectionId": 56,
        "connectionCount": 2
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:16.588-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn56",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 394
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:16.588-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn56",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 536
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:16.588-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22944,
    "ctx": "conn56",
    "msg": "Connection ended",
    "attr": {
        "remote": "x.x.x.x:57012",
        "uuid": "37fea4df-d53d-47d9-95a8-cde1d44cc3e4",
        "connectionId": 56,
        "connectionCount": 1
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:17.230-04:00"
    },
    "s": "I",
    "c": "NETWORK",
    "id": 22943,
    "ctx": "listener",
    "msg": "Connection accepted",
    "attr": {
        "remote": "x.x.x.x:57013",
        "uuid": "148ab088-4f1a-4986-938a-824366f94e9b",
        "connectionId": 57,
        "connectionCount": 2
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:17.236-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn57",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 394
    }
}{
    "t": {
        "$date": "2021-08-23T13:32:17.236-04:00"
    },
    "s": "D1",
    "c": "-",
    "id": 23074,
    "ctx": "conn57",
    "msg": "User assertion",
    "attr": {
        "error": "HostUnreachable: Connection reset by peer",
        "file": "src/mongo/transport/service_state_machine.cpp",
        "line": 536
    }
}

Help is appreciated!

1 Like

Possibly you started mongod on the host server only on the internal loopback (127.0.0.1) interface.

2 Likes

Another avenue is that your firewall setting is preventing incoming connection to the given address or port.

2 Likes

I started the mongo with a --bind_ip_all, which should allow it to be started on 0.0.0.0
I suspect this is related to some firewall issue. I am going to investigate this a bit more and update once I find the root cause.

2 Likes