I plan to do TLS termination on Istio.
I have used replicaSetHorizons to implement split DNS. But during the initialization of the mongodb instance via the mongodb operator, I’m seeing only the first member of the replica being created. The rest are stalling / not being created.
In the replicaSetHorizons setting, I assume the key on the left is just an arbitrary name (that is used to group the replicaset members’ DNS names) and the value on the right should be a publicly accessible fqdn because this is the value returned to any clients that try to connect whereas the mongo agent uses internal kubernetes DNS names? how does mongodb know which line in the replicaSetHorizons correspond with which replicaset member?
replicaSetHorizons:
- mongo-replica: [public fqdn]:30667
- mongo-replica: [public fqdn]:30668
- mongo-replica: [public fqdn]:30669
I want to use the same public fqdn but run the 3 members of the replicasets on different tcp ports. This is because the [public fqdn] above will be my Istio load balancer. I have created nodeports to the individual replicaset members and configured Istio to route the traffic destined for each tcp port to each of the corresponding nodeports in my Kubernetes cluster. In the YAML below , external-mongodb-service-0.mongodb.svc.cluster.local corresponds to the nodeport’s internal DNS name in the cluster. Do I need to expose the nodeport to the internet and use the public IPs instead?
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: mongodb
namespace: mongodb
spec:
gateways:
- mongodb-gateway
hosts:
- "*"
tcp:
- match:
- port: 30667
route:
- destination:
host: external-mongodb-service-0.mongodb.svc.cluster.local
port:
number: 30667
- match:
- port: 30668
route:
- destination:
host: external-mongodb-service-1.mongodb.svc.cluster.local
port:
number: 30668
- match:
- port: 30669
route:
- destination:
host: external-mongodb-service-2.mongodb.svc.cluster.local
port:
number: 30669
The problem I’m now facing is
{"t":{"$date":"2022-02-24T02:05:11.208+00:00"},"s":"W", "c":"NETWORK", "id":21207, "ctx":"ReplCoord-0","msg":"getaddrinfo() failed","attr":{"host":"mongodb-1.mongodb-svc.mongodb.svc.cluster.local","error":"Name or service not known"}}
{"t":{"$date":"2022-02-24T02:05:11.217+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.152.3.4:48070","uuid":"083866ce-82c8-4126-a6a9-a3c5959534fb","connectionId":1,"connectionCount":1}}
{"t":{"$date":"2022-02-24T02:05:11.218+00:00"},"s":"I", "c":"NETWORK", "id":4834700, "ctx":"ReplCoord-0","msg":"isSelf could not connect via connectSocketOnly","attr":{"hostAndPort":"mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017","error":{"code":6,"codeName":"HostUnreachable","errmsg":"couldn't connect to server mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017, connection attempt failed: HostNotFound: Could not find address for mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017: SocketException: Host not found (authoritative)"}}}
{"t":{"$date":"2022-02-24T02:05:11.218+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.152.3.4:48072","uuid":"230f2036-866c-44bd-ad2a-53870eb9adfd","connectionId":2,"connectionCount":2}}
{"t":{"$date":"2022-02-24T02:05:11.218+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn1","msg":"Connection ended","attr":{"remote":"10.152.3.4:48070","uuid":"083866ce-82c8-4126-a6a9-a3c5959534fb","connectionId":1,"connectionCount":1}}
{"t":{"$date":"2022-02-24T02:05:11.218+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.152.3.4:48074","uuid":"c16bd786-3be6-4dad-81d2-121ef51c79cb","connectionId":3,"connectionCount":2}}
{"t":{"$date":"2022-02-24T02:05:11.219+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn2","msg":"Connection ended","attr":{"remote":"10.152.3.4:48072","uuid":"230f2036-866c-44bd-ad2a-53870eb9adfd","connectionId":2,"connectionCount":1}}
{"t":{"$date":"2022-02-24T02:05:11.219+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn3","msg":"client metadata","attr":{"remote":"10.152.3.4:48074","client":"conn3","doc":{"driver":{"name":"mongo-go-driver","version":"v1.3.8+prerelease"},"os":{"type":"linux","architecture":"amd64"},"platform":"go1.15.9","application":{"name":"MongoDB Automation Agent v11.0.5.6963 (git: 65ba9b52de741e00e747a80f1a93442f9fe40695)"}}}}
{"t":{"$date":"2022-02-24T02:05:11.220+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"10.152.3.4:48076","uuid":"ac0a1dea-34ce-4168-a2a9-74f6b1946105","connectionId":4,"connectionCount":2}}
{"t":{"$date":"2022-02-24T02:05:11.220+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn4","msg":"client metadata","attr":{"remote":"10.152.3.4:48076","client":"conn4","doc":{"driver":{"name":"mongo-go-driver","version":"v1.3.8+prerelease"},"os":{"type":"linux","architecture":"amd64"},"platform":"go1.15.9","application":{"name":"MongoDB Automation Agent v11.0.5.6963 (git: 65ba9b52de741e00e747a80f1a93442f9fe40695)"}}}}
Any guidance from anyone?