`mongos` process down with "msg":"Invariant failure" log


I have 3 mongos, 3 mongod ( config ), and 1 replica set with 2 mongo 1 arbiter server for data.
All mongodb versions are same v6.0.3 on CentOS 7

db version v6.0.3
Build Info: {
    "version": "6.0.3",
    "gitVersion": "f803681c3ae19817d31958965850193de067c516",
    "openSSLVersion": "OpenSSL 1.0.1e-fips 11 Feb 2013",
    "modules": [],
    "allocator": "tcmalloc",
    "environment": {
        "distmod": "rhel70",
        "distarch": "x86_64",
        "target_arch": "x86_64"

Recently mongos process down couple of times, with these logs,

**{"t":{"$date":"2023-08-24T08:43:41.392+00:00"},"s":"F",  "c":"ASSERT",   "id":23079,   "ctx":"conn29491736","msg":"Invariant failure","attr":{"expr":"shardResult.shardHostAndPort","file":"src/mongo/db/pipeline/sharded_agg_helpers.cpp","line":1440}}**

**{"t":{"$date":"2023-08-24T08:43:41.392+00:00"},"s":"F",  "c":"ASSERT",   "id":23080,   "ctx":"conn29491736","msg":"\n\n***aborting after invariant() failure\n\n"}**

{"t":{"$date":"2023-08-24T08:43:41.392+00:00"},"s":"F",  "c":"CONTROL",  "id":6384300, "ctx":"conn29491736","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}

It occurred while querying through Compass, query got stuck and mongos has down at that time.
I tried to execute query again but it returned result in a short time through the other mongos.

I was wondering what can caused this error, and how I can prevent mongos process down.

Any leads and suggestions are much appreciated.

Kind regards,

A recommended step is to upgrade to the latest 6.0 version 6.0.9

If this issue persists after that log an issue at jira.mongodb.com

@chris ,
Do you know the cause of the issue?
It’s happening the same way for me.

Same advice. Upgrade to the latest patch release and if the issue persists create an issue on the mongodb jira.