Should i change json format?.Aggregation lookup for nested documents

Stuart_S · November 22, 2022, 2:07am

i have two collections . schedlist and sched_info

db={
  "sched_info": [
    {
      "_id": {
        "$oid": "63739f0de6b9aae9c681aeba"
      },
      "cluster_info": {
        "attached_clients": {
          "bng-emake-9a": {
            "IP": "10.223.37.24",
            "MaxJobs": 20,
            "NoRemote": true,
            "Speed": 79.484589
          }
        },
        "attached_daemons": {
          "bng-ea-agt-3a": {
            "IP": "10.223.36.42",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 90.203011
          },
          "bng-ea-agt-3b": {
            "IP": "10.223.36.43",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 55.782074
          },
          "bng-ea-agt-7a": {
            "IP": "10.223.36.62",
            "MaxJobs": 18,
            "NoRemote": false,
            "Speed": 89.654556
          },
          "bng-ea-agt-7b": {
            "IP": "10.223.36.63",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 87.926308
          },
          "bng-ea-agt-7c": {
            "IP": "10.223.36.64",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 89.026802
          },
          "bng-ea-agt-7d": {
            "IP": "10.223.36.65",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 89.416687
          }
        },
        "cluster_nodes_info": {
          "active": 0,
          "available_clients": 1,
          "available_daemons": 6,
          "client_and_daemon": 7
        },
        "daemon_cpu_available_and_free_info": {
          "active": 0,
          "free": 158,
          "local": 0,
          "pending": 0,
          "total_cpu": 158
        },
        "description": {
          "analysis": "free cpu available",
          "health": true
        },
        "dump_time": "Tue Nov 15 19:44:02 2022",
        "scheduler_ip": "bng-ea-agt-7a"
      }
    },
    {
      "_id": {
        "$oid": "63739f72e6b9aae9c681aebb"
      },
      "cluster_info": {
        "attached_daemons": {
          "qnc-ea-agt-175b": {
            "IP": "10.44.138.82",
            "MaxJobs": 28,
            "NoRemote": false,
            "Speed": 0.0
          }
        },
        "cluster_nodes_info": {
          "active": 0,
          "available_clients": 0,
          "available_daemons": 1,
          "client_and_daemon": 1
        },
        "daemon_cpu_available_and_free_info": {
          "active": 0,
          "free": 28,
          "local": 0,
          "pending": 0,
          "total_cpu": 28
        },
        "description": {
          "analysis": "free cpu available",
          "health": true
        },
        "dump_time": "Tue Nov 15 19:45:53 2022",
        "scheduler_ip": "qnc-ea-agt-175a"
      }
    },
    {
      "_id": {
        "$oid": "63739fd6e6b9aae9c681aebc"
      },
      "cluster_info": {
        "attached_clients": {
          "bng-ea-agt-76a": {
            "IP": "10.224.152.170",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 46.928242
          },
          "bng-ea-agt-76b": {
            "IP": "10.224.152.171",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 48.899769
          },
          "bng-ea-agt-76c": {
            "IP": "10.224.152.172",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 53.941841
          },
          "bng-ea-agt-76d": {
            "IP": "10.224.152.173",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 48.147972
          },
          "bng-ea-agt-77a": {
            "IP": "10.224.152.174",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 48.398701
          },
          "bng-ea-agt-77b": {
            "IP": "10.224.152.175",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 45.801868
          },
          "bng-ea-agt-77c": {
            "IP": "10.224.152.176",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 43.399036
          },
          "bng-ea-agt-77d": {
            "IP": "10.224.152.177",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 43.504852
          },
          "bng-ea-agt-78a": {
            "IP": "10.224.152.178",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 46.324295
          },
          "bng-ea-agt-78b": {
            "IP": "10.224.152.179",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 48.12402
          },
          "bng-ea-agt-78c": {
            "IP": "10.224.152.180",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 43.496323
          },
          "bng-ea-agt-78d": {
            "IP": "10.224.152.181",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 37.699486
          },
          "bng-ea-agt-79a": {
            "IP": "10.224.152.182",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-79b": {
            "IP": "10.224.152.183",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-79c": {
            "IP": "10.224.152.184",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-79d": {
            "IP": "10.224.152.185",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-80a": {
            "IP": "10.224.152.186",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-80b": {
            "IP": "10.224.152.187",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-80c": {
            "IP": "10.224.152.188",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          },
          "bng-ea-agt-80d": {
            "IP": "10.224.152.189",
            "MaxJobs": 40,
            "NoRemote": true,
            "Speed": 0.0
          }
        },
        "attached_daemons": {
          "bng-ea-agt-65d": {
            "IP": "10.224.152.117",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.715843
          },
          "bng-ea-agt-66a": {
            "IP": "10.224.152.118",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.045591
          },
          "bng-ea-agt-66b": {
            "IP": "10.224.152.119",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.129501
          },
          "bng-ea-agt-66c": {
            "IP": "10.224.152.120",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.980595
          },
          "bng-ea-agt-66d": {
            "IP": "10.224.152.121",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.966152
          },
          "bng-ea-agt-67a": {
            "IP": "10.224.152.122",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.545408
          },
          "bng-ea-agt-67b": {
            "IP": "10.224.152.123",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.567656
          },
          "bng-ea-agt-67c": {
            "IP": "10.224.152.124",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.602665
          },
          "bng-ea-agt-67d": {
            "IP": "10.224.152.125",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.978537
          },
          "bng-ea-agt-68a": {
            "IP": "10.224.152.126",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.515978
          },
          "bng-ea-agt-68b": {
            "IP": "10.224.152.127",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.063932
          },
          "bng-ea-agt-68c": {
            "IP": "10.224.152.128",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.261913
          },
          "bng-ea-agt-68d": {
            "IP": "10.224.152.129",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.822884
          },
          "bng-ea-agt-69a": {
            "IP": "10.224.152.130",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.858036
          },
          "bng-ea-agt-69b": {
            "IP": "10.224.152.131",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.856615
          },
          "bng-ea-agt-69c": {
            "IP": "10.224.152.132",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.345226
          },
          "bng-ea-agt-69d": {
            "IP": "10.224.152.133",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.255632
          },
          "bng-ea-agt-70a": {
            "IP": "10.224.152.134",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.546576
          },
          "bng-ea-agt-70b": {
            "IP": "10.224.152.135",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.122202
          },
          "bng-ea-agt-70c": {
            "IP": "10.224.152.136",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.60383
          },
          "bng-ea-agt-70d": {
            "IP": "10.224.152.137",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.3407
          },
          "bng-ea-agt-71a": {
            "IP": "10.224.152.150",
            "MaxJobs": 38,
            "NoRemote": false,
            "Speed": 27.695211
          },
          "bng-ea-agt-72a": {
            "IP": "10.224.152.154",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.563095
          },
          "bng-ea-agt-72b": {
            "IP": "10.224.152.155",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.593485
          },
          "bng-ea-agt-72c": {
            "IP": "10.224.152.156",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.327879
          },
          "bng-ea-agt-72d": {
            "IP": "10.224.152.157",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.512634
          },
          "bng-ea-agt-73a": {
            "IP": "10.224.152.158",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.542143
          },
          "bng-ea-agt-73b": {
            "IP": "10.224.152.159",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.394144
          },
          "bng-ea-agt-73c": {
            "IP": "10.224.152.160",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.713577
          },
          "bng-ea-agt-73d": {
            "IP": "10.224.152.161",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.258692
          },
          "bng-ea-agt-74a": {
            "IP": "10.224.152.162",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.941082
          },
          "bng-ea-agt-74b": {
            "IP": "10.224.152.163",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.259108
          },
          "bng-ea-agt-74c": {
            "IP": "10.224.152.164",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.658989
          },
          "bng-ea-agt-74d": {
            "IP": "10.224.152.165",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.375834
          },
          "bng-ea-agt-75a": {
            "IP": "10.224.152.166",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.81979
          },
          "bng-ea-agt-75b": {
            "IP": "10.224.152.167",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 27.066696
          },
          "bng-ea-agt-75c": {
            "IP": "10.224.152.168",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 25.778721
          },
          "bng-ea-agt-75d": {
            "IP": "10.224.152.169",
            "MaxJobs": 48,
            "NoRemote": false,
            "Speed": 26.476954
          }
        },
        "cluster_nodes_info": {
          "active": 50,
          "available_clients": 20,
          "available_daemons": 38,
          "client_and_daemon": 58
        },
        "daemon_cpu_available_and_free_info": {
          "active": 1674,
          "free": 140,
          "local": 50,
          "pending": 0,
          "total_cpu": 1814
        },
        "description": {
          "analysis": "free cpu available",
          "health": true
        },
        "dump_time": "Tue Nov 15 19:49:02 2022",
        "scheduler_ip": "bng-ea-agt-71a"
      }
    }
  ],
  "schedlist": [
    {
      "_id": {
        "$oid": "6371db5f6b3a2232b4e71f58"
      },
      "active_schedulers": {
        "bng-ea-agt-7a": {
          "ice_version": "4.1",
          "netname": "icecc_bng_test"
        },
        "qnc-ea-agt-175a": {
          "ice_version": "4.0",
          "netname": null
        }
      }
    },
    {
      "_id": {
        "$oid": "6371db7f244f385faa8d0802"
      },
      "active_daemons": {
        "bng-ea-agt-7a": {
          "ice_version": "4.1",
          "netname": "icecc_bng_test"
        },
        "bng-ea-agt-3b": {
          "ice_version": "4.1",
          "netname": "icecc_bng_test"
        },
        "bng-ea-agt-7d": {
          "ice_version": "4.1",
          "netname": "icecc_bng_test"
        },
        "bng-ea-agt-7b": {
          "ice_version": "4.1",
          "netname": "icecc_bng_test"
        }
      }
    }
  ]
}

i want to join these two collections to get scheduler info a and daemon info (respective netname and version) from schedlist .
How to do lookup based on nested documents. $cluster_info.scheduler_info in local field of lookup throws a error

db.sched_info.aggregate([
  {
    "$lookup": {
      "from": "schedlist",
      "localField": {
        $getField: "$cluster_info.scheduler_ip"
      },
      "foreignField": {
        $getField: "$active_schedulers."
      },
      "as": "testing"
    }
  }
])

Pavel_Duchovny · November 22, 2022, 6:44am

Hi @Stuart_S ,

Because the lookup value is a field name and not a value it cannot be performed with such a standard lookup.

It will have to use a pipeline syntax of lookup with $objectToArray conversion just to get the field name into [{k : "active_schedulers", "v" : ....}] In order to lookup.

Now later on it will need to be assembled back to object using $arrayToObject …

As you can see it is overcomplex … What limit you to change the data model to store them together prejoined or at least turn the lookup values to be placed as a value (even additionally) to ease the lookup.

At the moment the current model is not designed good for those queries

Thanks
Pavel

Pavel_Duchovny · November 22, 2022, 8:03am

Hi @Stuart_S ,

Here is a sample idea for doing the join with current structure

db.sched_info.aggregate([{
 $lookup: {
  from: 'schedList',
  'let': {
   sched_ip: '$cluster_info.scheduler_ip'
  },
  pipeline: [
   {
    $addFields: {
     keys: {
      $objectToArray: '$active_schedulers'
     }
    }
   },
   {
    $unwind: {
     path: '$keys'
    }
   },
   {
    $match: {
     $expr: {
      $eq: [
       '$keys.k',
       '$$sched_ip'
      ]
     }
    }
   },
   {
    $project: {
     keys: 0
    }
   }
  ],
  as: 'schedList'
 }
}])

Thanks
Pavel

Stuart_S · November 22, 2022, 8:52am

Actually this is the output im getting
Empty array in the joined field “schedList”

[
  {
    "_id": ObjectId("63739f0de6b9aae9c681aeba"),
    "cluster_info": {
      "attached_clients": {
        "bng-emake-9a": {
          "IP": "10.223.37.24",
          "MaxJobs": 20,
          "NoRemote": true,
          "Speed": 79.484589
        }
      },
      "attached_daemons": {
        "bng-ea-agt-3a": {
          "IP": "10.223.36.42",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 90.203011
        },
        "bng-ea-agt-3b": {
          "IP": "10.223.36.43",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 55.782074
        },
        "bng-ea-agt-7a": {
          "IP": "10.223.36.62",
          "MaxJobs": 18,
          "NoRemote": false,
          "Speed": 89.654556
        },
        "bng-ea-agt-7b": {
          "IP": "10.223.36.63",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 87.926308
        },
        "bng-ea-agt-7c": {
          "IP": "10.223.36.64",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 89.026802
        },
        "bng-ea-agt-7d": {
          "IP": "10.223.36.65",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 89.416687
        }
      },
      "cluster_nodes_info": {
        "active": 0,
        "available_clients": 1,
        "available_daemons": 6,
        "client_and_daemon": 7
      },
      "daemon_cpu_available_and_free_info": {
        "active": 0,
        "free": 158,
        "local": 0,
        "pending": 0,
        "total_cpu": 158
      },
      "description": {
        "analysis": "free cpu available",
        "health": true
      },
      "dump_time": "Tue Nov 15 19:44:02 2022",
      "scheduler_ip": "bng-ea-agt-7a"
    },
    "schedList": []
  },
  {
    "_id": ObjectId("63739f72e6b9aae9c681aebb"),
    "cluster_info": {
      "attached_daemons": {
        "qnc-ea-agt-175b": {
          "IP": "10.44.138.82",
          "MaxJobs": 28,
          "NoRemote": false,
          "Speed": 0
        }
      },
      "cluster_nodes_info": {
        "active": 0,
        "available_clients": 0,
        "available_daemons": 1,
        "client_and_daemon": 1
      },
      "daemon_cpu_available_and_free_info": {
        "active": 0,
        "free": 28,
        "local": 0,
        "pending": 0,
        "total_cpu": 28
      },
      "description": {
        "analysis": "free cpu available",
        "health": true
      },
      "dump_time": "Tue Nov 15 19:45:53 2022",
      "scheduler_ip": "qnc-ea-agt-175a"
    },
    "schedList": []
  },
  {
    "_id": ObjectId("63739fd6e6b9aae9c681aebc"),
    "cluster_info": {
      "attached_clients": {
        "bng-ea-agt-76a": {
          "IP": "10.224.152.170",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 46.928242
        },
        "bng-ea-agt-76b": {
          "IP": "10.224.152.171",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 48.899769
        },
        "bng-ea-agt-76c": {
          "IP": "10.224.152.172",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 53.941841
        },
        "bng-ea-agt-76d": {
          "IP": "10.224.152.173",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 48.147972
        },
        "bng-ea-agt-77a": {
          "IP": "10.224.152.174",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 48.398701
        },
        "bng-ea-agt-77b": {
          "IP": "10.224.152.175",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 45.801868
        },
        "bng-ea-agt-77c": {
          "IP": "10.224.152.176",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 43.399036
        },
        "bng-ea-agt-77d": {
          "IP": "10.224.152.177",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 43.504852
        },
        "bng-ea-agt-78a": {
          "IP": "10.224.152.178",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 46.324295
        },
        "bng-ea-agt-78b": {
          "IP": "10.224.152.179",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 48.12402
        },
        "bng-ea-agt-78c": {
          "IP": "10.224.152.180",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 43.496323
        },
        "bng-ea-agt-78d": {
          "IP": "10.224.152.181",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 37.699486
        },
        "bng-ea-agt-79a": {
          "IP": "10.224.152.182",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-79b": {
          "IP": "10.224.152.183",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-79c": {
          "IP": "10.224.152.184",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-79d": {
          "IP": "10.224.152.185",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-80a": {
          "IP": "10.224.152.186",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-80b": {
          "IP": "10.224.152.187",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-80c": {
          "IP": "10.224.152.188",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        },
        "bng-ea-agt-80d": {
          "IP": "10.224.152.189",
          "MaxJobs": 40,
          "NoRemote": true,
          "Speed": 0
        }
      },
      "attached_daemons": {
        "bng-ea-agt-65d": {
          "IP": "10.224.152.117",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.715843
        },
        "bng-ea-agt-66a": {
          "IP": "10.224.152.118",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.045591
        },
        "bng-ea-agt-66b": {
          "IP": "10.224.152.119",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.129501
        },
        "bng-ea-agt-66c": {
          "IP": "10.224.152.120",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.980595
        },
        "bng-ea-agt-66d": {
          "IP": "10.224.152.121",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.966152
        },
        "bng-ea-agt-67a": {
          "IP": "10.224.152.122",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.545408
        },
        "bng-ea-agt-67b": {
          "IP": "10.224.152.123",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.567656
        },
        "bng-ea-agt-67c": {
          "IP": "10.224.152.124",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.602665
        },
        "bng-ea-agt-67d": {
          "IP": "10.224.152.125",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.978537
        },
        "bng-ea-agt-68a": {
          "IP": "10.224.152.126",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.515978
        },
        "bng-ea-agt-68b": {
          "IP": "10.224.152.127",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.063932
        },
        "bng-ea-agt-68c": {
          "IP": "10.224.152.128",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.261913
        },
        "bng-ea-agt-68d": {
          "IP": "10.224.152.129",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.822884
        },
        "bng-ea-agt-69a": {
          "IP": "10.224.152.130",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.858036
        },
        "bng-ea-agt-69b": {
          "IP": "10.224.152.131",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.856615
        },
        "bng-ea-agt-69c": {
          "IP": "10.224.152.132",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.345226
        },
        "bng-ea-agt-69d": {
          "IP": "10.224.152.133",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.255632
        },
        "bng-ea-agt-70a": {
          "IP": "10.224.152.134",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.546576
        },
        "bng-ea-agt-70b": {
          "IP": "10.224.152.135",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.122202
        },
        "bng-ea-agt-70c": {
          "IP": "10.224.152.136",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.60383
        },
        "bng-ea-agt-70d": {
          "IP": "10.224.152.137",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.3407
        },
        "bng-ea-agt-71a": {
          "IP": "10.224.152.150",
          "MaxJobs": 38,
          "NoRemote": false,
          "Speed": 27.695211
        },
        "bng-ea-agt-72a": {
          "IP": "10.224.152.154",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.563095
        },
        "bng-ea-agt-72b": {
          "IP": "10.224.152.155",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.593485
        },
        "bng-ea-agt-72c": {
          "IP": "10.224.152.156",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.327879
        },
        "bng-ea-agt-72d": {
          "IP": "10.224.152.157",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.512634
        },
        "bng-ea-agt-73a": {
          "IP": "10.224.152.158",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.542143
        },
        "bng-ea-agt-73b": {
          "IP": "10.224.152.159",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.394144
        },
        "bng-ea-agt-73c": {
          "IP": "10.224.152.160",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.713577
        },
        "bng-ea-agt-73d": {
          "IP": "10.224.152.161",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.258692
        },
        "bng-ea-agt-74a": {
          "IP": "10.224.152.162",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.941082
        },
        "bng-ea-agt-74b": {
          "IP": "10.224.152.163",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.259108
        },
        "bng-ea-agt-74c": {
          "IP": "10.224.152.164",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.658989
        },
        "bng-ea-agt-74d": {
          "IP": "10.224.152.165",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.375834
        },
        "bng-ea-agt-75a": {
          "IP": "10.224.152.166",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.81979
        },
        "bng-ea-agt-75b": {
          "IP": "10.224.152.167",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 27.066696
        },
        "bng-ea-agt-75c": {
          "IP": "10.224.152.168",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 25.778721
        },
        "bng-ea-agt-75d": {
          "IP": "10.224.152.169",
          "MaxJobs": 48,
          "NoRemote": false,
          "Speed": 26.476954
        }
      },
      "cluster_nodes_info": {
        "active": 50,
        "available_clients": 20,
        "available_daemons": 38,
        "client_and_daemon": 58
      },
      "daemon_cpu_available_and_free_info": {
        "active": 1674,
        "free": 140,
        "local": 50,
        "pending": 0,
        "total_cpu": 1814
      },
      "description": {
        "analysis": "free cpu available",
        "health": true
      },
      "dump_time": "Tue Nov 15 19:49:02 2022",
      "scheduler_ip": "bng-ea-agt-71a"
    },
    "schedList": []
  }
]

Pavel_Duchovny · November 22, 2022, 10:41am

@Stuart_S ,

Are you running the query on a MongoDB server? What version? Don’t test on mongo playground or other emulators.

I am getting results on 6.0 version.

Thanks
Pavel

Stuart_S · November 23, 2022, 3:02am

@Pavel_Duchovny
Ya at last i am able to join but why is the entire active scheduler joining (2 records) instead of just one matched record. The match seems to be not working in your screenshot too right?

Pavel_Duchovny · November 23, 2022, 6:16am

Hi @Stuart_S

What do you mean 2 records?

In your screen there is only one object under schedList array so its 1 record.
.
All the fields are projected beside temporary “keys”.

Thanks
Pavel

Stuart_S · November 23, 2022, 6:55am

hi @Pavel_Duchovny ,
I mean in the matched array schedList we noticed both the schedulers are present . Shouldnt it match with scheduler_ip from sched_info so only one object should be under active schedulers(the one that matches with the sched_info)

Pavel_Duchovny · November 23, 2022, 9:28am

No.

It brings the document that was matched. On the application side you can filter out the others.

Stuart_S · November 23, 2022, 10:51am

ya @Pavel_Duchovny ,

Thanks i got how to do that .

Now my last thing is , i need to match daemon_ip from sched_info to active_daemons from sched_list collection . And again active daemons are not a single entity? how can i get another array like daemon_list (similar to schedList that we joined) . Should i join them or rather just bring the whole active_daemons list and then filter them out on application side

Pavel_Duchovny · November 24, 2022, 6:16am

@Stuart_S ,

If that in a different query you can change the keys array to act on the “active_deamons” array:

$addFields: {
     keys: {
      $objectToArray: '$active_deamons'
     }
    }

Then use it to join to the relevant field in the deamons_list.

Now to be honest the queries this way are far from sufficient.

You should consider changing the data model to store this data together maybe , or at least do 2 queries :

Get the info and build a $in clause
Send the reference list as predicates to the “joined” data

Thanks

system · November 29, 2022, 10:04am

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.