Mongodb Function: _id (ObjectId) missing in query aggregate results

[Josh_Whitehouse]

I’m having an issue with db.collection.aggregate in a MongoDB function where the results returned to me is missing the _id, it’s just blank. It’s a simple aggregate, with a $lookup, $match, and a $project, but anything I’ve tried to actually output the _id in the results is failing.

here is a simple example of the schema and query I’m doing this with…

a_collection: 

{
    _id : ObjectId(‘63f381b50ee158b55cc82b1a’),
	a_name : ‘This is an example’,
	b_tags: [ 
		ObjectId(‘640624f7dace963b6d2865c3’),
		ObjectId(‘640624f7dace963b6d2865c4’),
		ObjectId(‘640624f7dace963b6d2865c5’),
	]
}

b_collection:
   
{
    {
        _id: ObjectId(‘640624f7dace963b6d2865c3’),
        b_tag : 'This',
     },
     {
        _id: ObjectId(‘640624f7dace963b6d2865c4’),
        b_tag : 'demo',
     },
     {
        _id: ObjectId(‘640624f7dace963b6d2865c5’),
        b_tag : 'only',
     },

}

My function is doing an aggregate to search in the joined table for a string passed in as a parameter.

var searchStr = 'demo';` //  example string for matching

tagSearchQueryResult = await a_collection.aggregate([ 
      {
        // begin pipeline for  
        $lookup: {
          "from": "b_collection",
          "localField": "b_tags",
          "foreignField": "b_tags._id",
          "as": "b_tags",
          "pipeline": [
            { 
              "$addFields": { "b_tags": "$b_tags.b_tag" }
            }
         ]
        },
      },
      {
        $match: { 
          $expr: { "$in": [ searchStr, "$b_tag" ] }
        },
      },
      {
        "$project": {
        "id": "$_id" ,
        "a_name": 1,
      }
    }
    ]);
return { tagSearchQueryResult };

I do get back results as I expect, but without the _id of the document returned from the aggregate. I’m stumped why. I originally had foreignField: "_id" for the $lookup, changed it to foreignField: "_b_tags._id" thinking there was a conflict, but this did not fix the issue.

The results I’m returned look like this…

result (JavaScript): 
EJSON.parse('{"tagSearchQueryResult":[{"_id”:{},”a_name”:”This is an example","id":{}}]}')

I’ve read and researched, found nothing that tells me what’s the cause.

Thanks.

[John_Sewell]

I tried to run this on a local collection but it came up with some issues, can you put it in mongo playgound(https://www.mongoplayground.net/) to give a working example?

[Josh_Whitehouse]

Hi John,

the issue seemed to be with the double quotes when I formatted the code here. I corrected it and put it in Mongo Playground for you. It seems to work differently in Mongo Playground vs. the results I get running it inside an Atlas Function. Here I do see the _id field, but it’s not coming back in the Atlas Function.

Mongo playground

[Josh_Whitehouse]

Also, the $addFields should overwrite the existing one in the collection, but it doesn’t seem to.

[John_Sewell]

The pipeline is running on the joined collection so it does not find anything that matches that path, so the pipeline is running on data in “inventory” which does not have “b_tags.b_tag”, but it does have “b_tag”

[Josh_Whitehouse]

ok, but when I run this in an Atlas Function, I’m still losing the _id in the results.
EJSON.parse(‘{“atlasSearchQueryResult”:[{“_id”:{},“event_name”:“demo”}]}’)

[Josh_Whitehouse]

It’s working fine in the playground, only when being run in the Atlas Function is it failing.

I’m converting the aggregate with a toArray() method.

[John_Sewell]

That’s really weird, I just tried it on Atlas and this is my Atlas function:

exports = async function(arg){
  // This default function will get a value and find a document in MongoDB
  // To see plenty more examples of what you can do with functions see: 
  // https://www.mongodb.com/docs/atlas/app-services/functions/

  // Find the name of the MongoDB service you want to use (see "Linked Data Sources" tab)
  var serviceName = "mongodb-atlas";

  // Update these to reflect your db/collection
  var dbName = "Lookup";
  var collName = "orders";

  // Get a collection from the context
  var collection = context.services.get(serviceName).db(dbName).collection(collName);

  var results = await collection.aggregate([
  {
    "$lookup": {
      "from": "inventory",
      "localField": "b_tags",
      "foreignField": "_id",
      "as": "b_tags",
      "pipeline": [
        {
          "$addFields": {
            "b_tags": "$b_tags.b_tag"
          }
        }
      ]
    }
  },
  {
    $match: {
      $expr: {
        "$in": [
          "demo",
          "$b_tags.b_tag"
        ]
      }
    }
  },
  {
    "$project": {
      "id": "$_id",
      "a_name": 1,
      "b_tags": 1
    }
  }  
]).toArray();

return results

};

And this is the output of running it:

[{
	"_id": {
		"$oid": "63f381b50ee158b55cc82b1a"
	},
	"a_name": "This is an example",
	"b_tags": [{
		"_id": {
			"$oid": "640624f7dace963b6d2865c3"
		},
		"b_tag": "This"
	}, {
		"_id": {
			"$oid": "640624f7dace963b6d2865c4"
		},
		"b_tag": "demo"
	}, {
		"_id": {
			"$oid": "640624f7dace963b6d2865c5"
		},
		"b_tag": "only"
	}],
	"id": {
		"$oid": "63f381b50ee158b55cc82b1a"
	}
}]

[Josh_Whitehouse]

ok, i found the weird. in my function i’m calling aggregate on the same collection before I call this aggregate.
That aggregate is using $search and works fine. If I only run one or the other in the same function they work fine, it’s just when I run one after the other I get the “JSON.parse('{“tagSearchQueryResult”:[{”_id”:{},”a_name”:”This is an example",“id”:{}}]}')" results, with the empty _id field.

[Josh_Whitehouse]

For now, i’ll break this down into two separate Atlas Functions, but I’m curious to know why running aggregate in the same collection one after the other does result in the _id field missing.

[Josh_Whitehouse]

Turns out I need to do a Javascript deep copy when putting the results from the to aggregates together.