Filter bson.M or bson.D in Find

Working with Data Drivers & ODMs
#1

Hello, I met a strange case, when I try to Find doc by the ID (here the id is not objectID, it is a string), I implement the Find in two ways:

err := coll.FindOne(context.TODO(), bson.M{"_id": id}).Decode(result)

can pass all of the local test but fail on gitlab CI. Failed to find the result.

err := coll.FindOne(context.TODO(), bson.D{{"_id", id}}).Decode(result)

can pass all of the local test and the test on gitlab CI.

my local DB version, 3.6.17, the version of db used on gitlab CI testing, 3.6.11.

Thanks,

James

#2

These should be generating the same query. Are there any configuration parameters for your database in CI that could be affecting this? For example, the number of nodes in the cluster, the read/write concern used, the read preference used, etc?

#3

Hello Divjot,

Thank you for the support. A little more information but not sure if it is useful. I tried three test cases: one I use bson.M {"_id" : id} in a FindOne, where the id is a string, the other use bson.D{{"_id", id}} in a FindOneAndUpdate, where the id is a string, the last one use bson.M{"_id", id} in a FindOne, where the id is objectID, all can work locally but the bson.M with string id will always fail on C, while bson.D and bson.M with objectID will always work on CI.

After I change bson.M with string id to bson.D with string id, in the FindOne, the CI can successfully run this case, I try a few times and it looks very “stable” pass. Any comments?

Thanks,

James

#4

Hi. I’ve found a similar case that doesn’t seem to work.

// Try to find all projects in list with mongo-driver/mongo
var results []bson.M
cursor, err := ProjectCollection.Find(ctx,
    bson.D{{
        Key: "_id",
        Value: bson.D{{Key: "$in", Value: obj.ProjectIDs}},
    }},
)
cursor.All(ctx, &results)
fmt.Printf("id strings: %v\n", obj.ProjectIDs)
fmt.Printf("id types: %T\n", obj.ProjectIDs) // showing type of project ids list
fmt.Printf("results: %v\n", results)

id strings: ["idhexstring1", "idhexstring2", ...]
id types: []string
results []

//This works in Atlas in db.projects collection
{"_id": {"$in": [ObjectId("idhexstring1"), ObjectId("idhexstring2"), ...]} }

[{document1...},{document2...},...]

For the mongo-driver/mongo code, changing to bson.M does not change the result. Changing the project id list to be of type []primitive.ObjectId yields an error that interface{} is string, not primitive.ObjectID. Can someone confirm that the “$in” search does in fact work with db.Collection.Find() and possibly provide an example?

Edit: See @Divjot_Arora suggestion for the answer to my question

#5

Hi @Eric_Solomon,

Per the logs you gave, obj.ProjectIDs is a slice of strings, but it seems like the data in the server is of type ObjectID, so the filter won’t match any documents. You’re right that obj.ProjectIDs should be of type []primitive.ObjectID. The error you provided (interface{} is string...) seems like a compilation error, not a driver error, suggesting that you’re trying to put strings into a slice of ObjectIDs. If you have a bunch of hex strings that represent object IDs, you can convert them using this code:

strings := []string{"idhexstring1", "idhexstring2", ...}
objectIDs := make([]primitive.ObjectID, 0, len(strings)

for _, str := range strings {
    oid, err := primitive.ObjectIDFromHex(str)
    if err != nil {
        // handle error
    }

    objectIDs = append(objectIDs, oid)
}

From there, you can use the new objectIDs slice in your filter. This is a little verbose, so ideally you could restructure your application to make sure the obj.ProjectIDs array is always filled with ObjectIDs from the start. The primitive.ObjectID type has an UnmarshalJSON function that will handle converting hex strings to ObjectIDs, so if you’re getting these string values from a JSON source, you might be able to leverage that.

– Divjot

#6

Thank you @Divjot_Arora! I had previously tried converting the strings to primitive.ObjectIDs in the way that you described. Doing this a second time caused me to realize that this is in fact correct and that my bug was further down in my collection-processing function during another conversion.