Mapping from C# driver to MongoSH query problem within Aggregation Project stage

Hello all !
Not sure about the right place to post my problem, but I’m pretty nooby in Mongo DB, and didn’t found how to repport Issues (sorry)…

Working on a project with MongoDb, I’m facing an issue while using Projection in my Aggregation.

I have the following piece of code in my function, which is supposed to Project some data :

    var query = RepositoryCollection.Aggregate()
                        .Project(p => new QueryProductWithSingleMovement
                        {
                            Id = p.Id,
                            CreationDate = p.CreationDate,
                            CreationOperatorId = p.CreationOperatorId,
                            CreationSiteId = p.CreationSiteId,
                            DestructionDate = p.DestructionDate,
                            ProductNumber = p.ProductNumber,
                            ProductNumberId = p.ProductNumberId,
                            ProductNumberReleaseVersion = p.ProductNumberReleaseVersion,
                            SerialNumber = p.SerialNumber,
                            Movement = p.Movements.First(m => m.Id == p.LastMovementId) 
                        });
    var result = await query.ToListAsync();

So here the MongoSH query that is supposed to be generated should look like :

    db.myCollection.aggregate([
            {
                "$project": {
                    "Id": "$_id",
                    "CreationDate": "$creationDate",
                    "CreationOperatorId": "$creationOperatorId",
                    "CreationSiteId": "$creationSiteId",
                    "DestructionDate": "$destructionDate",
                    "ProductNumber": "$pnNumber",
                    "ProductNumberId": "$pnId",
                    "ProductNumberReleaseVersion": "$pnReleaseVersion",
                    "SerialNumber": "$serialNumber",
                    "Movement": {
                        "$arrayElemAt": [
                            {
                                "$filter": {
                                    "input": "$movements",
                                    "as": "m",
                                    "cond": {
                                        "$eq": [
                                            "$$m._id",
                                            "$lastMovementId"
                                        ]
                                    }
                                }
                            },
                            0
                        ]
                    },
                    "_id": 0
                }
            }
        ])

But instead I get the following result :

    db.myCollection.aggregate([
            {
                "$project": {
                    "Id": "$_id",
                    "CreationDate": "$creationDate",
                    "CreationOperatorId": "$creationOperatorId",
                    "CreationSiteId": "$creationSiteId",
                    "DestructionDate": "$destructionDate",
                    "ProductNumber": "$pnNumber",
                    "ProductNumberId": "$pnId",
                    "ProductNumberReleaseVersion": "$pnReleaseVersion",
                    "SerialNumber": "$serialNumber",
                    "Movement": {
                        "$arrayElemAt": [
                            {
                                "$filter": {
                                    "input": "$movements",
                                    "as": "m",
                                    "cond": {
                                        "$eq": [
                                            "$$m._id",
                                            "$$m.lastMovementId"
                                        ]
                                    }
                                }
                            },
                            0
                        ]
                    },
                    "_id": 0
                }
            }
        ])

The difference is quite tricky to find out but it is located inside the “Movement” property in the “$filter” condition. Actually in the “$eq” expression we are supposed to compatre the Movement._id value with the Product.LastMovementId value (which are both ObjectId), but instead having

    "$eq": [
        $$m._id",
        $lastMovementId"
    ]

we have

    "$eq": [
        $$m._id",
        $$m.lastMovementId"
    ]

For more clarty, here are my models :
Product.cs

    public class ProductItemBase 
        {
            /// <summary>
            /// Gets or sets the identifier.
            /// </summary>
            [BsonId]
            [BsonRepresentation(BsonType.ObjectId)]
            public ObjectId Id { get; set; } = default!;

            /// <summary>
            /// Gets or sets the product number identifier.
            /// </summary>
            [BsonElement("pnId")]
            public int ProductNumberId { get; set; }

            /// <summary>
            /// Gets or sets the product number Number value.
            /// </summary>
            [BsonElement("pnNumber")]
            public string ProductNumber { get; set; } = default!;

            /// <summary>
            /// Gets or sets the creation date.
            /// </summary>
            [BsonElement("creationDate")]
            public DateTime CreationDate { get; set; }

            /// <summary>
            /// Gets or sets the creation site identifier.
            /// </summary>
            [BsonElement("creationSiteId")]
            public int CreationSiteId { get; set; }

            /// <summary>
            /// Gets or sets the creation operator identifier.
            /// </summary>
            [BsonElement("creationOperatorId")]
            public int CreationOperatorId { get; set; }

            /// <summary>
            /// Gets or sets the product number release version.
            /// </summary>
            [BsonElement("pnReleaseVersion")]
            public string ProductNumberReleaseVersion { get; set; } = default!;

            /// <summary>
            /// Gets or sets the serial number.
            /// </summary>
            [BsonElement("serialNumber")]
            public string SerialNumber { get; set; } = default!;

            /// <summary>
            /// Gets or sets the destruction date.
            /// </summary>
            [BsonElement("destructionDate")]
            public DateTime? DestructionDate { get; set; }

            /// <summary>
            /// Gets or sets the last movement identifier.
            /// </summary>
            [BsonElement("lastMovementId")]
            public ObjectId LastMovementId { get; set; }

            /// <summary>
            /// Gets or sets the last movement identifier.
            /// </summary>
            [BsonElement("lastOperationId")]
            public ObjectId? LastOperationtId { get; set; }

            /// <summary>
            /// Gets or sets the movements.
            /// </summary>
            [BsonElement("movements")]
            public List<MovementItem>? Movements { get; set; }
        }

Movement.cs

    public class MovementItem : CollectionItemBase
        {
            /// <summary>
            /// Gets or sets the identifier.
            /// </summary>
            [BsonId]
            [BsonRepresentation(BsonType.ObjectId)]
            public ObjectId Id { get; set; } = default!;

            /// <summary>
            /// Gets or sets the product identifier.
            /// </summary>
            [BsonElement("productId")]
            public ObjectId? ProductId { get; set; }

            /// <summary>
            /// Gets or sets the type of the movement.
            /// </summary>
            [JsonConverter(typeof(StringEnumConverter))]        // JSON.Net
            [BsonRepresentation(BsonType.String)]               // Mongo
            [BsonElement("movementType")]
            public MovementType MovementType { get; set; }

            /// <summary>
            /// Gets or sets the movement date.
            /// </summary>
            [BsonElement("movementDate")]
            public DateTime MovementDate { get; set; }
        }

So if anyone of you have ever met this issue, or can tell me how to report it (if it is not the right place), I’ll appreciate it a lot .

Hi @Roman_Lushkin,

If I understand the issue this appears to be a known defect which has been reported as CSHARP-3524.

The LINQ interface has been completely redesigned and these types of issues may no longer manifest once 2.14.0 is released. If you would like to test this there is a beta version of the driver available.

Let us know if the issue continues to reproduce with the latest driver.

Alex

Hello @alexbevi,

Thank you for your quick support.
Unfortunately the issue still persists even with the 2.14.0-beta1 version.
The problem remains exactly the same.

Roman.

@Roman_Lushkin my apologies, but to use the new LINQ3 LINQ provider some additional configuration is needed.

For example:

var connectionString = "mongodb://localhost";
var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
clientSettings.LinqProvider = LinqProvider.V3;
var client = new MongoClient(clientSettings);

@alexbevi, is there any other packages dependency I should add (with the new version) ?
I have Initilized the Client as indicated but…

The generated query is still incorrect, and now I also have the following exception :

Cannot find serializer for value(ProductRepository+<>c__DisplayClass3_0).
 at MongoDB.Driver.Linq.Linq3Implementation.Serializers.KnownSerializers.KnownSerializersRegistry.GetSerializer(Expression expression, IBsonSerializer defaultSerializer)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ConstantExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, ConstantExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.BinaryExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, BinaryExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateUsingAggregationOperators(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.Translate(TranslationContext context, Expression expression, Boolean exprOk)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionTranslators.AndExpressionToFilterTranslator.Translate(TranslationContext context, BinaryExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateUsingQueryOperators(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.Translate(TranslationContext context, Expression expression, Boolean exprOk)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionTranslators.OrExpressionToFilterTranslator.Translate(TranslationContext context, BinaryExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateUsingQueryOperators(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.Translate(TranslationContext context, Expression expression, Boolean exprOk)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateLambda(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer)
   at MongoDB.Driver.Linq.Linq3Implementation.LinqProviderAdapterV3.TranslateExpressionToFilter[TDocument](Expression`1 expression, IBsonSerializer`1 documentSerializer, IBsonSerializerRegistry serializerRegistry)
   at MongoDB.Driver.ExpressionFilterDefinition`1.Render(IBsonSerializer`1 documentSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.PipelineStageDefinitionBuilder.<>c__DisplayClass27_0`1.<Match>b__0(IBsonSerializer`1 s, IBsonSerializerRegistry sr, LinqProvider linqProvider)
   at MongoDB.Driver.DelegatedPipelineStageDefinition`2.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.MongoCollectionImpl`1.<AggregateAsync>d__23`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at MongoDB.Driver.MongoCollectionImpl`1.<UsingImplicitSessionAsync>d__107`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at MongoDB.Driver.IAsyncCursorSourceExtensions.<ToListAsync>d__16`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Madic.Magtrack.DataAccess.MongoDb.Repositories.ProductRepository.<GetInventory>d__3.MoveNext()

This happens when I’m calling the query.ToListAsync() extensions method.

@Roman_Lushkin my apologies for the delay, but I’ve been unable to reproduce this failure with the 2.14.0-beta1 driver.

using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using System;
using System.Collections.Generic;
using System.Linq;

namespace ConsoleApp3
{
    public class A
    {
        public C Item { get; set; }
    }

    public class B
    {
        [BsonId]
        [BsonRepresentation(BsonType.ObjectId)]
        public ObjectId Id { get; set; } = default!;

        [BsonElement("lastMovementId")]
        public ObjectId LastMovementId { get; set; }

        [BsonElement("movements")]
        public List<C>? Movements { get; set; }
    }

    public class C
    {
        [BsonId]
        [BsonRepresentation(BsonType.ObjectId)]
        public ObjectId Id { get; set; } = default!;
    }

    internal class Program
    {
        private static void Main(string[] args)
        {
            var connectionString = "mongodb://localhost";
            var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
            clientSettings.LinqProvider = LinqProvider.V3;
            var client = new MongoClient(clientSettings);
            var collection = client.GetDatabase("test").GetCollection<B>("foo");
            var query = collection.Aggregate()
                    .Project(p => new A
                    {
                        Item = p.Movements.First(m => m.Id == p.LastMovementId)
                    });
            var result = query.ToList();
        }
    }
}

The code above is what I’m working with. If you can adapt this and trigger the failure I can share the results with our .NET Drivers engineers as it’s likely a defect in the LINQ3 interface.

@Roman_Lushkin

I’m having a similar issue with Linq v3:

The following gives “Cannot find serializer for value(SharedModel.Statistics.IntBucketStore)”

(Note that I’d like to have “null” instead of “new IntBucketStore()”, but that gives me “Cannot find serializer for null.”)

(long select, only showing the line where it fails)
.
ResponseTimes = statisticsRequest.IncludeResponseTimes ? s.First().ResponseTimes : new IntBucketStore(),
.

IntBucketStore looks like this:

    [BsonIgnoreExtraElements]
    [ProtoContract]
    public class IntBucketStore
    {
        [BsonElement("buckets")]
        [BsonDictionaryOptions(DictionaryRepresentation.ArrayOfArrays)]
        [ProtoMember(1)]
        public SortedDictionary<int, int> Buckets { get; set; } = new SortedDictionary<int, int>();
        [BsonElement("bucketSize")]
        [ProtoMember(2)]
        public int BucketSize { get; set; }
        [ProtoMember(3)]
        public int MaxBuckets { get; set; }
        [ProtoMember(4)]
        public int MinValue { get; set; }
        [ProtoMember(5)]
        public bool AreBuckets {get;set;}

        public IntBucketStore(int buckets, int bucketSize, int minValue, bool startAsBuckets = true)
        {
            MaxBuckets = buckets;
            BucketSize = bucketSize;
            MinValue = minValue;
            AreBuckets = startAsBuckets;
        }

        public IntBucketStore()
        {
            BucketSize = 20;
            MaxBuckets = 40;
        }
    }

Forgot to also show how responseTimes is defined:

[BsonElement("responseTimes")]
public IntBucketStore ResponseTimes { get; set; }

@Bjorn_Andersson, @alexbevi : Hi guys !
Just want to inform you that the version 2.14.1 fixes the issue. Queries working as expected :+1:
This thread can be closed :slight_smile:

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.