Change stream pipeline on nested fields

Hi,

I have a schema liket this in a ‘portal’ collection

{
  enterprise: {
    name: 'Company',
    street: 'foo',
    iban: '123456789',
    bic: '123456789'
  },
  salt: 'mskfjlzjrjzjreofskfns',
  hashPassword: 'pjsdfknsfjiejgfqkùgjlr',
}

To detect change on iban or bic field. I use this pipeline

    const pipeline = [
      {
        $match: {
          operationType: 'update',
          $or: [        
            { 'updateDescription.updatedFields.enterprise.bic': { $exists: true } },
            { 'updateDescription.updatedFields.enterprise.iban': { $exists: true } },
          ],
        },
      },
    ];

But it’s don’t work

I have test different strategy, but with no success:

    const pipeline = [
      {
        $match: {
          operationType: 'update',
          $or: [        
            { "updateDescription.updatedFields.enterprise.iban": { $exists: true } },
            { "updateDescription.updatedFields.'enterprise.iban'": { $exists: true } },
            { "updateDescription.updatedFields['enterprise.iban']": { $exists: true } },
            { "updateDescription.updatedFields.['enterprise.iban']": { $exists: true } },
          ],
        },
      },
    ];

If i test the same thing on a field with no dot notation it’s work

    const pipeline = [
      {
        $match: {
          operationType: 'update',
          $or: [        
            { 'updateDescription.updatedFields.salt': { $exists: true } },
            { 'updateDescription.updatedFields.hashPassword': { $exists: true } },
          ],
        },
      },
    ];

How to write or working pipeline on fields with dot notation ?

Thank in advance for your help

Read the following thread. At first sight it looks like it is the same issue.

Yes it’s the same issue.

The result is : It’s not possible. It’s not a bug, it’s a limitation.

I was told by a MongoDB staff member that it was a feature limitation, rather than a bug! I don’t have the level of paid service with Mongo for specific queries to be answered, but a sales rep chased it up for me and mentioned it wasn’t a bug.
I suspect still that it’s a bug but that was the information I was told.

I have not worked with change streams a lot but as for your problem i think it can be solved by using $getField operator which mongo recently launched just to cater use cases like this where the field names contains special characters like dot (.) and dollar ($).

Really thanks for your response it’s work

here the code :

    const bankFields = ['bank', 'iban', 'rib', 'bic'];
    const pipeline = [
      {
        $match: {
          operationType: 'update',
          $expr: {
            $or: bankFields.map((field: string) => ({
              $ne: [
                {
                  $type: {
                    $getField: {
                      field: `enterprise.${field}`,
                      input: '$updateDescription.updatedFields',
                    },
                  },
                },
                'missing',
              ],
            })),
          },
        },
      },
    ];

Too bad $getField doesn’t support the $ operator for arrays.

I have a new problem: I’m trying to detect a push in an array.

Here’s a basic sample schema:

{
  userIds: [
    '123',
    '456'
  ]
}

during push ‘789’ in userIds I receive
updateDescription.updatedFields: { 'userIds.3': '789'}

during remove ‘789’ in userIds I receive
updateDescription.updatedFields.userIds: ['123', '456']

Thise pipeline don’t work:

   const pipeline = [
      {
        $match: {
          operationType: 'update',
          $expr: {
            $ne: [
              {
                $type: {
                  $getField: {
                    field: 'userIds.$',
                    input: '$updateDescription.updatedFields',
                  },
                },
              },
              'missing',
            ],
          },
        },
      },
   }

if this can help other users, actually I use this pipeline

  • I don’t filter event in the $match, I filter received event in the changeStream watch function
  • I use a $project, to delete the case of remove an element of the array
const pipeline = [
      {
        $match: {
          operationType: 'update',
        },
      },
      {
        $project: {
          // with this projection, we can delete the case of removing an id of userIds
          'updateDescription.updatedFields.userIds': 0,
        },
      },
    ];
1 Like