Create Index and Merge to create a change history for document schema

Working with Data
, , ,
1

I have a collection of transaction documents uploaded weekly. Its necessary for me to track changes to the fields/schema, and with the generous help of StackOverflow and user nimrod serok I have an aggregation pipeline which can produce a document of changes.

Following what I’ve learned from the Aggregation course M121, I now wish to ‘merge for Rollup’ these results.

I created a new index:

coll = mongo_conn['reports']['myreport_change_history']
coll.create_index([
    ('all_keys', pymongo.DESCENDING),
    ('my_unique_id', pymongo.DESCENDING)
])

[Strangely, pymongo has this string output when I create the change_history collection: all_keys_-1_goblin_unique_id_-1', as if the field names were saved with _-1 appended to the name.]

I used Mongo Shell to log into my mongodb server to discover this step did create a new collection:

> show collections
__schema__
myreport
myreport_change_history

Then I added to the end of the successful aggregation pipeline the merge stage:

coll = mongo_conn['reports']['myreport']
result = coll.aggregate([
    [...]
    {'$merge': {
        'into': 'myreport_change_history',
        'on': ['all_keys', 'my_unique_id']
        }
    }

This produces an error:

OperationFailure: Cannot find index to verify that join fields will be unique, full error: {'ok': 0.0, 'errmsg': 'Cannot find index to verify that join fields will be unique', 'code': 51183, 'codeName': 'Location51183'}

But I just created the index. What’s going on?
I also tried the merge stage with the mysterious _-1 appended to each. No luck. Finally, both of these new fields in the index will be unique as part of the result of the aggregation pipeline. No issue with the data, so it’s just a matter of syntax or …?

2

hey lately even im facing the same issue are you able to resolve this?