I have a collection that has mixed types of _id fields. Some documents have strings while some documents have ObjectId. When I load the data using spark connector, by default I only see non ObjectId documents. TO see the ObjectId records I have to specifically use a pipeline { ‘_id’ : {‘$type’: ‘objectId’} }. I am not able to find a way to query all the documents.
Is there a known solution to this problem.