I am trying to filter data using aggregation pipeline. In spark mongo connector v3.0, it is working and in V10, the filter is not applied and all documents from collection is returned
v3.0 code: (filter is working)
spark.read.format("mongo").option("uri",_uri).option("database",_database) \
.option("collection", _collection).option("pipeline","{$match:{flag:0}}").load()
v10 code : (not working)
spark.read.format("mongodb").option("spark.mongodb.connection.uri",_uri) \
.option("spark.mongodb.database",_database) \
.option("spark.mongodb.collection", _collection)\
.option("pipeline","{$match:{flag:0}}")\
.load()
Can you anyone please help me on this?