Hello
I am using pyspark to connect with MongoDB database, however I get this error while running on a huge collection and I don’t know where to pass the parameter allowDiskUse
pipeline = [{'$unwind': {'path': 'list'}}]
df=df.option("pipeline", pipeline).load()
Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true