Hi, I am currently using mongodb v10.1.1 connector to read some tables from my mongo collection. (source)
However I am able to read some collections and some collections are giving errors.
Example (1)
batch_df1 = spark.read.format ("mongodb").option("spark.mongodb.connection.uri", "mongodb+srv://xxx: xxx@xxx/admin?readPreference=secondaryPreferred&retryWrites=true&w=majority")\
.option('spark.mongodb.database', 'db1') \
.option('spark.mongodb.collection' , 'collection1') \
.load()
Example (2)
batch_df2 = spark.read.format ("mongodb").option("spark.mongodb.connection.uri", "mongodb+srv://xxx: xxx@xxx/admin?readPreference=secondaryPreferred&retryWrites=true&w=majority")\
.option('spark.mongodb.database', 'db1') \
.option('spark.mongodb.collection' , 'collection2') \
.load()
Example (1) works fine, but example (2) gives an error:
Error in callback <bound method UserNamespaceCommandHook.post_command_execute of <dbruntime. DatasetInfo.UserNamespaceCommandHook obiect at 0x7fb94374e250>> (for post_execute):
How can I troubleshoot this?