Help using mongo-spark-2.2.x with Atlas

First of all, i really sorry if my question it is simply for this forum. I am new in machine learning community and I am heaving to figure solutions during my Phd’s project flows.

I am trying to integrate Spark with my MongoDb Cluster in Atlas and unfortunately, I was unable to do it until now.

I’ve cloned the github repository and run sbt check, but until now I did not realize where is the jars files or what I supposed to do.

I have already the jars files in .ivy2 directory from the haddop-mongoDb connector but it does not working too. Even I am trying to config during launching SparkSession.

If someone could help me i really thanks

It might be easier to just use the compiled Spark Connector that is already available in Maven.

config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.12:3.0.0")

Something like this works in Python -

from pyspark.sql import SparkSession

spark = SparkSession.\




config("spark.executor.memory", "1g").\



config("spark.jars.packages", "org.mongodb.spark:mongo-spark-connector_2.12:3.0.0").\


Here is an example of using the Spark Connector with MongoDB

1 Like