Docs Home → MongoDB Spark Connector
Write Configuration Options
Write Configuration
The following options for writing to MongoDB are available:
Note
If you use SparkConf to set the connector's write configurations,
prefix spark.mongodb.write. to each property.
Property name | Description | |
|---|---|---|
mongoClientFactory | MongoClientFactory configuration key. You can specify a custom implementation which must implement the
com.mongodb.spark.sql.connector.connection.MongoClientFactory
interface.Default: com.mongodb.spark.sql.connector.connection.DefaultMongoClientFactory | |
connection.uri | Required. The connection string configuration key. Default: mongodb://localhost:27017/ | |
database | Required. The database name configuration. | |
collection | Required. The collection name configuration. | |
maxBatchSize | Specifies the maximum number of operations to batch in bulk
operations. Default: 512 | |
ordered | Specifies whether to perform ordered bulk operations. Default: true | |
operationType | Specifies the type of write operation to perform. You can set
this to one of the following values:
Default: replace | |
idFieldList | Field or list of fields by which to split the collection data. To
specify more than one field, separate them using a comma as shown
in the following example: Default: _id | |
writeConcern.w | Specifies w, a write concern option to acknowledge the level to
which the change propagated in the MongoDB replica set. You can
specify one of the following values:
For more information on w values, see the MongoDB server guide on
the WriteConcern w option.Default: ACKNOWLEDGED | |
writeConcern.journal | Specifies j, a write concern option to enable request for
acknowledgment that the data is confirmed on on-disk journal for
the criteria specified in the w option. You can specify
either true or false.For more information on j values, see the MongoDB server
guide on the
WriteConcern j option. | |
writeConcern.wTimeoutMS | Specifies wTimeoutMS, a write concern option to return an error
when a write operation exceeds the number of milliseconds. If you
use this optional setting, you must specify a non-negative integer.For more information on wTimeoutMS values, see the MongoDB server
guide on the
WriteConcern wtimeout option. | |
upsertDocument | When true, replace and update operations will insert the data
if no match exists.For time series collections, you must set upsertDocument to
false.Default: true |
connection.uri Configuration Setting
You can set all Write Configuration via the write connection.uri.
Note
If you use SparkConf to set the connector's write configurations,
prefix spark.mongodb.write. to the setting.
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/test.myCollection
The configuration corresponds to the following separate configuration settings:
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/ spark.mongodb.write.database=test spark.mongodb.write.collection=myCollection
If you specify a setting both in the connection.uri and in a separate
configuration, the connection.uri setting overrides the separate
setting. For example, given the following configuration, the
database for the connection is foobar:
spark.mongodb.write.connection.uri=mongodb://127.0.0.1/foobar spark.mongodb.write.database=bar