Docs Menu

Docs HomeMongoDB Spark Connector

Write Configuration Options

On this page

  • Write Configuration
  • connection.uri Configuration Setting

The following options for writing to MongoDB are available:

Note

If you use SparkConf to set the connector's write configurations, prefix spark.mongodb.write. to each property.

Property name
Description
mongoClientFactory
MongoClientFactory configuration key.
You can specify a custom implementation which must implement the com.mongodb.spark.sql.connector.connection.MongoClientFactory interface.

Default: com.mongodb.spark.sql.connector.connection.DefaultMongoClientFactory
connection.uri
Required.
The connection string configuration key.

Default: mongodb://localhost:27017/
database
Required.
The database name configuration.
collection
Required.
The collection name configuration.
maxBatchSize
Specifies the maximum number of operations to batch in bulk operations.

Default: 512
ordered
Specifies whether to perform ordered bulk operations.

Default: true
operationType
Specifies the type of write operation to perform. You can set this to one of the following values:
  • insert: insert the data.

  • replace: replace an existing document that matches the idFieldList value with the new data. If no match exists, the value of upsertDocument indicates whether or not the connector inserts a new document.

  • update: update an existing document that matches the idFieldList value with the new data. If no match exists, the value of upsertDocument indicates whether or not the connector inserts a new document.


Default: replace
idFieldList
Field or list of fields by which to split the collection data. To specify more than one field, separate them using a comma as shown in the following example:
"fieldName1,fieldName2"
Default: _id
writeConcern.w
Specifies w, a write concern option to acknowledge the level to which the change propagated in the MongoDB replica set. You can specify one of the following values:
  • MAJORITY

  • W1, W2, or W3

  • ACKNOWLEDGED

  • UNACKNOWLEDGED


For more information on w values, see the MongoDB server guide on the WriteConcern w option.

Default: ACKNOWLEDGED
writeConcern.journal
Specifies j, a write concern option to enable request for acknowledgment that the data is confirmed on on-disk journal for the criteria specified in the w option. You can specify either true or false.

For more information on j values, see the MongoDB server guide on the WriteConcern j option.
writeConcern.wTimeoutMS
Specifies wTimeoutMS, a write concern option to return an error when a write operation exceeds the number of milliseconds. If you use this optional setting, you must specify a non-negative integer.

For more information on wTimeoutMS values, see the MongoDB server guide on the WriteConcern wtimeout option.
upsertDocument
When true, replace and update operations will insert the data if no match exists.

For time series collections, you must set upsertDocument to false.

Default: true

You can set all Write Configuration via the write connection.uri.

Note

If you use SparkConf to set the connector's write configurations, prefix spark.mongodb.write. to the setting.

spark.mongodb.write.connection.uri=mongodb://127.0.0.1/test.myCollection

The configuration corresponds to the following separate configuration settings:

spark.mongodb.write.connection.uri=mongodb://127.0.0.1/
spark.mongodb.write.database=test
spark.mongodb.write.collection=myCollection

If you specify a setting both in the connection.uri and in a separate configuration, the connection.uri setting overrides the separate setting. For example, given the following configuration, the database for the connection is foobar:

spark.mongodb.write.connection.uri=mongodb://127.0.0.1/foobar
spark.mongodb.write.database=bar
←  Configuration OptionsRead Configuration Options →
Share Feedback
© 2023 MongoDB, Inc.

About

  • Careers
  • Investor Relations
  • Legal Notices
  • Privacy Notices
  • Security Information
  • Trust Center
© 2023 MongoDB, Inc.