> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

# Release Notes

Learn what's new in:

- [Version 11.1](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-11.1)

- [Version 11.0](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-11.0)

- [Version 10.7](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.7)

- [Version 10.6](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.6)

- [Version 10.5](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.5)

- [Version 10.4](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.4)

- [Version 10.3](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.3)

- [Version 10.2](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.2)

- [Version 10.1.1](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.1.1)

- [Version 10.1.0](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.1.0)

- [Version 10.0](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-10.0.0)

## MongoDB Connector for Spark 11.1

The 11.1 connector release includes the following changes and new features:

- Adds the `change.stream.micro.batch.max.rows` read configuration property, which limits the number of change stream events consumed in each micro-batch. When you set this option, the connector limits micro-batches by event count rather than by elapsed time between batches. Rate limiting requires the `change.stream.micro.batch.max.partition.count` configuration property to be set to `1` and write access to the sidecar collection in the source database.

- Adds the `change.stream.micro.batch.max.rows.offset.collection` read configuration property, which specifies the sidecar collection used to persist resume tokens when the `change.stream.micro.batch.max.rows` property is set. Defaults to `__spark_resume_tokens`.

## MongoDB Connector for Spark 11.0

The 11.0 connector release adds support for Spark 4.0 and later. Spark 4.0 requires Scala 2.13. To learn more about Spark 4.0, see the [Spark documentation.](https://spark.apache.org/releases/spark-release-4-0-0.html)

## MongoDB Connector for Spark 10.7

The 10.7 connector release includes the following changes and new features:

**Warning: Breaking Change**

If you are using Spark 3.1.x, you must upgrade to Apache Spark 3.2 or later to use Spark Connector version 10.7. The Spark Connector now only supports Apache Spark versions 3.2 through 3.5.

- Adds the `change.stream.micro.batch.max.rows` and `change.stream.micro.batch.max.rows.offset.collection` read configuration properties introduced in v11.1. These properties enable rate limiting for change stream micro-batches. To learn more, see the [v11.1 release notes.](https://www.mongodb.com/docs/spark-connector/current/release-notes.md#std-label-version-11.1)

## MongoDB Connector for Spark 10.6

The 10.6 connector release includes the following changes and new features:

- Adds support for the `fullDocumentBeforeChange` option in change stream read configurations, which allows you to specify whether to include the full document before the change event in the change stream output. To learn more about this option, see the [Change Stream Configuration](https://www.mongodb.com/docs/spark-connector/current/streaming-mode/streaming-read-config.md#std-label-change-stream-conf) section of the Streaming Read Configuration Options guide.

- Adds the `truncateMode` write configuration option, which specifies how to truncate a collection when performing an overwrite. To learn more about this option, see the [Streaming Write Configuration Options](https://www.mongodb.com/docs/spark-connector/current/streaming-mode/streaming-write-config.md#std-label-spark-streaming-write-conf) or the [Batch Write Configuration Options](https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-write-config.md#std-label-spark-batch-write-conf) guide for your relevant mode.

- Adds the `ignoreDuplicatesOnInsert` write configuration option, which determines whether to ignore duplicate key errors when performing unordered insert operations. You can use this option in both batch and streaming write configurations. To learn more about this option, see the [Batch Write Configuration Options](https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-write-config.md#std-label-spark-batch-write-conf) and [Streaming Write Configuration Options](https://www.mongodb.com/docs/spark-connector/current/streaming-mode/streaming-write-config.md#std-label-spark-streaming-write-conf) guides.

## MongoDB Connector for Spark 10.5

The 10.5 connector release includes the following changes and new features:

- Adds support for using [Atlas Data Federation](https://www.mongodb.com/products/platform/atlas-data-federation) with the Spark Connector in the `SamplePartitioner`, `PaginateBySizePartitioner` and `AutoBucketPartitioner` batch reading configurations.

  For more information about batch configuration, see the [Batch Read Configuration Options](https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-read-config.md#std-label-spark-batch-read-conf) page.

- Changes the default batch read partitioner configuration to be `AutoBucketPartitioner`

## MongoDB Connector for Spark 10.4

The 10.4 connector release includes the following changes and new features:

- Updates the Java Sync Driver version to v5.1.x.

## MongoDB Connector for Spark 10.3

The 10.3 connector release includes the following new features:

- Added support for Spark v3.3, v3.4, and v3.5.

- Added support for reading multiple collections when using micro-batch or continuous streaming modes.

  **Warning: Breaking Change**

  Support for reading multiple collections introduces the following breaking changes:

  - If the name of a collection used in your `collection` configuration option contains a comma, the Spark Connector treats it as two different collections. To avoid this, you must escape the comma by preceding it with a backslash (\\).

  - If the name of a collection used in your `collection` configuration option is "\*", the Spark Connector interprets it as a specification to scan all collections. To avoid this, you must escape the asterisk by preceding it with a backslash (\\).

  - If the name of a collection used in your `collection` configuration option contains a backslash (\\), the Spark Connector treats the backslash as an escape character, which might change how it interprets the value. To avoid this, you must escape the backslash by preceding it with another backslash.

  To learn more about scanning multiple collections, see the [collection configuration property](https://www.mongodb.com/docs/spark-connector/current/streaming-mode/streaming-read-config.md#std-label-spark-streaming-input-conf) description.

## MongoDB Connector for Spark 10.2

The 10.2 connector release includes the following new features:

- Added the `ignoreNullValues` write-configuration property, which enables you to control whether the connector ignores null values. In previous versions, the connector always wrote `null` values to MongoDB.

- Added options for the `convertJson` write-configuration property.

- Added the `change.stream.micro.batch.max.partition.count` read-configuration property, which allows you to divide micro-batches into multiple partitions for parallel processing.

- Improved change stream schema inference when using the `change.stream.publish.full.document.only` read-configuration property.

- Added the `change.stream.startup.mode` read-configuration property, which specifies how the connector processes change events when no offset is available.

- Support for adding a comment to operations.

## MongoDB Connector for Spark 10.1.1

- Corrected a bug in which aggregations including the `$collStats` pipeline stage did not return a count field for Time Series collections.

## MongoDB Connector for Spark 10.1.0

- Support for Scala 2.13.

- Support for micro-batch mode with Spark Structured Streaming.

- Support for BSON data types.

- Improved partitioner support for empty collections.

- Option to disable automatic upsert on write operations.

- Improved schema inference for empty arrays.

- Support for null values in arrays and lists. The Connector now writes these values to MongoDB instead of throwing an exception.

See [this post](https://www.mongodb.com/blog/post/introducing-mongodb-spark-connector-version-10-1) on the MongoDB blog for more information.

## MongoDB Connector for Spark 10.0.0

- Support for Spark Structured Streaming.