Docs Home → MongoDB Kafka Connector
Change Stream Properties
Overview
Use the following configuration settings to specify aggregation pipelines for change streams and read preferences for change stream cursors when working with the MongoDB Kafka source connector.
For a list of source connector configuration settings organized by category, see the guide on Source Connector Configuration Properties.
Settings
Name | Description | |
|---|---|---|
pipeline | Type: string Description: An array of aggregation pipelines to run in your change stream.
You must configure this setting for the change stream
event document, not the fullDocument field.ExampleTipDefault: []Accepted Values: Valid aggregation pipeline stage | |
change.stream.full.document | Type: string Description: Determines what values your change stream returns on update
operations. The default setting returns the differences between the original
document and the updated document. The updateLookup setting returns the differences between the
original document and updated document as well as a copy of the
entire updated document at a point in time after the update.The whenAvailable setting returns the updated document,
if available.The required setting returns the updated document and
raises an error if it is not available.TipFor more information on how this change stream option works, see the MongoDB server manual guide on Lookup Full Document for Update Operations. Default: ""Accepted Values: "", "updateLookup", "whenAvailable", or "required" | |
change.stream.full.document.before.change | Type: string Description: Configures the document pre-image your change stream returns on update
operations. The pre-image is not available for source records
published while copying existing data, and the pre-image configuration
has no effect on copying. TipTo learn how to configure a collection to enable pre-images, see the Server manual entry on pre- and post-images. The default setting suppresses the document pre-image. The whenAvailable setting returns the document pre-image if
it's available, before it was replaced, updated, or
deleted.The required setting returns the document pre-image and
raises an error if it is not available.Default: ""Accepted Values: "" or "whenAvailable" or "required" | |
publish.full.document.only | Type: boolean Description: Whether to return only the fullDocument field from the
change stream event document produced by any update event. The fullDocument field
contains the most current version of the updated document. To
learn more about the fullDocument field, see the Server
manual page on update events.When set to true, the connector overrides the
change.stream.full.document setting and sets it to
updateLookup so that the fullDocument field contains
updated documents.Default: falseAccepted Values: true or false | |
publish.full.document.only.tombstone.on.delete | Type: boolean Description: Whether to return tombstone events when documents are deleted.
Tombstone events contain the keys of deleted documents with
null values. This setting applies only when
publish.full.document.only is true.Default: falseAccepted Values: true or false | |
collation | Type: string Description: A JSON collation document
that specifies language-specific ordering rules that MongoDB
applies to the documents returned by the change stream. Default: ""Accepted Values: A valid collation JSON document | |
batch.size | Type: int Description: The change stream cursor batch size. Default: 0Accepted Values: An integer | |
poll.await.time.ms | Type: long Description: The amount of time in milliseconds to wait before checking the change
stream cursor for new results. Default: 5000Accepted Values: An integer | |
poll.max.batch.size | Type: int Description: Maximum number of documents to read in a single batch when polling
a change stream cursor for new data. You can use this setting to
limit the amount of data buffered internally in the connector. Default: 1000Accepted Values: An integer |