Connector Message Processing Properties
Overview
Use the settings on this page to configure the message processing behavior of the sink connector including the following:
- Message batch size
- Rate limits
- Number of parallel tasks
For a list of sink connector configuration settings organized by category, see the guide on Sink Connector Configuration Properties.
Settings
Name | Description | ||
|---|---|---|---|
max.batch.size | Type: int Description: Maximum number of sink records to batch together for processing. Consider the batch that contains the following records: When set to 0, the connector performs a single bulk write for
the entire batch.When set to 1, the connector performs one bulk write for each
record in the batch, for a total of five bulk writes as shown in
the following example:Default: 0Accepted Values: An integer | ||
bulk.write.ordered | Type: boolean Description: Whether the connector writes a batch of records
as an ordered or unordered bulk write operation.
When set to true, the default value, the
connector writes a batch of records as an ordered bulk write
operation.To learn more about ordered and unordered
bulk write operations, see
Bulk Write Operations
in the MongoDB Java driver documentation. Tip Performance Benefits If you can write your data to MongoDB in any order, use unordered bulk write operations. Unordered bulk writes provide the following benefits:
To learn more about the performance benefits of unordered bulk write operations, see Unordered Operations in the MongoDB Manual. Default: trueAccepted Values: true or false | ||
rate.limiting.every.n | Type: int Description: Number of batches of records the sink connector processes in
order to trigger the rate limiting timeout. A value of 0 means no
rate limiting. Default: 0Accepted Values: An integer | ||
rate.limiting.timeout | Type: int Description: How long (in milliseconds) to wait before the sink connector
should resume processing after reaching the rate limiting
threshold. Default: 0Accepted Values: An integer | ||
tasks.max | Type: int Description: The maximum number of tasks to create for this connector. The
connector may create fewer than the maximum tasks specified if it
cannot handle the level of parallelism you specify. Important Multiple Tasks May Process Messages Out of Order If you specify a value greater than Default: 1Accepted Values: An integer |