No diffs in messages from MongoDB Kafka source connector

Hello,

Faced the following behavior while using connector 1.6.1:
Need to publish messages in topic as a diffs, according to documentation it were a common connector use case some time ago. But now, according to docs, for this case the following parameter must be used:
"change.stream.full.document" : "default"

The default setting returns the differences between the original document and the updated document.

Created connector with following configuration:

{
  "name" : "msc-connector-test",
  "config" : {
    "batch.size" : "8192",
    "connection.uri" : "***",
    "connector.class" : "com.mongodb.kafka.connect.MongoSourceConnector",
    "copy.existing" : "false",
    "collection" : "***",
    "database" : "***",
    "heartbeat.interval.ms" : "5000",
    "heartbeat.topic.name" : "__msc_heartbeat-topic",
    "key.converter" : "org.apache.kafka.connect.storage.StringConverter",
    "key.converter.schemas.enable" : "false",
    "name" : "msc-connector-test",
    "output.json.formatter" : "com.mongodb.kafka.connect.source.json.formatter.SimplifiedJson",
    "change.stream.full.document" : "default",
    "topic.creation.default.partitions" : "3",
    "topic.creation.default.replication.factor" : "3",
    "transforms" : "dropPrefix",
    "transforms.dropPrefix.regex" : "(.*)<db_name>(.*)",
    "transforms.dropPrefix.replacement" : "msc-topic-test",
    "transforms.dropPrefix.type" : "org.apache.kafka.connect.transforms.RegexRouter",
    "value.converter" : "org.apache.kafka.connect.storage.StringConverter",
    "value.converter.schemas.enable" : "false"
  }
}

But published messages have following format:

{
  "_id" : {
    "_data" : "***"
  },
  "operationType" : "replace",
  "clusterTime" : {
    "$timestamp" : {
      "t" : 1646896094,
      "i" : 1
    }
  },
  "fullDocument" : {
    ***
  },
  "ns" : {
    "db" : "***",
    "coll" : "***"
  },
  "documentKey" : {
    "_id" : "***"
  }
}

There’s no diffs in messages, just an updated fullDocument.
Is this an expected behavior since 1.6 update or something wrong or missed in connector configuration?

Hi,

I have faced the same issue and I had to came up with the idea of EventCollection where backend is posting events and connector is just gathering insert operations. It’s a workaround for that lack of diff in messages for “update” operation. Ideally would be if I could relay on connector functionality.

Where you able to solve Your issue ?