Is there "update"(NOT "replace") as operation type of the result from MongoSinkConnector?

Working with Data Connectors & Integrations
#1

Hi!

I am doing sink from Postgres to MongoDB by MongoSinkConnector with Kafka. (Postgres → Kafka → MongoDB)
I can check there is u(= update) in op(= operationType) when I consume Kafka messages produced by Postgres from update operation.

However, all the operation types from updating Postgres’s records are not update but replace in op log of MongoDB.
Is replace default operation type when updating document by MongoSinkConnector?
If so, Is there any option to change it to update?

Thank you.

#2

Replace is actually an upsert so it is intended to update the document if the _id matches else it create a new document if it does not. You can change the replace/upsert behavior using Write Strategies. You can chose the _id via DocumentIdAddr Configuration

2 Likes
#3

Thank you for your kind reply!!

Then, do you mean that I can change operation type from REPLACE to UPDATE by using Write Strategies? I cannot find the strategy for it when I read the docs. ;(

As you said:

  • When _id matches: Replace
  • When _id doesn’t match: Create
    for now,

but I want to know how I can set the operation type like below:

  • When _id matches: Update
  • When _id doesn’t match: Create
#4

Currently there is not a way to do an update the way you are describing, it would be a replace. I added https://jira.mongodb.org/browse/KAFKA-255.

2 Likes