Hello guys,
according to the docs from debezium source it supports different capture modes
in particular capture.mode: change_streams and change_streams_update_full (first one does not contain full document on update and the latter one has full update per each update).
We would like to use capture.mode change_streams because our documents can be megabytes in size so it’s not efficient to send full document per one field update to Kafka topic.
According to the source code of debezium cdc , it uses an “after” field which is only available for capture.mode change_streams_update_full thus we can’t leverage partial updates one.
Is there any plan to support change_streams mode in cdc handler for debezium? Or are there any reasons why the field “after” is used and partial updates are ignored?
We are using debezium source because it connects to each shard’s replica set separately and can be parallelized by using tasks.max property, while default mongo one only opens change stream and can’t be parallelized.
Thanks