Mongo log issue

Working with Data Connectors & Integrations
#1

Hello,
i have replication from 3 nodes ( 1 primary and 2 secondry)
from 2 weeks i get this message and when i get this message the insert to db stopped

2022-05-23T22:46:03.329+0300 E - [conn8139] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:04.311+0300 E - [conn8140] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:05.293+0300 E - [conn8141] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:06.298+0300 E - [conn8142] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:07.318+0300 E - [conn8143] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:08.334+0300 E - [conn8144] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:09.302+0300 E - [conn8145] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102
2022-05-23T22:46:10.315+0300 E - [conn8146] Assertion: BSONObjectTooLarge: BSONObj size: 17064926 (0x10463DE) is invalid. Size must be between 0 and 16793600(16MB) First element: note: “all times in microseconds” src/mongo/bson/bsonobj.cpp 102

because of this message make big LAG in Kafka

how can solve this issue?

thanks

#2

There is change stream metadata that is included in the kafka topic message. When the document itself is large this extra metadata makes it go above the 16MB limit. As of now your only option is to reduce the size of the document. You could also enable writing errors to the DLQ on the sink so your connector will keep working if they run into these errors.