Change stream is failing to connect

Working with Data Connectors & Integrations
, , ,
1

We have a producer application which gets data via changestream from mongodb and posts to kafka. But it started failing from last night with below error. What could be the cause?

MongoServerError: Entry field "o.version" should be string, found: long
    at MessageStream.messageHandler (/usr/src/app/node_modules/mongodb/lib/cmap/connection.js:467:30)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:475:12)
    at processIncomingData (/usr/src/app/node_modules/mongodb/lib/cmap/message_stream.js:108:16)
    at MessageStream._write (/usr/src/app/node_modules/mongodb/lib/cmap/message_stream.js:28:9)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:731:22)
    at Socket.emit (events.js:400:28)
    at Socket.emit (domain.js:475:12) {
  operationTime: new Timestamp({ t: 1682491411, i: 106 }),
  ok: 0,
  code: 40532,
  codeName: 'Location40532',
  '$clusterTime': {
    clusterTime: new Timestamp({ t: 1682491411, i: 108 }),
    signature: {
      hash: new Binary(Buffer.from("e72109e0cc06eed5230ee17b39ae1db873ae91fa", "hex"), 0),
      keyId: new Long("7186622876357755161")
3

Hello :wave: @Balu_Daggubati,

I noticed that you haven’t received a response yet. Are you still experiencing the issue?

Although, based on the error message it indicates that there is an issue with the “o.version” field in the data being processed by the producer application. The error specifically states that the “o.version” field should be a string, but it is found to be of type long.

  • Are you working with any of the internal collections such as oplog or system collection?
  • Could you confirm if there were any recent changes in the data that are getting published?
  • Could you also confirm if there is any schema validation you are doing or modifying the change stream output?

Best,
Kushagra

4

Hi Kushagra, Thanks for replying!
Yes, we are depending on oplog. We are saving it in our memory and reusing it incase of crashing to start fetching from the time where it was left. The issue got resolved after creating new that key and restarting the service.

We didn’t have any changes in the recent times. But all of sudden it started failing. We are using this as a producer to replicate the data and it will be used by the consumers via kafka. The data modification happens only in the consumer’s side but not at the producer.

Please check and let me understand on the issue.

Thanks,
Bala