CappedPositionLost

Hi,

Our Prod Mongo is a running on version 3.6.0. We currently have a synchronisation service that is reading from the change stream to update index in Elastic search using monstache (GitHub - rwynn/monstache: a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.). After we made some update on our data we got this:

ERROR 2021/05/03 16:00:59 Error starting change stream. Will retry: CappedPositionLost: CollectionScan died due to position in capped collection being deleted.

And now I am not able to enable the synchronisation again.

Do you know a way I can enable the synchronisation process?

Thank you in advance

Hi @Jonatan_Aponte and welcome in the MongoDB Community :muscle: !

First time I hear about Monstache so I don’t know anything about it. But looks like they are pulling data from change streams and copying them into Elastic.

CappedPositionLost sounds like Monstache was stopped or had a temporary incident and couldn’t sync for a bit. When it tried to restart the change stream were it stopped earlier, the last entry they processed couldn’t be found, probably because your Oplog window is too small and had already rolled over that last processed entry.

Can you run db.getReplicationInfo() and see how large your oplog is? You need to make sure that your timeDiff is large enough to cover any incident you could have. Else your Elastic will be desync from your MongoDB collections and you will need to recreate your Elastic indexes from scratch as you can’t recover at this point, as the history of the write operations has already rolled over some operations that you couldn’t sync.

I don’t know if you are on Community, Enterprise Advanced or Atlas, but if you are on Atlas, I would recommend that you have a look at Atlas Search instead, because the sync is done automatically for you.
With this solution, you don’t need to run 3 nodes for MongoDB and 3 nodes for Elastic and x2 the price of your infra & (potentially) licence costs. Atlas Search comes out of the box with an Atlas cluster and doesn’t need extra infra or licensing and the sync is done automatically, which is less troubles…

Finally, I would recommend that you upgrade to MongoDB 4.4 as MongoDB 3.6 has now reached EOL last month and isn’t supported anymore. MongoDB Support Policies | MongoDB
This is also a trivial thing to do in Atlas.

I hope this helps.
Cheers,
Maxime.

1 Like