MongoDB Atlas replication lag occasional spikes to 10-15 seconds

Hi,

I have a mongoDB atlas cluster deployed in AWS. It is a non-sharded, replica set cluster with the following distribution:

  1. Frankfurt has the primary with 1 replica
  2. Ireland has 1 replica (also AWS)

The ireland ones replica lag goes upto 10-15 seconds every few hours for a short amount of time according to the metrics that I can see in MongoDB Atlas’s monitoring tab for the database. While Ireland is far, it isn’t that far and most of the day the replication is fine, there are no large number of writes happening during that time and I can see no other metric spiking around the same time including network, disk, CPU, memory etc.

Having the replication lag spiking to that large an amount periodically is sending alarm bells ringing in my head, any suggestions?