AWS Atlas Data Transfer Usage

MongoDB Atlas
1

I have about 50KB of application data in an atlas cluster. However, the daily AWS data transfer is a few GB every day. How do I figure out why a few KiloBytes is resulting in a few GigaBytes of data transfer every single day ?

2 Likes
2

Hey @Ashish_Sahni

I was wondering this very same thing when I upgraded my M2 cluster to M10. Plus the some 30 connections.

The reason I received was the intercommunication between your nodes which make up your cluster. They are in ‘constant’ contact with each other. Plus Atlas provides monitoring of your cluster which also required data transfer. At the end of the month I believe it was around 2% of the total bill.
The documentation mentions this about data transfer

The vast majority of Atlas customers spend less than 10% of their budget on data transfer.

Hope this helps.

3

Hi @Ashish_Sahni,

There are 3 types of data transfer:

  • Same (AWS) region
  • Different (AWS) region
  • Internet

Knowing which one of these types you are referring to would help us understand your usage.

The " Atlas AWS Data Transfer (Same Region) " is for data transfers and traffic between cluster nodes and data transferred between an application located in the same region as your cluster.

The traffic between the nodes in your Atlas cluster includes heartbeats and data replication between nodes.

The heartbeats occur regularly whenever the cluster is running, but are small in size. Replication involves propagating changes (writes, deletions, index builds, etc.) from the Primary to the Secondaries.

Backup type and schedule also affects data transfer. Please refer to this documentation.

Kind regards,
Martin

1 Like
4

@Natac13 Thanks, I was told the same thing but the math still does not make sense to me.

1 Like
5

The bulk of the data transfer usage (>99%) is for the ‘Same Region’.

A sample for a 72 hour period
02/12/20 Atlas AWS Data Transfer (Same Region) (N. Virginia) 7.477 GB

Since, this is a dev server that was just created and is not used frequently ie, application data stored in mongodb is not changing very often - I’m not how 50KB of application data could result in 7+ GB of data transfer usage.

Any ideas ?

1 Like
6

All replica set members exchange heartbeats with each other. Any driver that connects will also communicate to each member to maintain correct state of the replica set. That’s before any writes come to the server. Any write will be replicated to each of the secondaries.

Isn’t 7GBs something like 7 cents? Is your worry that this will be a lot higher in production?

1 Like
7

@Asya_Kamsky @Martin
My worry is I don’t understand the math. Why is it so difficult for MongoDB to explain the math ?

3 Likes
8

Hi Aasish, a typical MongoDB replica set in MongoDB Atlas will push approximately 6.2 to 6.8 GB per day in the form of the heartbeats between replicas that Asya mentioned (which in turn do cost $0.06 to $0.07 per day in in-region data transfer).

Note that the heartbeats enable the distributed system to maintain high availability and quorum in order to enable auto-failover. This enables continuous read and write availability in the event of loss of a node or availability zone. MongoDB leverages the RAFT distributed consensus algorithm.

4 Likes