MongoDB Atlas online archive getting stuck

MongoDB Atlas
1

Created a custom query online archiving rule.But the archiving gets stuck once it reaches 500MB.Even after pausing and resuming the process multiple times its showing no progress.Trying to archive about 4GB of data

3

Hey @Varun_Sumesh,

Welcome to the MongoDB Community Forums! :leaves:

Please contact the Atlas in-app chat support regarding this. They will be better able to help you out in this case.

Regards,
Satyam

4

Hi @Varun_Sumesh ,

Can you please let us know if this issue is still existing. If so, can you please clarify by “archiving gets stuck”, does the archive display an “error” status or does it show the “archiving” status.

Thanks !

5

The issue is still existing.The status is shown as archiving and the cpu utilisation goes up but there is no decrease in documents count in the collection we are archiving.Its stuck at 2.03 GB.

6

We’d like to help you further, but have a few clarifying questions to help diagnose:

  1. Can you clarify how much time it takes for it to archive 2.03GB of data
  2. Are there high cardinality string fields in the partitioned fields
  3. What are the data types of the partition fields
7

The time taken to archive 2.03GB is approximately 45 minutes.
We have two partition fields in the archive.One is a timestamp value in seconds which is an integer.It has high cardinality.The other is a random number user id which is stored as string,it also has high cardinality

8

Thanks for your response. We do not recommend providing high cardinality string fields as partition keys for Online Archive. This is mentioned in the documentation and we also request you to refer this blog: Optimizing your Online Archive for Query Performance | MongoDB

As part of our roadmap, we are changing the backend storage service of Online Archive in our next version and as part of this new release, we will allow the option to choose high cardinality string fields.

  1. Details about the Private Preview program are here: https://www.mongodb.com/community/forums/t/invitation-to-participate-in-the-early-access-program-of-online-archives-query-performance-improvements/204188
  2. If you’d like to participate in the program, please contact me at : prem.krishna@mongodb.com

If you’d like to test out without the Private Preview mode in your non-production environment, then you can delete the archive and create a new archive without high cardinality string field. Also you can “schedule” the archive to run during non-peak hours, so that way you are ensuring the CPU utilization doesn’t peak.