__realm_sync history taking up too much storage

Hi!
I created an app following the O-Fish app mongo blog post.
In this app the image is first attached to the document and when the document is synced it is removed.
This post is using partitionned sync, the only thing I did different is using the flexible sync.

My problem is with the history collection in the __realm_sync database created by Mongo. This collection keeps track of every change in each document of my database.
The problem here is that since it tracks each change it stores a document for the brief instant when the document is synced with the image embedded as base64 and it is taking a lot of storage.

I was wondering if there was something I could do to prevent this. I already implemented trimming but I’m using a shared cluster and don’t have that much storage space.

Is it possible to only delete some part of the history? My photo document is always synced with the base64 image embedded which is deleted directly by a trigger. In the end I only care about the photo document without the embedded image. Like if my document goes from A => B => C, is it possible to only keep A => C ?

1 Like

Hello! What you’re describing is known as “compaction” and exists only for partition-based sync. This is because, for flexible sync, trimming is far more powerful - as it will remove all history entries that are older than the Max Offline Time.

You have three options here:

  1. increase your cluster size to support the larger history
  2. don’t write base64-encoded images to realm. Instead, upload them to s3 or something similar, and write the URLs to realm isntead
  3. Set your Max offline time to a lower value to have trimming kick in more aggressively
1 Like

Hi @Sudarshan_Muralidhar

Thank you for your reply.
The second option is what I’m doing, but as mentionned in the MongoDB blog post for the O-Fish app I’m first attaching the image to the document and when the document is synced I’m uploading the image to S3 and deleting the image from the document. Just like the Mongo team in the blog post.

Deleting history is great but my app is used during short “burst” and the history cannot be trimmed fast enough.
Is there no way to compact the history even in flexible sync?
Or to select which documents to keep in the history table?
As mentionned since the image is uploaded to S3 I don’t need to keep a history document specifying that my image was attached to the document.

Best regards
Renaud Aubert

Unfortunately, Flexible Sync does not support compaction today, as we instead delete all history that is older than your set “Max Offline Time”. I’ll take this feedback to the team though.

Sudarshan

1 Like