Our customers have been occasionally experiencing a bug where Realm databases become subtly corrupted after a database compaction operation on startup. Our app keeps two encrypted databases open simultaneously, and IIRC, in every log I’ve seen where this happens, they are both being compacted. When this happens, attempts to write new data occasionally fail with an decryption error. (I plan to file a bug with more detailed information once we are able to reliably reproduce this.)
We recently upgraded from realm-swift 10.30 to 10.46 in the hopes that one of the recent fixes would resolve this issue, but unfortunately it persists. Our next step to try to work around this is to disable compaction on startup and rely on Realm’s automatic compaction logic introduced in realm-swift 10.35 to prevent files from getting problematically large. However, I don’t have a good mental model of how the automatic compaction mechanism works, and I worry that, if it’s executing the same codepath to compact the database, the bug will still be just as likely to manifest.
So for now, my question is: From a high level, how does the automatic compaction mechanism work? Is it the same as the compaction-on-startup mechanism, where the old file is read completely and a new file is written from scratch? Or is there something more sophisticated happening?
I believe compaction (automatic) only takes place when the file is not being accessed. Are you saying that compaction is happening while the app is in a write transaction? Or are you manually compacting and during that process (via shouldCompactOnLaunch()) , there’s a write transaction?
I would think that since compacting cannot occur if another process is accessing the realm, there wouldn’t be a write in either case.
Interesting bug which could have other implications if it can be duplicated.
We have been using shouldCompactOnLaunch to control compacting on startup. The behaviour we’re consistently seeing is this:
The app launches
The app opens encrypted Realm #1 (generally fairly small), which it decides to compact.
The app opens encrypted Realm #2 (often quite large), which it decides to compact.
Some time later (usually not very long afterwards), the app attempts to perform a database write to encrypted Realm #2 that fails with a “Decryption failed” exception. This is often, but not always, uncatchable. Once we reach this state, this happens consistently every time a write to the same table is attempted. Not all tables are affected.
Prior to the dual-compaction scenario, writes to that table happened without complaint.
This strongly suggests to me that something is happening during compaction that is corrupting the newly-written database file. My understanding is that compaction-on-startup involves reading the entire database and writing it to a new file, so there’s not many other places corruption could have snuck in - presumably, if the problem was already present at that point, Realm would have complained when decrypting the data from the old file.
I was really hopeful that this was the bug, since it was specifically something that happened when dealing with multiple encrypted Realm files, but upgrading hasn’t solved it.
My testing seems to confirm that the online compaction functionality doesn’t seem to trigger the bug, so hopefully removing shouldCompactOnLaunch will work around this for now. I’ll sleep better once the root cause is found, though.