Does MongoDB Realm actually store my data in a specific DB?

I have a Realm app that is linked to a M10 MongoDB cluster.
Data sync between clients works and there are no errors in the logs.
I am in development mode and have reinitialized sync several times. In earlier versions of the Realm SDKs, all user data could be found in the database I specified in the Sync settings under development mode. Now, I cannot access the data anymore through Compass or atlas.

Is this expected behavior? In Compass I see a __realm_sync database but that does not contain the data in a readable format. Also I noted, that if development mode was disabled there would be no place where I set an actual database* where data should be synced to.

This behavior confuses me, is there anything I´m overlooking?

I have a similar issue. This is new for me, since yesterday.
It is like the sync connection between Realm and the Database in Atlas is broken.
I have logged a support ticket - but not having much luck from support.
Clients seem to be syncing to Realm OK. You can see new Collections at the Realm level but even tho i have Dev mode on … there seems to be no prorogation of change or data between Realm Servers and Atlas.

after resetting up a new realm app with a new cluster etc it worked for me again - but my other app is still broken. If we don´t get an answer here I´ll file an issue to support as well with my app.

There is likely some clue in the logs, either on the client side or on the server side - can you please share the logs please?

@Ian_Ward i could not find many errors, there are no errors coming when I start a session, but instead normal start/end logs. However I can see these logs in the history, but I´m not sure they are related to the issue I´m seeing ( realms are synced but not apparent in my db):

Error: 

there are no mongodb/atlas services with sync enabled
Error.

Error:

the MongoDB-Realm translator is being shut down and will be restarted
Source:

Error syncing MongoDB write

Ian … i have shared the logs with support already …
Cloud: MongoDB Cloud.
i can send to you … if you tell me how you want them ?

thanks

Ian can you confirm . . that client data / schema is stored at the Realm Server before it is replicated to Atlas ?
and this data is used to service client requests without Atlas ?

As newly Client generated data and schema seems to be flowing up and down from the Realm Server to the Clients but is not seen at the Atlas level (in Dev mode btw)(i can see the new schema at the Realm Server). Consequently, it looks like connection between Realm Server and Atlas is not functioning.

@Damian_Raffell Thanks for the support link Damian - yeah I see some unexpected behavior on your cloud instance. I am reaching out to the operations team now, we will update your ticket shortly

Just to let you know …
I had the same issue : “there are no mongodb/atlas services with sync enabled”
For some reason this service stopped working, so the connection between Atlas and the Realm Server stopped working. Support found it eventually. No idea why it happened. It’s working again now … not sure what they did … they haven’t said yet. I had to terminate and restart Sync to flush some old meta data and delete an object from a Collection to trigger something and then data started replicating both ways.

1 Like

Thank you @Damian_Raffell. Since my app is still in development, taking the more radical step of recreating it worked for me. Next time I´ll have to approach the support. Tbh. I´m just very happy that this is not the expected behavior, meaning hat the data becomes invisible to MongoDB.

Hi Ian … did you find out what happened ? … support acknowledged a problem, it mysteriously started working again… but they never provided info on what the problem was or how i could deal with it.

It also looks like it has happened again to me !

@Damian_Raffell Are you wiping your local simulator/emulator every time you terminate/re-enable sync? If not, you need to be. Your issues appear to stem from schema errors, see error here:

{{error encoding document during initial sync: document is missing required field(s): [productid] }}.

I’d recommend fixing your schema to prevent these errors

@Ian_Ward speaking of that, is there a recommended approach how to handle client resets automatically ? I´m currently backing up and deleting the whole mongodb-realm folder on the iOS device which seems to work well for client resets.

@Christian_Huck - I am currently trying to migrate data from Realm Cloud and seem to have the same issue when creating objects in the new MongoDG Realm from the client. All the data gets created in the client realm file and is accessible from the client SDK but no data shows up in Atlas.

Where exactly are the client logs located and where are the sync logs to be found on the server ? I assume you are using an Atlas cloud instance as well. If I use a web browser and look at the logs in the Realm Apps logs screen I just see the following:

Is there some other location one can look for sync errors ?

Error:

could not decode next message: error reading body failed to read: read limited at 16777217 bytes

Logs:

[ “Connection was active for: 15s” ]

SDK:

darwin vRealmJS/10.1.1

Platform Version:

v12.20.0

Ah - there is an error - what does this mean ?

So you found an error, great. Unfortunately I cannot tell what it means, maybe @Ian_Ward can help here?
Do you see at least some of your synced data in the mongodb after all?

There is a 16MB limit of data per realm.write uploaded from the client.

1 Like

@Ian_Ward - I figured as much and so break the copy up into smaller chunks now - but still there were issues until I put a 10 second delay between realm.write() - even then there are a bunch of errors messages but it’s not clear what they mean. All the records counts seem to match up now but I have not checked that all the references or lists are correct yet.

If I try and use a 2 second delay then the sync fails to complete and not all the data gets synced to MongoDB Realm. And when trying to connect from a new client the call to open the synced realm fails to return at all.

A bit fragile relying on some client side timing for transactional updates to be reliable.

What error messages are you referring to?

You should not need to insert delays in between realm.write commands. It sounds like there might be a knowledge gap when using Javascript - it is an asynchronous and non-blocking language. Sync is performed asynchronously, those tasks are dispatched to the JS event loop, and the program continues executing - if it then reaches the end of the program, it exits, even if there are still dispatched sync tasks on the event loop. Those events will never be synced.

You need to design your app so that it uses Promises and Callbacks and chains your execution of logic so that the program does not exit before sync is complete

It seems pointless having this discussion. I have raised a support ticket and am willing to provide the realm file so you guys can run the script and test things yourselves to see why data is not loading properly.

Right now despite having delete the Realm App and the associated database from Atlas, Atlas is reporting that we have exceeded the 512MB storage limit. Creating a new Realm App with a new database does not allow data sync because the storage limit has been exceeded - even though there are no databases or collections in the Atlas cluster.

Perhaps we should come back in a few months time to see if things have improved.

Hello,

I have the same problem because of the 16MB limitation but I don’t know how to get around this. Can we temporarily remove this limit?

My application is still on Realm Cloud today and I am in the process of migrating to MongoDB. For this I have created an application in Swift which uploads all the data from the .realm file and when I run it, I have the error message : “could not decode next message: error reading body failed to read: read limited at 16777217 bytes”

How can I do ? Because I have to transfer all the data to MongoDb and of course the size of each realm files exceed 16MB.

Thanks for your help.
Regards,