Struggling to sync about 511 documents to the client app using flexible sync

Greetings, I’ve recently started developing a flutter app which needs to be able to work offline and sync data and figured I’d try realm but I am struggling to sync about 511 documents to the client app from the server.

Below error constantly shows up on App Service Logs.

connection(ac-xfugm72-shard-00-02.bke6qnv.mesh.mongodb.net:30444[-15871]) incomplete read of message header: context canceled; connection(ac-xfugm72-shard-00-02.bke6qnv.mesh.mongodb.net:30444[-15871]) incomplete read of message header: context canceled (ProtocolErrorCode=201)

Hi, all of those errors are transient and will actually no longer be logged in our next release. The log indicates that the client terminated the sync connection. This can happen for a variety of reasons including networking issues, closing the application, it transitioning to the background, etc.

What other issues are you seeing/is there any bad behavior here or is it just the error messages worrying you?

Everything seems to be fine in the logs outside of these errors (which I admit are confusing).

Another thing I noticed is that you have an M0, which should be fine for nowm but we recommend trying to test with at least an M10 wen testing anything performance related.

Thanks,
Tyler

If I update the subscription query to only sync student collection that has a particular payment method for example I can see the students being synced to the client app (This syncs down a few documents compared to all the students of course ) but if the subscription query is for all the students which is about 511 documents, It doesn’t sync any students. Below is all it shows on the IDE logs and nothing ever happens. I left it overnight thinking that maybe it syncs in the background but nothing.

I/flutter ( 3327): [INFO] Realm: Connection[1]: Session[1]: client_reset_config = false, Realm exists = true, client reset = false

I/flutter ( 3327): [INFO] Realm: Connected to endpoint ‘3.9.230.118:443’ (from ‘192.168.232.2:41213’)

I/flutter ( 3327): [INFO] Realm: Verifying server SSL certificate using 155 root certificates

I/flutter ( 3327): [INFO] Realm: Connection[1]: Connected to app services with request id: “6432fec6d270bb39fb5a7e27”

Hi, I am taking a deeper look into this and I think the error is just that you are on an M0 which is not very powerful and has a lot of rate limiting occuring. Can you possibly upgrade to an M10 and try again to see if the error still exists?

Please note that upgrading from the Shared Tier to the Dedicated Tier involves terminating sync first.

Thanks,
Tyler

I’ve upgraded to M10 and the following are errors i see on the IDE logs

I/flutter (10087): [ERROR] Realm: Failed to connect to endpoint ‘35.177.238.136:443’: Connection refused

I/flutter (10087): [ERROR] Realm: Failed to connect to ‘ws.eu-west-2.aws.realm.mongodb.com:443’: All endpoints failed

I/flutter (10087): [INFO] Realm: Connection[4]: Closing the websocket with status=‘SystemError: Connection refused’, was_clean=‘false’

I/flutter (10087): [ERROR] Realm: SyncWebSocketError message: WebSocket: Connection Failed category: SyncErrorCategory.webSocket code: SyncWebSocketErrorCode.websocketConnectionClosedClient

I/flutter (10087): [ERROR] Realm: Failed to connect to endpoint ‘35.177.238.136:443’: Connection refused

I/flutter (10087): [ERROR] Realm: Failed to connect to ‘ws.eu-west-2.aws.realm.mongodb.com:443’: All endpoints failed

I/flutter (10087): [INFO] Realm: Connection[4]: Closing the websocket with status=‘SystemError: Connection refused’, was_clean=‘false’

I/flutter (10087): [ERROR] Realm: SyncWebSocketError message: WebSocket: Connection Failed category: SyncErrorCategory.webSocket code: SyncWebSocketErrorCode.websocketConnectionClosedClient
/flutter (10087): [ERROR] Realm: Failed to connect to endpoint ‘35.177.238.136:443’: Connection refused

I/flutter (10087): [ERROR] Realm: Failed to connect to ‘ws.eu-west-2.aws.realm.mongodb.com:443’: All endpoints failed

I/flutter (10087): [INFO] Realm: Connection[4]: Closing the websocket with status=‘SystemError: Connection refused’, was_clean=‘false’

I/flutter (10087): [ERROR] Realm: SyncWebSocketError message: WebSocket: Connection Failed category: SyncErrorCategory.webSocket code: SyncWebSocketErrorCode.websocketConnectionClosedClient

Hi, I believe those errors are transient and due to a deploy doing on. It seems like your app is now back to connecting but running into a different issue. Do you know how large these objects are? We seem to have some indications in the logs that they are all on the order of 3 megabytes and that is what is causing the problem. Going to toggle a setting for you that should hopefully unblock you.

I am not sure how big each document is but It has a base64 image in it which shouldn’t be that big

Got it. Our metrics show that the average document size is 3MB. We applied a setting and think we have confirmed you should be working now. Let me know if you are still seeing issues.

Thanks,
Tyler

It is syncing now. I can see the documents come up on the client. just the to confirm, the issue was a setting on your side that needed to be changed ?

This is a setting on our side. We also just pulled in some work to prevent this from happening for anyone else. The TLDR is that the initial bootstrap was generating too large of entries due to the large average size of documents you have.

1 Like