Flexible Sync Questions:

Realm Team thanks for bringing Flexible Sync!

I have some questions to help plan development (knowing it is in Preview and not GA).

  1. Am I correct, in my assumption that eventually (i.e., once out of preview) Flexible Sync can be ran with Partition-Based Sync in the same Realm App?
  2. Do you have a GA date estimate? - or even when we can test with both Partition Sync & Flexible Sync in same Realm App.
  3. Will the limit of 10 Queryable Fields be changeable in future (or handled by a cost factor)? -to avoid using generic names like ‘queryField01’ across classes.
  4. Will the embedded object or current operators in RQL Limitations for Flexible Sync change by GA or soon after? -If so, can you disclose which are likely?
  5. With ‘superseded’ is there a suggested wait time to handle before acquiring query state again (assuming connected and all)?
  6. Out of curiosity, if a query is duplicated in a SubscriptionSet, which will prevail first or last written?

Document typo found:
I believe the words in the name are transposed for QuerySubscription (written as SubscriptionQuery) in first paragraph of API reference: QuerySubscription Structure Reference

Roop, welcome back! :smiley: - I told you we’d get there eventually!

  1. I am working on a design that should hopefully allow partition-based sync clients to automatically convert their requests to a flexible sync query. I feel like this should be obtainable because you could potentially think of a partition-key request as the simplest flexible sync query ever - ie. give me all documents on all sync collections where this field equals this value. I don’t have approval yet but I am pushing for it so stay tuned. At this moment a Realm Sync App is either partition-based sync or flexible sync.
  2. I hope to go GA this year - the more people use it, the more comfortable I will be in making this decision earlier :slight_smile:
  3. We can change this limit if needed but be advised that this will inflate your storage costs the higher you go - its not linear but should be considered. I would also think that if you need a ton of queryable fields that something is off in your schema design or query patterns and you should probably reconsider your architecture - happy to meet and discuss though
  4. Embedded objects and queries on Arrays for Flexible Sync are currently being worked on right now and will be there for GA. Anything else you are looking for?
  5. For superseded - does this help: https://github.com/realm/realm-core/blob/6fce382ef7e7c89c8bbfc88e48458620932e556f/src/realm/sync/subscriptions.hpp#L109
    basically it means that the server has finished bootstrapping a later set of subscriptions so that set of subscriptions that is Superseded has been replaced by a newer one.
  6. Generally, you should try to avoid this because there is no point - a duplicated query may make the server do more work, but should otherwise not change your results.

I hope this helps

-Ian

2 Likes

Hey Ian – yes you did - glad to see Flexible Sync! Hope all is well.

  1. That sounds interesting to use the partition key. I have some suggestions, if it is something you’d like to discuss LMK. That could be handy since anything that was QBS that I cannot convert to Flexible (for now) - would be Partition-Based via Permissions. For now, it sounds like one way or another we will be able to use then both together in a Realm App :clap:t4:. Is it safe to say by the first GA?
  2. That’s not so bad – thanks!
  3. OK. Would something like a declaration attribute on class properties (instead of defining queryable fields) give flexibility to developers yet help prevent storage bloat?
  4. Yeah, currently my queries use all the unsupported operators. If I had (had to had to) choose some they would be: [c], IN, beginsWith, contains, ANY.
  5. Thanks.
  6. Of course will avoid, just wanted to make sure last written prevails[?].

Thanks!

Hi, good thing is that most of the string operators will be implemented and released very soon (beginsWith, contains, [c], endsWith, like). The array operators like ANY will be followed up quickly after and we can ping this thread when they get released if that works. Out of curiosity, when you say you want to use IN, do you mind letting us know what your query is intended to look like? There are two different interpretations with IN depending on if the right-hand side of the query is a string or a list of elements you are trying to OR together.

1 Like

Hi Tyler - that is good news - thanks!

I have both usages for IN. Some use cases in play for comparing collections/aggregates:

"%@ IN %K._id" - Left side is a constant value (not collection) and right side for a Collection

"%K IN %@" - Left side is a constant value (not collection) and right side is a Swift-Type Collection (non Realm)

Hi Tyler - Ian mentioned we will be able to compare Arrays to Realm Collections. That is something new and great, is that far away?

Hey, I’m not sure I follow you 100%. We are planning on rolling out support for querying on Realm Collections using the ANY, ALL, NONE syntax in RQL within the next quarter hopefully. Is that what you are referencing?

I am referring to #4 that Ian responded to my initial questions - “queries on Arrays for Flexible Sync.” Thank you in advance!

Hey @Ian_Ward - hope you are well!

Do you have any new info (timeline, probability, etc.) for having one Realm App with both Partition-based & Flexible sync simultaneously?

  • I am in a holding position to determine a path - before I can move forward on some things.

Thank you in advance!

I’m not sure what your use case is but I what I was mentioning above was to have some sort of way to convert partition-based sync clients to a flexible sync configured on the cloud side. The cloud side would only be configured with flexible sync - we haven’t started this work yet, it is likely a few quarters out.

Hi @Ian_Ward,

Thanks for that and it’s understood. Though, my main focus is on the ability to have both (Partition & Flexible) in same App. If that is going to happen? -because if not, managing 2 Apps per user won’t be fun :grimacing:.

Currently, I have 3 main designs for Realm with MongoDB, that I have separated as follows:

  1. Partition-based with EmbeddedObjects. When using EmbeddedObjects I will NOT have a relationship in Schema, this way the entire Object or List<> will be written in as the embedded value.
    –a) This will be for when user will have a copy an Object to EmbeddedObject, since want to preserve the ‘_id’ → ‘id’ value across objects (no duplicate ‘_id’ in a Collection).

  2. Partition-based without EmbeddedObjects. Using relationships via foreign_key references as the nested value.
    –a) This will be for simple user partitions, complex user partitions (via parsing ‘_id’ for further partitioning) and global partitions (for all users).

  3. Flexible Sync. To replace most of Legacy Realm’s QBS and which would be too burdensome to have on Partition-based.

At this time we do not plan to have partition-based sync and flexible sync configured on the same backend app.

Oh man :disappointed:.

@Ian_Ward -

Is that be something you all could consider?

I will dig into managing 2 Apps then. Could you advise on any concerns or issues for 2 apps across single users?

P.S. btw the feature being developed for converting Partition-based to Flexible now makes a lot more sense and its worth - thanks for building that!

@Ian_Ward,

What about offering ability to add multiple Atlas-DBs (e.g.: one for Partition and one for Flexible) to a single Realm App via separated Syncs (could even have separate App-IDs) This way there would only be 1 user to manage (since 1 Realm App). Of course, the Collections may be duplicative but that would be the case anyhow. Each Sync in the Realm App would be either Partition-based or Flexible.

1 Like