Realm schema for different object types in array

Hi

EDIT: For clarity I have explained this as different object types in array instead of multiple types per field.

NOTE: In the case of different scalar types in the same field there is an outstanding issue with GraphQL (not Realm GraphQL) to have this supported in GraphQL schemas see Additional Note 1 below.


Environment:

MongoDB Atlas cluster running MongoDB 4.4
MongoDB Realm connected to cluster


Issue:

I am trying to configure a Realm schema and Realm GraphQL (and later Sync) to support data where an array of objects can contain different objects, regardless of what I try I am unable to generate/create a valid schema that reflects the data and also works for Realm GraphQL.

I am investigating and asking for help in case anyone else has a solution or potential solution that could be implemented in Realm GraphQL.


What I am trying to achieve:

  1. A valid Realm schema for my data that will validate against existing data and allow the array to accept multiple object types.
  2. A schema that will also work correctly for Realm GraphQL, either ‘as is’ or if needed by customising Realm GraphQL
  3. (later) a schema that will also work for Realm Sync

The problem:

Here is a simplified sample of the data I am trying to model, specifically the “results” array should be capable of accepting different “result” objects, and ideally each “result” object should be unique with respect to “type”. (see Additional Note 2 below for alternate data structure)

{
  "name": "This is a text string",
  "results": [
    { "type": "count", "value": 12, "unit": "items" },
    { "type": "comment", "value": "A text comment", "unit": "n/a" },
    { "type": "weight", "value": 56.69, "unit": "kg" }
  ]
}

What I have tried so far:

1 - I have inserted the typed data into a MongoDB collection:

2 - I have written a MongoDB collection validation using json schema which validates the data, note the use of an array of bsonType for “value”:

{
  $jsonSchema: {
    bsonType: "object",
    properties: {
      _id: { bsonType: "objectId" },
      name: { bsonType: "string" },
      results: {
        bsonType: "array",
        items: {
          bsonType: "object",
          properties: {
            type: { bsonType: "string" },
            value: { bsonType: [ "string" , "int", "double" ] },
            unit: { bsonType: "string" }
          }
        }
      }
    }
  }
}

The following MongoDB collection validation also validates, note the use of anyOf to validate each “results” item:

{
  $jsonSchema: {
    bsonType: "object",
    properties: {
      _id: { bsonType: "objectId" },
      name: { bsonType: "string" },
      results: {
        bsonType: "array",
        items: {
          anyOf: [
            {
              bsonType: "object",
              properties: { type: { bsonType: "string" }, value: { bsonType: "string" }, unit: { bsonType: "string" } }
            },
            {
              bsonType: "object",
              properties: { type: { bsonType: "string" }, value: { bsonType: "int" }, unit: { bsonType: "string" } }
            },
            {
              bsonType: "object",
              properties: { type: { bsonType: "string" }, value: { bsonType: "double" }, unit: { bsonType: "string" } }
            }
          ]
        }
      }
    }
  }
}

3 - I then use Realm to generate a schema from the data (Realm → Schema → select collection → Schema tab → Generate Schema), Realm suggests the following schema - note that the bsonType suggested is the first type in the array - in this case int, but if a different type was first in the array the generator would suggest that but not multiple types.
NOTE: you can set the bsonType to string which will allow a Realm GraphQL query to return the data as strings but not the type, this also means that mutations are not strongly typed and do not accept int or double values as desired.

{
  "title": "example",
  "properties": {
    "_id": { "bsonType": "objectId" },
    "name": { "bsonType": "string" },
    "results": {
      "bsonType": "array",
      "items": {
        "bsonType": "object",
        "properties": {
          "type": { "bsonType": "string" },
          "unit": { "bsonType": "string" },
          "value": { "bsonType": "int" }
        }
      }
    }
  }
}

4 - This schema is accepted by Realm GraphQL (no warnings) but when a query is run the results are not correct - note that the string value is returned as null and the double value has been truncated:

GraphQL query run in GraphiQL:

query { example { _id name results { type unit value } } }

Result:

{
  "data": {
    "example": {
      "_id": "60755d9c0000b6ffe15cf906",
      "name": "This is a text string",
      "results": [
        { "type": "count", "unit": "items", "value": 12 },
        { "type": "comment", "unit": "n/a", "value": null },
        { "type": "weight", "unit": "kg", "value": 56 }
      ]
    }
  }
}

5 - If I try to modify the Realm schema to match the data using an array of bsonTypes as suggested by the Realm documentation specifically in the note:

The fields available in a JSON schema object depends on the type of value that the schema defines. See the document schema types reference page for details on all of the available schema types.

that links to this documentation and shows “The following fields are available for all schema types” should accept an array for bsonType:

  ...
  "bsonType": "<BSON Type>" | ["<BSON Type>", ...],
  ...

The Realm schema should become:

{
  "title": "example",
  "properties": {
    "_id": { "bsonType": "objectId" },
    "name": { "bsonType": "string" },
    "results": {
      "bsonType": "array",
      "items": {
        "bsonType": "object",
        "properties": {
          "type": { "bsonType": "string" },
          "unit": { "bsonType": "string" },
          "value": { "bsonType": [ "string", "int", "double" ] }
        }
      }
    }
  }
}

however this Realm schema causes Realm GraphQL to show a warning:

results.value InvalidType error processing “type” property in JSON Schema

and trying to run a query returns an error:

GraphQL query run in GraphiQL:

query { example { _id name results { type unit value } } }

Result (error):

{
  "data": null,
    "errors": [
    {
      "message": "Cannot query field \"value\" on type \"ExampleResult\".",
      ...
    }
  ]
}

6 - I have also tried generating a valid JSON Schema using online tools and by manually constructing a schema, in these cases the tools sometimes suggest the use of “anyOf” however in each instance even if the schema is accepted by Realm it still shows the same GraphQL Schema generation warning.


Latest / TLDR:

Based on my investigation and the information about GraphQL union types described in the GraphQL GitHub issues it should be possible to use a union type in the Realm GraphQL schema to handle different objects nested in arrays.

In the GraphQL schema:

type CustomTypeComment { type: String, value: String, unit: String }
type CustomTypeCount { type: String, value: Int, unit: String }
type CustomTypeWeight { type: String, value: Double, unit: String }
union CustomResultType = CustomTypeComment | CustomTypeCount | CustomTypeWeight

type Event { id: String!, name: String, results: [CustomResultType] }

However I cannot see any way to customise the Realm GraphQL schema - there is an open issue to allow custom Realm GraphQL schema - please vote for this:

.
.


Additional:

Additional Note 1:

I have found there is a GraphQL proposal to support multiple scalar types in fields, for example using a string or double for the same field: Proposal: Support union scalar types #215

This is so that for the data:

[
  { "_id": "...", "multi": "A string" },
  { "_id": "...", "multi": 10.25 },
]

you could specify a GraphQL schema:

...
field: Int | String
...

Currently it seems a workaround is to define a ‘union’ type in GraphQL and then use that union type rather than scalar types:

For example in GraphQL:

type CustomTypeOne { id: Int, value: String }
type CustomTypeTwo { id: Int, value: Int }
union CustomType = CustomTypeOne | CustomTypeTwo

type Event { id: Int!, results: [CustomType] }

Additional Note 2:

If possible each “result” object should contain similar field names, however if this causes problems
the data could be modelled with each object having different field names, for example:

{
  "name": "This is a text string",
  "results": [
    { "type": "count", "count": 12, "unit": "items" },
    { "type": "comment", "comment": "A text comment" },
    { "type": "time", "time": 5600, "unit": "s" }
  ]
}
3 Likes

Did you found any solution for this ? I am also facing the same issue with Realm.

Edit - details moved to original post.

We don’t currently support a union of scalar types in our default GraphQL service to stay as close to the spec as possible. That being said, there is a workaround to achieve something similar by setting the schema of the field like so:

{ "field": { "anyOf": [{ "type": "string" }, { "type": "null" }] } }

The generated schema will ignore this field, but it can still be accessed via the custom resolver, (likely by having to define two different types).

1 Like

Hi Sumedha

I tried using anyOf in my Realm schema you suggested:

{
  "title": "example",
  "properties": {
    "_id": { "bsonType": "objectId" },
    "name": { "bsonType": "string" },
    "results": {
      "bsonType": "array",
      "items": {
        "bsonType": "object",
        "properties": {
          "type": { "bsonType": "string" },
          "value": { "anyOf": [{ "type": "string" }, { "type": "null" } ] },
          "unit": { "bsonType": "string" }
        }
      }
    }
  }
}

As expected the Realm GraphQL still gives me a schema error and ignores the field:

Field Path     Error Code         Message
results.value  MissingSchemaType  error processing "type" property in JSON Schema

Can you explain how I can access the collection using a custom resolver / define two different types

Hi @Sumedha_Mehta1. I tried your recommendation but it doesn’t seems to be working. Below is the schema that I am using:

{
 "title": "BuildingEquipment",
 "properties": {
    "_id": { "bsonType": "objectId"},
    "configs": {
       "bsonType": "array",
       "items": {
          "bsonType": "object",
          "properties": {
             "field": {"bsonType": "string"},
             "value": { "anyOf": [{"bsonType": "string"}, {"bsonType": "double"}]}
          }
       }
     }
  }
}

And following is the error I am getting when generating Realm Data Models

@Dhananjay_Puglia

Are you using Sync in this application? That error refers to that service specifically, not GraphQL. GraphQL will just ignore that field in your schema.

@Sumedha_Mehta1 Yes I am using Sync in this application. Will this solution not working with Sync? If no then is there any way where I can sync multiple type of data for a single field ?

We are working on a way to to introducing this for Sync pretty soon (~1 mo) - you can stay up to date on new data types by subscribing here: Realm Community Projects | Realm.io

Great. Thanks a lot @Sumedha_Mehta1 for the update.

I see that the “mixed” type has now been added to Realm DB / Sync – I presume this would now enable you to implement union types in GraphQL? Or has it already been done? @Sumedha_Mehta1

Is there any progress on this subject?

1 Like

You can apply multiple types to a field in the schema like this [“string”, “null”].
Refer: https://docs.mongodb.com/realm/schemas/types/
img: https://i.imgur.com/vulUoem.png

But those type will be ignored by the graphql schema generation.
Refer: https://docs.mongodb.com/realm/graphql/types-and-resolvers/
img: https://i.imgur.com/eXxqljz.png

currently I cant find a workaround for this.

2 Likes

Has anyone managed to get multiple types to be accepted at all? Using anyOf in the Payload Type has been mentioned as a way but this hasn’t worked for me (and seemingly for others on this thread).

I’d really like to know if this can actually be achieved and see a working example.

cc @bolokos_bolokos