How to store user defined schema and save data based on that schema

[Level_0_You]

we want to allow user to define schema and upload data to save based on that schema,
what’s the best way to store the “user define schema” in mongodb, as Text ? or Json?

one user can have
var userSchemaB = {
title: String,
body: String
};

another user can have
var userSchemaA= {
productName: String,
productDescription: String
};

[slava]

Hello, @Level_0_You!

MongoDB has the flexible schema. It means, that you can insert documents with completely different structure in your collection. But in this case, you would need to do the data validation on application side, before each write-operation (create, update).

Although, MongoDB supports schema validation, but it will validate all the documents in a collection against the same schema. It means, that you can not have multiple schemas for a single collection.

You can, however, use MongoDB schema validation with different schemas by creating a dedicated collection with its own schema per each user:

const userId = <id>;
db.getCollection(`{userId}_products`).insertOne(<payload>);

That would work, but you can easily reach limitations collection number if your db instance is provided by some Cloud provider. However, if you have deployed MongoDB on your own servers, the limitations are much more compromising. So, use this at with extreme caution and only if you’re sure it will not create any complications in your application code.

Better to think, if you can standardize the data format, coming from your users. This could make your developer’s life much easier: you could use only 1 schema for your single collection and all the validation could be done on database side.

If you can not control user’s payload, then consider this 3-step solution:

1. Define schema for each user. You can use shema-inspector shemas for schema definitions and validations, if you have Node.js on backend side. If you use other programming language - search a lib, that can support validation of documents, based on predefined schema.

2. Persist the schemas in some user-related collection. You will need to fetch those schemas each time, when user would want to create/update the product.

3. Validate user’s payload object before inserting or updating document:

• fetch schema for dedicated user
• validate payload against that schema
• if there are errors, return them to the client
• if everything is OK - write to the collection

[Level_0_You]

@slava

Persist the schemas in some user-related collection. You will need to fetch those schemas each time, when user would want to create/update the product.

@slava thanks a lot for the response. after schema1 and schema2 is defined from the UI, how do i save it in the database before i fetch it? I tried to save it as Object Data type, but it doesnt work. each schema is for a different collection, so user can upload data into each collection based on the defined schema.

[slava]

Please, share:

• example of the schema, that you’re trying to save
• lines of code, where you write the schema to your db

[Level_0_You]

This is the code to save userDefinedSchema in the database, idea is that later i can read it back from tableSetup collection and use it to save new data for the user defined collection.

			var userDefinedSchema = {
				  title: String,
				  body: String,
				  date: 
					{
					  type: String,
					  default: Date.now()
					}
				  }; 

			var tableUploadSetup = {
			  setupId:String,
			  userDefinedSchemaDetails:Object
			  };

			const tableSetupSchema = new Schema(tableUploadSetup);
			const newTableSetupModel = moogoose.model('TableSetup',tableSetupSchema);


			const newTableSetupData = {
			  setupId: 'SETUP001',
			  userDefinedSchemaDetails:userDefinedSchema
			};

			const newTableSetup = new newTableSetupModel(newTableSetupData);

			newTableSetupModel.save((error) =>{
			if(error){
				console.log('error: '+error)
			  } else{
				console.log('data is saved !!!')
			   }
			 })

still new to mongodb, not sure whether this is the right approach. thanks for your reply.

[slava]

Ok, I must admit, schema management with mongoose is easy, but the schema can not be easily saved to db. Try schema-inspector instead. It’s schema definitions can be safely stored in DB. Here is an example of how it can be used:

const { createConnection } = require('./helpers');
const { ObjectId } = require('bson');

const inspector = require('schema-inspector');

(async () => {
  // connect to your mongodb somehow
  const url = 'url-to-your-db';
  const connection = await createConnection(url);

  // create links to your collections for later usage
  const dbName = 'for-test';
  const usersColl = connection.db(dbName).collection('users');
  const productsColl = connection.db(dbName).collection('products');

  // get userId somehow
  const userId = new ObjectId();

  // accept user's schema definition
  // from request.body, for example
  const schemaDefinition = {
    type: 'object',
    properties: {
      productName: {
        type: 'string',
        optional: false,
        minLength: 2
      },
      productDescription:  {
        optional: true
      },
    }
  };

  // write that definition for later usage
  await usersColl.insertOne({
    _id: userId,
    productSchema: schemaDefinition
  });

  // when user wants to create / update document,
  // fetch the definitions from the collection
  const user = await usersColl.findOne({ _id: userId });

  // get create / update payload somehow
  const payload = {
    productName: 'B',
  };

  // validate payload against your schema
  const validationResult = inspector.validate(user.productSchema, payload)

  if (!validationResult.valid) {
    console.log('errors:', validationResult.error);
    // return errors back to user
  }
  else {
    // insert or update the payload to your db
    productsColl.insertOne(payload);
  }

  console.log('done!');
  process.exit(0);

})();

[Level_0_You]

@slava thank you so much so much, i will try it out.