How to handle mass changes to a large document with frequent reads in MongoDB?

Let’s say I have a document in the following format in MongoDB:

students: Array
grades: Array

I currently have about 10,000 students and grades that are constantly changing. The number of students is constantly growing and students are removed from the document. I have a process to update the document every 30 minutes. At the same time, I’ve built an ExpressJS API where various teachers query the database as often as every minute to view info about their students.

  1. What is the best way to update the data? Since there are so many possibilities of students being added, removed, and with grades changed, should I just overwrite the document every 30 minutes? The dataset is only a couple MB in size overall.
  2. How can I ensure that the teachers will have no downtime if I happen to be updating at the same time they’re making a GET request?

Hi @John_N_A1 and welcome in the MongoDB Community :muscle: !

There is no “down time” when you update a document. Your driver (serving the API) will either read the previous version or the new one of the document.
If your update isn’t bundled in ONE single update operation, then you are doing it wrong and it means that any intermediate subversion can be accessed as well. A single update operation (on one doc) is atomic, even if it contains 3 updates in it. But if you split these 3 updates into 3 single update operation, these can be queried by the API.

That being said, I think your schema design isn’t well designed for the service you are trying to serve here.
Why not have a students collection where one doc represents ONE student and also contains its grades?

This would avoid most of the mess with the arrays and it would still be possible to build an API serving all the students from a classroom to a teacher by using the aggregation framework, calculate means, etc.

Cheers,
Maxime.

2 Likes

thanks Maxime! that makes sense. having a single document per student because we get the updates in batches and we have administrators that need to read data for all students frequently.

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.