Need a hand to help me with some logic

Hello, I got hired (non paid, basically I got chosen and I love what I do) to do a website used to browse logs for a videogame, so currently I have multiple documents each containing different fields, each document is a log object (since they contain different fields I have to create one document for each type of log), my question is simple but the answer is quite the opposite I’m afraid.

I want to be able to, query all documents and sort every single result based on a field that is a timestamp, so I know I can query each collection and sort by timestamp, the problem is I cannot do a sort for all of the collections, I’ve tried google and came across some threads about using JS’s sort function to do it but I never get accurate results it’s just a mess (also mongoose .sort is slower than JS’s sort??)

The other problem comes with filtering data, I can easily regex for input data but not type data, without having a complete mess on the backend to check if filter contains value then query this collection, which would lead to ridiculous wait times when trying to gather data, I’m guessing this is not the first time this problem has occurred? How do big companies do it? Am I missing something critical? Also if there’s a better alternative, which? Thanks for your time!

Hi @lexi_hvh ,

I am a bit confused by the paragraphs, in the first one you mentioned that each document in the “logs” collection is different as each log may have different set of fields.

However, later on you look into

query each collection and sort by timestamp, the problem is I cannot do a sort for all of the collections

So do you store all logs in a single collection? Or you have a collection per log type?

All logs can sit on the same collection and have different fields in one collection as long as the predicates are common:

{
  logType : "one",
 "game" : "one",
 "playerDetails" : {
      name: "..."
 }
  timestamp : 11111
...
},
{
  logType : "two",
 "game" : "two",
 "races" : [ ... ],
  timestamp : 222222
...
},

No we can query all logs and sort by timestamp:

db.logs.find({}).sort({timestamp : -1});

Does that help your questions?

Thanks
Pavel

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.