Get documents from multiple collections and append them as per condition

[Himanshu_Singh]

Hi Everyone!
I’ve two collections likes and posts. For every like on a post, I’m creating a like document.
Now, I’ve to make data such that it can contain if a provided user liked which posts.
For example, If there are 10 posts and a user liked 5 of them. I’ve to get all 10 posts but with a field or anything which can infer that a specific post was liked by the user.

Like Schema

{
    performer: {
      username: {
          type: String
      }
    },
    post_id: {
        type: Schema.Types.ObjectId,
        required: true,
        ref: 'posts'
    },
    title: {
        type: String
    }
  }

Post Schema:

{
  user: {
    username: {
      type: String
    }
  },
  post_id: {
    type: String,
    required: true
  },
  title: {
    type: String,
  }
}

I need a JSON if 'performer.username': "X" requests for data like:

{
  user: {
    username: {
      type: String
    }
  },
  post_id: {
    type: String,
    required: true
  },
  title: {
    type: String,
  },
liked: Boolean
}

Please guide me on what should I look for in this case scenario.

[shrey_batra]

Hi @Himanshu_Singh,

See a simple workaround is to fetch all the posts from your collection, then have a $lookup stage in your aggregation to like collection. This will join all the likes per post, giving you an array. While doing this, in $lookup you can have the pipeline attribute to limit the documents you fetch from Like collection (fetch only Like documents that is of User X). This will give you something like this -

[
    {
        "post_id": 1,
        "likes": [
            {
                // like document
            }
        ]
    }
]

Now you can check that is the likes array is empty, user did not like the post, else he liked the post. Can do this using a $project stage.

My worries - Why are we fetching all posts and marking posts that are liked by the users? This could have serious performance issues. Ex - Number of Posts are 1M, so for each user, all 1M posts are fetched and then $lookup to Likes, which is very costly. Obviously, you can have indexing and pagination and all, but do evaluate the performance angle of your use case.

Happy to see what others come up with.!

[Himanshu_Singh]

Actually, I’m making a social network clone as my project.
When a user likes a post I’ll have to show it liked in UI and for that I’ll be needing something by which I can manage the like state when user browses feed.

Is there any alternative way to do this?
Please let me know I’ll be really thankful for that.

[shrey_batra]

You can follow the above approach of aggregation, but then make sure to have pagination in Posts collection (taking only 20/40 documents at once), and to apply pipeline parameter in $lookup. This is for production level performance, if you are making a small project these won’t make more sense.

[Himanshu_Singh]

Thank you so much @shrey_batra, May I get any resource related to this?
I’m using mongoose on nodejs.
Someone said to me that I can store an array of user_id of users who liked that specific post and that check on the frontend via .includes and map according to that.
What do you say about this approach?
I think it’ll lead to unnecessary data transmission from API.

[shrey_batra]

It may… Also it might lead to data scaling problem when your solution might reach 10-50k users. Imagine that every post is liked by each user, your array might be going boundless, with a max limit of 16 MB per document. Not sure about approach in NodeJs, but you can see the generic aggregation stage - $lookup. I am sure this stage can be applied in the Mongoose when applying an aggregation.

[Himanshu_Singh]

Exactly that’s what I thought! It’ll be too heavy and retaining that in client-side will end up in performance issues. Let me look around for #lookup.

Thank you