what index you have on each of collections?
user will have default _id index and the same with rest of the collections but if you have user_id index on address and address_id index on addressComments then lookup will be faster.
also if you are not using project for lookup then you lookup can be simplified ad below:
{$lookup:{
from: 'address',
localField: '$_id',
foreignField: '$user_id',
as: 'address'
}