Primary DB is taking too long to connect to mongo shell

Hey everyone,

We have 2 servers of different configuration. One is of 32 gb RAM on which Primary DB is running and other is server of 12 GB RAM on which Secondary and Arbiter DB is running.
So, the problem is Primary DB is taking too long to connect with mongo shell about 10 minutes whereas I am able to connect with Secondary DB mongo shell easily.
I also tried to make Secondary DB to Primary DB but it is also behaving same way. So, ideally situation is which so ever DB is Primary one. It’s taking too long to connect to mongo shell. So, do anyone knows what could be the issue here??

Just one more thing as Standalone DB is working fine. There is an issue with Replication I guess.Also, after we upgraded to 4.4 from 3.4 this issue is coming and we can’t go back to 3.4.
So, please help me out what could be an issue here.

Thanks in advance