Thanks you! The solution using 3 datacenters looks great but I am not sure about operator of planned system has that possibility.
I understand that there is many reasons to get unavailable primary site. It can be only connectivity issue, then mongo cluster on primary site will work the same way as if DR site goes down. In this case parallel changes are possible if a node in DR site will be available for write.
Or it can be a nature cataclysm that will brake all of primary site and secondary will remain single source of truth.
So, disaster recovery procedure shall include a decision, can we bring up a primary site pretty fast and remain DR site in read-only mode or we shall discard data on primary site, build a new cluster on DR site and then add primary site nodes to it and after finishing of synchronisation exclude two additional nodes from DR site.
How it looks? Are there good practices to get back to work until primary site is available for a replica set across 2 datacenters?