The MongoDB election process works based on the heartbeat process, which is basically a ping between nodes so that they can validate communication between themselves. If one of the nodes does not receive a response from another node, this node will request the election and the others will vote for it. So, in addition to the obvious priority process and lag in the replica, what can influence it is the network latency itself.
In other databases, they use the raft process to help with the elections, it is similar to the one used by MongoDB, but in MongoDB there are a few differences that make the election faster overall.