Hey @shuryanc,
Welcome to the MongoDB Community Forums! 
In addition to the problem you pointed out, there are always issues with having multiple arbiters in a replica set and you should be sure that you need two arbiters in your current replica set instead of adding another secondary. My recommendation would be to replace the arbiter with another data-bearing member. You can read more about the issues from the documentation: Concerns with Multiple Arbiters.
You can also check out this forum post on the consequences of having arbiters in a replica set, see: Replica set with 3 DB Nodes and 1 Arbiter - #8 by Stennie
Please let us know if this helps or not. Feel free to reach out for anything else as well.
Regards,
Satyam