Looks like this extra validation on field paths in projections was added in MongoDB 4.4. https://jira.mongodb.org/browse/SERVER-43613 was opened to request that the server return a more helpful error message. Please follow that ticket for updates.
As for how to workaround this issue, if the field_name is empty you can set the entire projection to be empty or None:
if field_name:
projection = {field_name: 1}
else:
projection = None
cursor = db.coll_name.find({}, projection=projection)