Ok, I think I know what the problem is.
When you initialize the replica set with the default settings, because you are inside docker, the replica set member uses the docker host name, which is unreachable from outside docker, and the latest version of the node driver that Compass uses doesn’t like that.
To work around that, you can initialize the replica set like this:
docker run -it --rm --link my-mongodb:mongo mongo:4 mongo --host mongo --eval "rs.initiate({_id: 'rs0', members: [{_id:1, host:'10.x.x.x:27017'}]})"
where 10.x.x.x is the IP you use when connecting to it from Compass.
That should work.