WiredTiger.wt handle-open preventing server start

Ops and Admin
,
1

I am running mongo inside a docker container. It shut down unexpectedly and I was able to run --repair. However whenever I try to restart the container I get an error saying something about WiredTiger.wt having a handle open.

_posix_open_file:805:/data/db/WiredTiger.wt: handle-open: open",“error_str”:“Operation not permitted”,“error_code”:1

I’ll paste more of the log output below.

Thanks for any help.

Bruce

{"t":{"$date":"2023-09-17T21:26:24.775+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":1,"message":{"ts_sec":1694985984,"ts_usec":775719,"thread":"1:0xffff8b25d040","session_dhandle_name":"file:WiredTiger.wt","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/data/db/WiredTiger.wt: handle-open: open","error_str":"Operation not permitted","error_code":1}}}
{"t":{"$date":"2023-09-17T21:26:24.782+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":1,"message":{"ts_sec":1694985984,"ts_usec":782742,"thread":"1:0xffff8b25d040","session_dhandle_name":"file:WiredTiger.wt","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/data/db/WiredTiger.wt: handle-open: open","error_str":"Operation not permitted","error_code":1}}}
{"t":{"$date":"2023-09-17T21:26:24.789+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":1,"message":{"ts_sec":1694985984,"ts_usec":789444,"thread":"1:0xffff8b25d040","session_dhandle_name":"file:WiredTiger.wt","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/data/db/WiredTiger.wt: handle-open: open","error_str":"Operation not permitted","error_code":1}}}
{"t":{"$date":"2023-09-17T21:26:24.790+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
3

This is at least a partial answer to my own question. I discovered that I could start the mongo container with a bind mount to an empty directory and it would start just fine. But if I stopped the container and remounted it I would get the exact same errors as above. And I did try messing with the permissions.

However I didn’t mention in my initial post that I am running this in the docker desktop on macOS. And I found this warning on the documentation associated with the mongo image on the docker hub.

WARNING (Windows & OS X): When running the Linux-based MongoDB images on Windows and OS X, the file systems used to share between the host system and the Docker container is not compatible with the memory mapped files used by MongoDB (docs.mongodb.org and related jira.mongodb.org bug). This means that it is not possible to run a MongoDB container with the data directory mapped to the host. To persist data between container restarts, we recommend using a local named volume instead (see docker volume create). Alternatively you can use the Windows-based images on Windows.

So I created a named volume “mongo-volume” and started the mongo container with the line

docker run -p 27017:27017 -v mongo-volume:/data/db --name tactic-mongo --network=tactic-net --restart always -d mongo:latest

This seems to do the trick, except that I wasn’t able to recover some data in the previous version of the database. Also, it’s unclear why my old way of running the container, which I had been using for years, suddenly stopped working.

1 Like