Log rotation on Arbiter

Ops and Admin Replication
,
#1

Good day all.

We are currently running a single instance of MongoDB within a replicaset that has only one member.

I’m in the process of migrating to a PSA architecture. The following will be my setup:

mongo1: primary, priority: 10
mongo2: secondary, priority: 1
mongoarb: arbiterOnly

* All Windows servers

The 3 servers are configured with:

security:
  authorization: enabled
  keyFile: <path>\mongo.key

replication:
  replSetName: rs0

systemLog:
  destination: file
  verbosity: 0
  quiet: false
  logAppend: true
  logRotate: rename
  path:  <path>\mongod.log

We already have a windows task to logRotate with a specific logsadm user account. The task is set to run every day, and we keep only 7 days worth of logs.

I found that on primary and secondary, I am able to issue the logRotate command no problem, i.e.:

mongosh.exe -u logsadm -p <password> --host mongo1 --port 27017 --eval "db.adminCommand({ logRotate: 1 })"
mongosh.exe -u logsadm -p <password> --host mongo2 --port 27017 --eval "db.adminCommand({ logRotate: 1 })"

However, because Arbiter doesn’t have a copy of the admin.users collection, it obviously cannot issue the command.

I tried connecting to the arbiter directly and create a logsadm user in it, but it of course won’t let me, because it is not the primary server.

My question: what is normally done for log rotation in regards of arbiters? I’m currently in a sandbox, testing it all (localhost, running PSA on different ports), so I can always redo steps…

For example, before adding arbiter to the replicaset, should I have went in and create the logsadm user, and only after add it as a member to the RS? Could this somehow affect the integrity of the replicaset afterwards?

Thanks for your returns.
Pat

#3

Hi @Patrick_Roy and welcome to the MongoDB community forum!!

As quoted in the MongoDB documentation, since the arbiters do not store data, they do not have access to the user and role mapping as those of the primary and secondary mongod process.
Hence, the log rotation on arbiter is not possible.

Unfortunately since you have auth in the replica set, this puts the arbiter in an awkward place where it needs an authorised user to perform some commands like logRotate, however it cannot store any user since by definition arbiters do not store data.

Let us know if you have any further queries.

Best Regards
Aasawari

#4

Thanks for your reply @Aasawari. My workaround is to simply have the log file rotate when the mongod service restarts on the arbiter. Could always force a restart nightly, or not. But that seems to be my workaround.

Thanks for your time. Pat.

#5

instead of trying to log in to the arbiter instance remotely, try logging into the host of the arbiter to use the local host exception.

for example, using docker, this simply works in P-S-A:

docker exec -it mongo3  mongosh --eval "db.adminCommand({logRotate:1})"

I haven’t tried this with a key file, so please share with us if it works :slight_smile:

#6

Hello @Yilmaz_Durmaz. Sorry for the delay in responding.

That doesn’t work… I’m getting a:

MongoServerError: not authorized on admin to execute command { logRotate: 1, $db: “admin” }

The only way I can get the Arbiter to rotate it’s log with the logRotate command is if I completely disable/comment out security from its config file:

#security:
#  authorization: enabled
#  keyFile: <path-to-keyfile>

But I am not sure if it’s right to do so… Plus, since all my servers use the key file authentication, won’t this break my replicaset communications or something (sorry, not that knowledgeable regarding security …)

#7

hey again :wink:

I was trying to find a solution. For the moment, I don’t have one to do it through the mongo shell, but I have found a single command to use on all members.
logRotate — MongoDB Manual (since 4.2, at least)

You may also rotate the logs by sending a SIGUSR1 signal to the mongod process

The following uses PID=1 as mongod is the first process when started by “docker compose”.

kill -SIGUSR1 1

You just need to log in to the host machine, that is all. Also, the same command for all members is easier to maintain. You will need to find the appropriate command to fetch PID if it differs in your case.

PS: This command will be valid unless someday MongoDB developers decide to deprecate it for some reason. Yet, logical thinking says it will stay with us :slight_smile:

1 Like
#8

I would LOVE to be able to use the -SIGUSR1 signal to force mongod to do a rotate log, but… we’re unfortunately on Windows servers…

I’ve just setup our arbiter’s server not to append in logs: they are rotated whenever we restart the mongod service normally (server restart,etc). Some day, maybe we’ll get on Linux :wink:

Thanks for your replies and time :slight_smile:

1 Like
#9

I missed that “windows” part. Then let me ask you a few questions about your setup:

  • Are they all physical machines you maintain, or are they containers or bare metal but in the cloud?
  • are members separated to their respective machines?

Physical is not easy to try scrap ideas, but if you use “easy to discard” cloud machines, or containers, then why don’t you consider putting the arbiter in a Linux machine/container. I haven’t tried such a setup myself, but that should be possible as members are communicating over the network already. Then managing arbiter might get easier. I would like to hear the result if you try this setup :wink:

#10

They are all physical / separate windows servers: primary, secondary and arbiter. Moving to Linux / Cloud is not an option. We can disregard this post for now as rotating the arbiter’s log is not that big of a deal… :wink:

But much thanks for the time you’ve placed in this :))

1 Like