Mongod processes

Today something goes different than yesterday…
when i try to start a mongod process:

`mongod -f mongodTest/replication/repl-node1/etc/mongod.cfg`

i receive the following output:

    about to fork child process, waiting until server is ready for connections.
    forked process: 2148

but after this nothing else.
I tried to stop the command with:

`^C (Ctrl+C)`

and run:

`ps -ef | grep mongod`

this is what i obtain:

    vagrant   2147     1  0 08:23 ?        00:00:00 mongod -f mongodTest/replication/repl-node1/etc/mongod.cfg
    vagrant   2148  2147  4 08:23 ?        00:00:03 mongod -f mongodTest/replication/repl-node1/etc/mongod.cfg
    vagrant   2179  2116  0 08:24 pts/0    00:00:00 grep --color=auto mongod

Someone knows what is happening? Thanks for the help :sweat_smile:

Hi @PC-Informatica,

Please try killing the mongod process and launch it again using the following command:

killall mongod

If you still the above output after re launching mongod, then remove the fork option in the mongod.cfg and then launch mongod, you will able to get the error.

If you still need help in solving the error, share the screenshot of the error and the configuration file.

Thanks,
Sonali

@Sonali_Mamgain

mongod.cfg:

storage:
    dbPath: "mongodTest/replication/repl-node1/data"

net:
    port: 27010
    bindIp: "localhost,192.168.103.100"

systemLog:
    verbosity: 1
    path: "mongodTest/replication/repl-node1/mongod.log"
    destination: file
    logAppend: true

operationProfiling:
    mode: off
    slowOpThresholdMs: 100

security:
    authorization: enabled
    keyFile: "mongodTest/pki/repl-keyFile"

replication:
    replSetName: replicationTest

Output: nothing

I would recommend to use full path name instead of relative path name. The problem with relative path is that it all depends of your working directory to know where are the files.

If the only output you get is:

it means mongod has started correctly and is simply awaiting connections.

Hi @steevej-1495 , thank you for the response.

no mongod is not working correctly because i remain blocked into the command, the command doesn’t end alone, i need to Ctrl+C

I’ve already completed the lab Configure a Sharded Cluster
so the configuration file of the replica set (shard 1) should be right…well, when i try to start the first mongod nothing happens, only the two lines of output and the command that doesn’t end

–fork option is missing in your config file
If you use --fork it runs in background and prompt comes back to you

Since you are not using --fork your mongod is running in foreground and came up fine

You should not ctl+c.It gets killed
Open another session and try to connect to mongod by using mongo --port

or use --fork and you can connect in same session

1 Like

@Ramachandra_37567 Thank you!

mongod.cfg:

storage:
    dbPath: "mongodTest/replication/repl-node1/data"

net:
    port: 27010
    bindIp: "localhost,192.168.103.100"

processManagement:
    fork: false

security:
    authorization: enabled
    keyFile: "mongodTest/pki/repl-keyFile"

replication:
    replSetName: replicationTest

this is the output that i receive running mongod on port 27010:

vagrant@m103:~$ mongod -f mongodTest/replication/repl-node1/etc/mongod.cfg
2019-06-02T15:49:20.378+0000 I CONTROL  [initandlisten] MongoDB starting : pid=2475 port=27010 dbpath=/home/vagrant/mongodTest/replication/repl-node1/data 64-bit host=m103
2019-06-02T15:49:20.379+0000 I CONTROL  [initandlisten] db version v3.6.12
2019-06-02T15:49:20.380+0000 I CONTROL  [initandlisten] git version: c2b9acad0248ca06b14ef1640734b5d0595b55f1
2019-06-02T15:49:20.380+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2019-06-02T15:49:20.381+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2019-06-02T15:49:20.381+0000 I CONTROL  [initandlisten] modules: enterprise
2019-06-02T15:49:20.381+0000 I CONTROL  [initandlisten] build environment:
2019-06-02T15:49:20.382+0000 I CONTROL  [initandlisten]     distmod: ubuntu1404
2019-06-02T15:49:20.382+0000 I CONTROL  [initandlisten]     distarch: x86_64
2019-06-02T15:49:20.382+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2019-06-02T15:49:20.382+0000 I CONTROL  [initandlisten] options: { config: "mongodTest/replication/repl-node1/etc/mongod.cfg", net: { bindIp: "localhost,192.168.103.100", port: 27010 }, processManagement: { fork: false }, replication: { replSetName: "replicationTest" }, security: { authorization: "enabled", keyFile: "mongodTest/pki/repl-keyFile" }, storage: { dbPath: "mongodTest/replication/repl-node1/data" } }
2019-06-02T15:49:20.383+0000 I -        [initandlisten] Detected data files in /home/vagrant/mongodTest/replication/repl-node1/data created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2019-06-02T15:49:20.383+0000 I STORAGE  [initandlisten]
2019-06-02T15:49:20.384+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-06-02T15:49:20.384+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-06-02T15:49:20.385+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=488M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2019-06-02T15:49:21.201+0000 I STORAGE  [initandlisten] WiredTiger message [1559490561:201528][2475:0x7f6c5cb82a80], txn-recover: Main recovery loop: starting at 27/7040
2019-06-02T15:49:21.330+0000 I STORAGE  [initandlisten] WiredTiger message [1559490561:330839][2475:0x7f6c5cb82a80], txn-recover: Recovering log 27 through 28
2019-06-02T15:49:21.409+0000 I STORAGE  [initandlisten] WiredTiger message [1559490561:409751][2475:0x7f6c5cb82a80], txn-recover: Recovering log 28 through 28
2019-06-02T15:49:21.477+0000 I STORAGE  [initandlisten] WiredTiger message [1559490561:477275][2475:0x7f6c5cb82a80], txn-recover: Set global recovery timestamp: 0
2019-06-02T15:49:21.516+0000 I STORAGE  [initandlisten] Starting WiredTigerRecordStoreThread local.oplog.rs
2019-06-02T15:49:21.516+0000 I STORAGE  [initandlisten] The size storer reports that the oplog contains 4182 records totaling to 494748 bytes
2019-06-02T15:49:21.517+0000 I STORAGE  [initandlisten] Scanning the oplog to determine where to place markers for truncation
2019-06-02T15:49:21.536+0000 I CONTROL  [initandlisten]
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten]
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-06-02T15:49:21.539+0000 I CONTROL  [initandlisten]
2019-06-02T15:49:21.545+0000 W SHARDING [initandlisten] Not started with --shardsvr, but a shardIdentity document was found on disk in admin.system.version: { _id: "shardIdentity", clusterId: ObjectId('5ceffaa4be7e1dc0798b4c7b'), shardName: "replicationTest", configsvrConnectionString: "replicationTest-csrs/192.168.103.100:28010,192.168.103.100:28011,192.168.103.100:28012" }
2019-06-02T15:49:21.546+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/home/vagrant/mongodTest/replication/repl-node1/data/diagnostic.data'
2019-06-02T15:49:21.549+0000 I REPL     [initandlisten] Rollback ID is 1
2019-06-02T15:49:21.551+0000 I REPL     [initandlisten] No oplog entries to apply for recovery. appliedThrough and checkpointTimestamp are both null.
2019-06-02T15:49:21.556+0000 I NETWORK  [initandlisten] waiting for connections on port 27010
2019-06-02T15:49:21.556+0000 I CONTROL  [LogicalSessionCacheRefresh] Sessions collection is not set up; waiting until next sessions refresh interval: Replication has not yet been configured
2019-06-02T15:49:21.557+0000 W NETWORK  [replexec-0] Failed to connect to 192.168.103.100:27011, in(checking socket for error after poll), reason: Connection refused
2019-06-02T15:49:21.581+0000 I REPL     [replexec-0] New replica set config in use: { _id: "replicationTest", version: 12, protocolVersion: 1, members: [ { _id: 0, host: "192.168.103.100:27010", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 1, host: "192.168.103.100:27011", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 }, { _id: 2, host: "192.168.103.100:27012", arbiterOnly: false, buildIndexes: true, hidden: false, priority: 1.0, tags: {}, slaveDelay: 0, votes: 1 } ], settings: { chainingAllowed: true, heartbeatIntervalMillis: 2000, heartbeatTimeoutSecs: 10, electionTimeoutMillis: 10000, catchUpTimeoutMillis: -1, catchUpTakeoverDelayMillis: 30000, getLastErrorModes: {}, getLastErrorDefaults: { w: 1, wtimeout: 0 }, replicaSetId: ObjectId('5ce999aaad7eee660f3dc04e') } }
2019-06-02T15:49:21.581+0000 I REPL     [replexec-0] This node is 192.168.103.100:27010 in the config
2019-06-02T15:49:21.582+0000 I REPL     [replexec-0] transition to STARTUP2 from STARTUP
2019-06-02T15:49:21.583+0000 I REPL     [replexec-0] Starting replication storage threads
2019-06-02T15:49:21.583+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:21.585+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27012
2019-06-02T15:49:21.586+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:21.587+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:21.588+0000 I REPL_HB  [replexec-1] Error in heartbeat (requestId: 4) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:21.588+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:21.590+0000 I REPL     [replexec-0] transition to RECOVERING from STARTUP2
2019-06-02T15:49:21.590+0000 I REPL     [replexec-0] Starting replication fetcher thread
2019-06-02T15:49:21.590+0000 I REPL     [replexec-0] Starting replication applier thread
2019-06-02T15:49:21.590+0000 I REPL     [replexec-0] Starting replication reporter thread
2019-06-02T15:49:21.591+0000 I REPL     [rsSync] transition to SECONDARY from RECOVERING
2019-06-02T15:49:21.591+0000 I REPL     [rsSync] Resetting sync source to empty, which was :27017
2019-06-02T15:49:21.592+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:21.592+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:21.595+0000 I REPL_HB  [replexec-1] Error in heartbeat (requestId: 8) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:21.597+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:21.598+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:21.599+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:21.599+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 11) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:21.599+0000 I REPL     [replexec-0] Member 192.168.103.100:27011 is now in state RS_DOWN
2019-06-02T15:49:21.599+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Successfully connected to 192.168.103.100:27012, took 16ms (1 connections now open to 192.168.103.100:27012)
2019-06-02T15:49:21.601+0000 I REPL     [replexec-1] Member 192.168.103.100:27012 is now in state SECONDARY
^C2019-06-02T15:49:21.844+0000 I CONTROL  [signalProcessingThread] got signal 2 (Interrupt), will terminate after current cmd ends
2019-06-02T15:49:21.845+0000 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
2019-06-02T15:49:21.845+0000 I NETWORK  [signalProcessingThread] removing socket file: /tmp/mongodb-27010.sock
2019-06-02T15:49:21.846+0000 I REPL     [signalProcessingThread] shutting down replication subsystems
2019-06-02T15:49:21.846+0000 I REPL     [signalProcessingThread] Stopping replication reporter thread
2019-06-02T15:49:21.847+0000 I REPL     [signalProcessingThread] Stopping replication fetcher thread
2019-06-02T15:49:21.848+0000 I REPL     [signalProcessingThread] Stopping replication applier thread
2019-06-02T15:49:22.101+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:22.101+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:22.101+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:22.102+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 16) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:22.105+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:22.108+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:22.108+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:22.108+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 18) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:22.109+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Connecting to 192.168.103.100:27011
2019-06-02T15:49:22.110+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Failed to connect to 192.168.103.100:27011 - HostUnreachable: Connection refused
2019-06-02T15:49:22.110+0000 I ASIO     [NetworkInterfaceASIO-Replication-0] Dropping all pooled connections to 192.168.103.100:27011 due to failed operation on a connection
2019-06-02T15:49:22.110+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 21) to 192.168.103.100:27011, response status: HostUnreachable: Connection refused
2019-06-02T15:49:22.601+0000 I REPL     [signalProcessingThread] Stopping replication storage threads
2019-06-02T15:49:22.614+0000 I FTDC     [signalProcessingThread] Shutting down full-time diagnostic data capture
2019-06-02T15:49:22.620+0000 I STORAGE  [WTOplogJournalThread] oplog journal thread loop shutting down
2019-06-02T15:49:22.621+0000 I STORAGE  [signalProcessingThread] WiredTigerKVEngine shutting down
2019-06-02T15:49:22.743+0000 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
2019-06-02T15:49:22.743+0000 I CONTROL  [signalProcessingThread] now exiting
2019-06-02T15:49:22.744+0000 I CONTROL  [signalProcessingThread] shutting down with code:0

if run mongo --port 27010 in another session all works fine(but i receive SECONDARY).

ps - ef | grep mongod

vagrant   2673  1959  3 15:54 pts/0    00:00:05 mongod -f mongodTest/replication/repl-node1/etc/mongod.cfg
vagrant   2769  2046  0 15:57 pts/2    00:00:00 grep --color=auto mongod

if run mongo --port 27010 in another session all works fine(but i receive SECONDARY).
If you are testing replication where are your other nodes?I see only one mongod up from your output
You should have min 3 nodes and once replication is setup one will be elected as primary and other two will be secondary

@Ramachandra_37567 hi, im sorry i messed all up mixing two examples :sweat_smile:

After clearing my mind a little, I try again to explain my problem.

Day 1: creation and use of the sharded cluster (result: done everything correctly),

Day 2: restart the sharded cluster and then use it to follow the lessons (result: I could not start the shards, not completely (as you can see from my first message))

Discovery of the error: when I tried to restart the sharded cluster, I started to bring up the shards replica sets, this is apparently a wrong step.

Error resolution: I took inspiration from this other thread: https://www.mongodb.com/community/forums/t/the-best-sequence-to-start-a-cluster-shards-config-servers-and-mongos / 10826
The best sequence to start a cluster:
1 bring up config servers
2 bring up shards servers
3 bring up mongos

I apologize again for how I expressed my problem. I thank all those who wanted to help me: @Ramachandra_37567, @steevej-1495, @Sonali_Mamgain :heart_eyes: