UPDATE:
I was taking the wrong approach in using the --gzip and --archive options for the incremental backups as the --oplog option did not seem to work well with these options.
I instead replaced it with:
mongodump -h localhost -d local -c oplog.rs --queryFile ${QUERY_DIR}/query.js --port 27017 -o ${INCREMENTAL_BACKUP_DIR}/$DATE >> $LOG_FILE 2>&1
where,
-h is the host id which is simply localhost locally on my server,
-d is the database I want to dump from, ‘local’ is the default location where oplog.rs is maintained
-c is the collection to dump which is oplog.rs
–queryFile - I replaced the timestamping method in my old code with a single query.js file (whose contents are {“ts”:{“$gt”:{“$timestamp”:{“t”:1722261163,“i”:1}}}}). The timestamp is generated in a simple “$date +%s” string format.
-o is the dump location
To restore full database:
I can still restore my archived base copy first using (from terminal not mongo shell):
$mongorestore --gzip --archive=<full_backup_name>.gz
Then proceed to apply my incremental backups, in the same sequence in which they were created as below:
$mongorestore -h localhost --port 27017 --oplogReplay --dir incremental_backup_main_directory/incremental_backup_<date1> --oplogFile=incremental_backup_main_directory/incremental_backup_<date1>/oplog.rs.bson
$mongorestore -h localhost --port 27017 --oplogReplay --dir incremental_backup_main_directory/incremental_backup_<date2> --oplogFile=incremental_backup_main_directory/incremental_backup_<date2>/oplog.rs.bson
Hope this is helpful to someone.