Mloginfo encountered an error

[Sandie_Zarate1 (Sandie Zarate)]

mloginfo is encountering an error because of “###” in the mongodb log file.
what is the workaround?

 source: mongodb.log
   host: <host>:27017
  start: 2022 Jun 01 09:03:47.876
    end: 2022 Jun 01 13:35:00.151

date format: iso8601-local
timezone: UTC +0800
length: 39195
binary: mongod
version: 4.2.18
storage: wiredTiger

QUERIES
[= ] 1.5 % Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/util/logevent.py”, line 835, in _extract_counters
‘’))
ValueError: invalid literal for int() with base 10: ‘’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/usr/local/bin/mloginfo”, line 11, in
load_entry_point(‘mtools==1.7.0’, ‘console_scripts’, ‘mloginfo’)()
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/mloginfo/mloginfo.py”, line 112, in main
tool.run()
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/mloginfo/mloginfo.py”, line 107, in run
section.run()
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/mloginfo/sections/query_section.py”, line 92, in run
allowDiskUse=le.allowDiskUse)
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/util/logevent.py”, line 678, in allowDiskUse
self._extract_counters()
File “/usr/local/lib/python3.6/site-packages/mtools-1.7.0-py3.6.egg/mtools/util/logevent.py”, line 884, in _extract_counters
self._txnNumber = int((split_tokens[t + 1 + self.datetime_nextpos + 2]).replace(’,’, ‘’))
ValueError: invalid literal for int() with base 10: ‘"###"’

[Stennie_X (Stennie)]

Welcome to the MongoDB Community @Sandie_Zarate1 !

Can you share some more details to help reproduce the issue:

• Offending log line including ###
• Output of mloginfo --version
• Exact command line you used (I’m guessing this might be mloginfo --queries)

Thanks,
Stennie

[Sandie_Zarate1 (Sandie Zarate)]

offending log line:
2022-06-01T09:04:39.015+0800 I COMMAND [initandlisten] command local.oplog.rs command: getMore { getMore: “###”, collection: “###”, $db: “###” } originatingCommand: { find: “###”, filter: { ts: { $gte: “###”, $lte: “###” } }, oplogReplay: “###”, $db: “###” } planSummary: COLLSCAN cursorid:8082644926738273313 keysExamined:0 docsExamined:22521 numYields:186 nreturned:22520 reslen:16749135 locks:{ ReplicationStateTransition: { acquireCount: { w: 187 } }, Global: { acquireCount: { r: 187 } }, Database: { acquireCount: { r: 187 } }, Mutex: { acquireCount: { r: 1 } }, oplog: { acquireCount: { r: 187 } } } flowControl:{ acquireCount: 7, timeAcquiringMicros: 9 } storage:{ data: { bytesRead: 17354199, timeReadingMicros: 1313057 } } protocol:op_msg 1362ms
2022-06-01T09:04:40.075+0800 I REPL [repl-writer-worker-1] applied op: CRUD { ts: “###”, t: “###”, h: “###”, v: “###”, op: “###”, ns: “###”, ui: “###”, wall: “###”, lsid: { id: “###”, uid: “###” }, txnNumber: “###”, stmtId: “###”, prevOpTime: { ts: “###”, t: “###” }, o: { _id: “###”, startDate: “###”, url: “###”, headers: { common: { Accept: “###” }, delete: {}, get: {}, head: {}, post: { Content-Type: “###” }, put: { Content-Type: “###” }, patch: { Content-Type: “###” } }, method: “###”, typeRequest: “###”, requestBody: { domain: “###”, username: “###”, password: “###”, uncFilePath: “###” }, createdOn: “###” } }, took 972ms

Command I run:

mloginfo mongodb.log --queries

mloginfo --version

mtools version 1.7.0 || Python 3.6.8 (default, Aug 13 2020, 07:46:32) [GCC

[Stennie_X (Stennie)]

Hi @Sandie_Zarate1,

Thank you for including the requested details. Unfortunately mtools 1.7.0 does not have support for Enterprise log redaction – many of the expected values have been replaced and regex extraction is failing ungracefully similar to your example.

There is an open issue in the mtools backlog to address this: https://github.com/rueckstiess/mtools/issues/854

Regards,
Stennie