MongoDB 4.0.27 crash

I have an application that uses mongoc/mongocxx for storing customer data. I recently had mongod crash. Here’s a relevant section from the mongod.log.

2022-10-31T16:34:23.151-0400 I NETWORK  [conn703] received client metadata from 127.0.0.1:51710 conn703: { driver: { name: "mongoc / mongocxx", version: "1.18.0-pre / 3.5.1-pre" }, os: { type: "Windows", name: "Windows", version: "6.2 (9200)", architecture: "x86_64" }, platform: "(null)cfg=0x0204170063 CC=MSVC 1926 CFLAGS="/DWIN32 /D_WINDOWS /W3" LDFLAGS="/machine:x64"" }
2022-10-31T16:34:24.417-0400 I COMMAND  [conn701] command DB.EventData_Events command: find { find: "EventData_Events", filter: { time: { $gte: 16666436642813825 }, time: { $lte: 16672484642813825 }, $or: [ { type: "BeginScriptExecution", object: "ExecutionEngine" }, { type: "EndScriptExecution", object: "ExecutionEngine" }, { type: "AbortScriptExecution", object: "ExecutionEngine" } ] }, sort: { time: 1 }, $db: "DB", lsid: { id: UUID("bb478ccd-6474-44d6-b115-eb2ac331c0ee") } } planSummary: IXSCAN { type: 1 }, IXSCAN { type: 1 }, IXSCAN { type: 1 } keysExamined:990 docsExamined:1980 hasSortStage:1 fromMultiPlanner:1 replanned:1 replanReason:"cached plan was less efficient than expected: expected trial execution to take 107 works but it took at least 1070 works" cursorExhausted:1 numYields:102 nreturned:77 reslen:12587 locks:{ Global: { acquireCount: { r: 103 } }, Database: { acquireCount: { r: 103 } }, Collection: { acquireCount: { r: 103 } } } storage:{ data: { bytesRead: 12674484, timeReadingMicros: 87289 } } protocol:op_msg 133ms
2022-11-01T10:43:11.007-0400 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a000032006183'), prevEnd: { $lte: 16604460000000000 }, nextStart: { $gte: 16601364000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("a3fa82c1-13d3-4a7c-97eb-0469b550d569") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:22904105064 keysExamined:16454 docsExamined:807 hasSortStage:1 numYields:136 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 137 } }, Database: { acquireCount: { r: 137 } }, Collection: { acquireCount: { r: 137 } } } storage:{ data: { bytesRead: 6276792, timeReadingMicros: 82753 } } protocol:op_msg 119ms
2022-11-03T20:42:56.497-0400 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a0000320060e2'), prevEnd: { $lte: 16675225763723166 }, nextStart: { $gte: 16673364000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("909964a5-221a-4cd6-9aea-c6d1664ec1cf") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24526763640 keysExamined:576 docsExamined:575 hasSortStage:1 numYields:10 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 11 } }, Database: { acquireCount: { r: 11 } }, Collection: { acquireCount: { r: 11 } } } storage:{ data: { bytesRead: 11460886, timeReadingMicros: 104965 } } protocol:op_msg 122ms
2022-11-04T10:33:49.900-0400 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a000032006183'), prevEnd: { $lte: 16669676080000000 }, nextStart: { $gte: 16666452000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("5d8ece5a-bb02-4e02-ad19-6b872ce20b28") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:22585501155 keysExamined:1906 docsExamined:446 hasSortStage:1 numYields:22 nreturned:101 reslen:413892 locks:{ Global: { acquireCount: { r: 23 } }, Database: { acquireCount: { r: 23 } }, Collection: { acquireCount: { r: 23 } } } storage:{ data: { bytesRead: 10478844, timeReadingMicros: 112739 } } protocol:op_msg 123ms
2022-11-04T10:35:17.264-0400 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a000032006183'), prevEnd: { $lte: 16663628080000000 }, nextStart: { $gte: 16660404000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("5d8ece5a-bb02-4e02-ad19-6b872ce20b28") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24141883352 keysExamined:3107 docsExamined:479 hasSortStage:1 numYields:29 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 30 } }, Database: { acquireCount: { r: 30 } }, Collection: { acquireCount: { r: 30 } } } storage:{ data: { bytesRead: 9330948, timeReadingMicros: 87027 } } protocol:op_msg 112ms
2022-11-07T13:38:00.927-0500 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a0000320060e2'), prevEnd: { $lte: 16678462807352986 }, nextStart: { $gte: 16675344000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("7aea5768-6949-4684-af35-923824fce68e") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:23866781782 keysExamined:963 docsExamined:962 hasSortStage:1 numYields:15 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 16 } }, Database: { acquireCount: { r: 16 } }, Collection: { acquireCount: { r: 16 } } } storage:{ data: { bytesRead: 22359105, timeReadingMicros: 168883 } } protocol:op_msg 191ms
2022-11-07T13:38:14.395-0500 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9616d3a000032006d60'), prevEnd: { $lte: 16678462940239568 }, nextStart: { $gte: 16675344000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("7aea5768-6949-4684-af35-923824fce68e") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24418897787 keysExamined:1928 docsExamined:1927 hasSortStage:1 numYields:32 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 33 } }, Database: { acquireCount: { r: 33 } }, Collection: { acquireCount: { r: 33 } } } storage:{ data: { bytesRead: 44095443, timeReadingMicros: 318462 } } protocol:op_msg 370ms
2022-11-07T13:38:14.678-0500 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9616d3a000032006d64'), prevEnd: { $lte: 16678462945151585 }, nextStart: { $gte: 16675344000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("7aea5768-6949-4684-af35-923824fce68e") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24306548295 keysExamined:958 docsExamined:957 hasSortStage:1 numYields:15 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 16 } }, Database: { acquireCount: { r: 16 } }, Collection: { acquireCount: { r: 16 } } } storage:{ data: { bytesRead: 20625114, timeReadingMicros: 148546 } } protocol:op_msg 162ms
2022-11-07T13:42:22.042-0500 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('62700c91261600001f0a1924'), prevEnd: { $lte: 16678465418985200 }, nextStart: { $gte: 16675344000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("7aea5768-6949-4684-af35-923824fce68e") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24487356266 keysExamined:898 docsExamined:897 hasSortStage:1 numYields:13 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 14 } }, Database: { acquireCount: { r: 14 } }, Collection: { acquireCount: { r: 14 } } } storage:{ data: { bytesRead: 18312795, timeReadingMicros: 125804 } } protocol:op_msg 142ms
2022-11-07T13:43:33.358-0500 I COMMAND  [conn699] command DB.ParameterData_ValueSegments command: find { find: "ParameterData_ValueSegments", filter: { series: ObjectId('624cc9606d3a000032006183'), prevEnd: { $lte: 16678466132266302 }, nextStart: { $gte: 16675344000000000 } }, sort: { prevEnd: 1 }, $db: "DB", lsid: { id: UUID("7aea5768-6949-4684-af35-923824fce68e") } } planSummary: IXSCAN { series: 1, nextStart: 1, prevEnd: 1 } cursorid:24752876176 keysExamined:898 docsExamined:897 hasSortStage:1 numYields:11 nreturned:101 reslen:413908 locks:{ Global: { acquireCount: { r: 12 } }, Database: { acquireCount: { r: 12 } }, Collection: { acquireCount: { r: 12 } } } storage:{ data: { bytesRead: 19010953, timeReadingMicros: 121548 } } protocol:op_msg 130ms
2022-11-07T16:17:34.544-0500 F CONTROL  [thread704] *** unhandled exception 0xC0000006 at 0x00007FF7D360A400, terminating
2022-11-07T16:17:34.544-0500 F CONTROL  [thread704] *** stack trace for unhandled exception:
2022-11-07T16:17:34.916-0500 I -        [thread704] mongod.exe    ...\src\third_party\wiredtiger\src\btree\bt_discard.c(366)     __free_skip_array+0x70
mongod.exe    ...\src\third_party\wiredtiger\src\btree\bt_discard.c(202)     __free_page_modify+0x131
mongod.exe    ...\src\third_party\wiredtiger\src\btree\bt_discard.c(108)     __wt_page_out+0x17c
mongod.exe    ...\src\third_party\wiredtiger\src\btree\bt_split.c(2078)      __split_multi+0x28a
mongod.exe    ...\src\third_party\wiredtiger\src\btree\bt_split.c(2121)      __wt_split_multi+0x18a
mongod.exe    ...\src\third_party\wiredtiger\src\evict\evict_page.c(383)     __evict_page_dirty_update+0x1d5
mongod.exe    ...\src\third_party\wiredtiger\src\evict\evict_page.c(192)     __wt_evict+0x26a
mongod.exe    ...\src\third_party\wiredtiger\src\evict\evict_lru.c(2207)     __evict_page+0x2f7
mongod.exe    ...\src\third_party\wiredtiger\src\evict\evict_lru.c(1125)     __evict_lru_pages+0x100
mongod.exe    ...\src\third_party\wiredtiger\src\evict\evict_lru.c(314)      __wt_evict_thread_run+0x162
mongod.exe    ...\src\third_party\wiredtiger\src\support\thread_group.c(31)  __thread_run+0x4d
ucrtbase.dll                                                                 o_exp+0x5a
KERNEL32.DLL                                                                 BaseThreadInitThunk+0x14
2022-11-07T16:17:34.917-0500 I CONTROL  [thread704] failed to open minidump file C:\Program Files\MongoDB\Server\4.0\bin\mongod.2022-11-07T21-17-34.mdmp : Access is denied.

You seem to have a pretty old server that uses a “pre” release of the driver:

{ driver: { name: "mongoc / mongocxx", version: "1.18.0-pre / 3.5.1-pre" }

is there any chance to upgrade the server, at least within its 4.xx family?

Hi @Jimmy_Harrington welcome to the community!

In addition to what @Yilmaz_Durmaz said, I would also add that MongoDB 4.0 series is out of support and will not receive any bugfixes anymore, so even if we get to the bottom of this, it’s either 1) not going to get fixed, and 2) likely to already fixed in newer versions if it’s crashing the server.

Please upgrade to a supported version (4.2.23, 4.4.17, 5.0.13, or 6.0.2), and if this happens again, we would be very interested in replicating this issue :slight_smile:

Best regards
Kevin

2 Likes

Thanks for the replies, I’ll look into upgrading Mongo and I’ll make a new post if I get the same crash again.