MongoDB-3.4 large collection insert error and leads to server shutdown

Hi,
This MongoDB was running on Windows server 2012(local SSD drive w/ 2T space) and suddenly an insert to a collection with 500M+ records failed with below error. The collection-4-8017886371145150964.wt file size is about 383G. And DB server can no longer be restarted(w/ similar error message upon restart attempt). Is this a windows file system issue or a MongoDB cfg issue?
Thanks
Herbert
================= error log ==================
2021-10-29T02:00:04.546-0400 E STORAGE [thread2] WiredTiger error (0) [1635487204:542272][1572:140721273901184], file:collection-4-8017886371145150964.wt, WT_SESSION.checkpoint: e:\data\db\collection-4-8017886371145150964.wt: handle-write: WriteFile: failed to write 12288 bytes at offset 393192337408: The requested operation could not be completed due to a file system limitation

2021-10-29T02:00:04.547-0400 E STORAGE [thread2] WiredTiger error (-31802) [1635487204:547277][1572:140721273901184], checkpoint-server: checkpoint server error: WT_ERROR: non-specific WiredTiger error
2021-10-29T02:00:04.547-0400 E STORAGE [thread2] WiredTiger error (-31804) [1635487204:547277][1572:140721273901184], checkpoint-server: the process must exit and restart: WT_PANIC: WiredTiger library panic

The error The requested operation could not be completed due to a file system limitation in general seems to be related to NTFS compression. Do you have that enabled ?

Similar reference made in this JIRA:
https://jira.mongodb.org/browse/SERVER-32808

1 Like

No. I looked at the drive properties and the “Compress” box is NOT checked. But this article mentioned the defrag limit with large file and it could be related here in some non-compress way…

Found the workaround. The windows contig utility reveals the .wt file has 1.6m fragments which hits NTFS limit. Defrag it and DB server now back to normal. Does MongoDB always map a large collection to a single OS file?

Hi @herbert_wu, we’ve just run across this same problem. May I ask how long it took for you to defrag the one file? I’m defragmenting it with defraggler but it has been stuck at 0% for an hour now.

It seems that our disk is very fragmented (97%). It sucks because our game server has now been down for the past 12 hours and counting and I’m not sure if this will resolve the issue.

Does anyone have any other tips/suggestions for this issue?