Mongorestore indexes error socket was unexpectedly closed: EOF

Working with Data Developer Tools
,
1

I have 1 mongodb cluster im am backing up and attempting to restore into another mongodb cluster. The data seems to all restore successfully, but we are having a problem with restoring the indexes.

2023-04-28T12:48:43.064+0000    Failed: <redacted>.games: error creating indexes for redacted.games: createIndex error: connection(mongo.redacted-test.com:27017[-6]) socket was unexpectedly closed: EOF

dump
mongodb: v5

mongodump --host=mongo.<redacted>.com --port=27017 --authenticationDatabase="admin" -u="admin" -p="<redacted>" --gzip --out=new

restore
mongodb: v6

 mongorestore --verbose --host=mongo.redacted.com --port=27017 --convertLegacyIndexes --authenticationDatabase="admin" -u="databaseAdmin" -p="<redacted>" --gzip ./new/
3

Hi @Kay_Khan and welcome to MongoDB community forums!!

Based on the above posts, it seems you are trying to do a mongodump and mongorestore between version 5.0 and 6.0
I tried to replicate the issue in my local environment with a sample dataset between the latest patch version for version 5.0 and version 6.0 which are version 5.0.17 and version 6.0.5 respectively.

I tried the following mongodump and mongorestore with index being created in the collection.

  1. Sample data:
est> db.sample.findOne()
{
  _id: ObjectId("6450e271f1a41838152c7349"),
  DaysSinceLastSale: 96,
  IsResidential: true,
  Owner1NameFull: 'Adrian Griffith',
  AddressCity: 'Oakland',
  AddressState: 'Oregon',
  AddressZIP: 890484
}
  1. Indexes defined.
test> db.sample.getIndexes()
[
  { v: 2, key: { _id: 1 }, name: '_id_' },
  { v: 2, key: { Owner1NameFull: 1 }, name: 'Owner1NameFull_1' }
]
  1. Mongodump:

mongodump --db=test --collection=sample --gzip --out=new

  1. Mongorestore

mongorestore --db=test --collection=sampleNew --gzip new/test/sample.bson.gz --port 27018

Please note that featureCompatibilityVersion for both the version are set to 5.0 and 6.0 respectively.

If you are still following issues could you help me with the folloing information:

  1. The exact versions you are using between mongodump and mongorestore.
  2. Is the connection successfully established during the process of mongodump and restore.
  3. The feature Compatibility version for both the version.
  4. Are you upgrading the versions manually, if yes, could you help me with the steps or documentation followed for the upgrade.
  5. Are you able to create index in the restored collection.

Regards
Aasawari

4

H thank you for the response,

Your example is quite small. If we are dealing with gb’s in size of data. The index takes long to complete and maybe thats why we are getting an error socket was unexpectedly closed ? What would be the solution to this problem?

5

Hi @Kay_Khan

Based on the error logs provided, I tried to replicate the issue in my local environment.

  1. Firstly, I tested the scenario with a dataset for ~ 3 GB and tried to follow the steps mentioned above with a change during the mongorestore.
    While the restore was creating the index, I tried to kill the mongod process using kill -9 <PID> and I could see the similar error logs as:
2023-05-04T12:33:42.648+0530	Failed: test.samplerestore2: error creating indexes for test.samplerestore2: createIndex error: connection(localhost:27018[-7]) socket was unexpectedly closed: EOF
2023-05-04T12:33:42.648+0530	24000000 document(s) restored successfully. 0 document(s) failed to restore.

Can you confirm if there has been any termination in the mongod process while the index was restored.

  1. I tried to replicate using the latest patch versions of mongoDB (5.0.17 and 6.0.5), could you help with the versions of mongod, mongodump and mongorestore ?

  2. If you find that the mongod process was shut down unexpectedly by the Linux OOMkiller, you might be seeing the effect of SERVER-68125: Index build on multi-key fields can consume more memory than limit especially if you have a lot of data and multi-key indexes. This was fixed in MongoDB 6.0.4, so if you’re not using the latest version, please upgrade to the latest one for bug fixes and improvements

  3. Lastly, this issue mainly would effect on the large collections and one possible workaround we could suggest would be to pre-create the collection and indexes before mongorestore. This would avoid large index building process.
    Also, --drop parameter should not be used in the mongorestore command if you decide to pre-create the collection and indexes.

Let us know if you have any further questions.

Regards
Aasawari