Performance issue loading data into MongoDB

Thanks everyone for your replies and insights! It’s been very helpful as I dive into this platform.

I ended up tearing down my whole environment and building a single mongod server on a r5.4xl box (16CPU 128GB of RAM) with a collection with a single index on username/teamnumber/stamp and initially the rate of data ingestion had it completely done in 2.7 days. I recognize that the index will slow things down a bunch but the SQL Server setup has this index on it and to make it an apples-to-apples comparison I wanted the environment to be as similar as possible.

When I woke up this morning the rate of ingestion had slowed to a completion of 4.32 days. Regardless of the completion date I’m going to let this one run its course to be able to manipulate the data set and see how it behaves in its more typical use case. If it runs well /then/ I’ll consider replica sets and sharding to see if I can boost performance.

1 Like