Performance improvement when changing from M0 to M2?

MongoDB Atlas
1

Hi, I’m considering switching my free cluster to a M2 cluster for my college project, since it’s pretty affordable. Some big queries I have to make take ~20 seconds, and I would like to know if I could expect that kind of performance to improve. Storage is not a problem.

Kind regards

2

Possibly.

By upgrading to M2 you do get more throughput allowance. However, if you have not optimised your indexes to support your queries then you may not see improvement my upgrading the cluster.

https://www.mongodb.com/docs/atlas/reference/free-shared-limitations/#service-m0–free-cluster—m2–and-m5-limitations

Throughput

M0 free clusters and M2/M5 shared clusters limit the number of read and write operations per second. The rate limits vary by cluster tier as follows:

M0: 100 operations per second

M2: 200 operations per second

M5: 500 operations per second

Atlas handles clusters that exceed the operations per second rate limit as follows:

Atlas throttles the network speed of the cluster.

Atlas triggers a one second cooldown period before resuming the cluster's operations on a given connection. If the queue is greater than the operations per second limit, operations might wait for more than a second in the queue.

If the number of operations per second drops below the rate limit threshold, Atlas resumes processing of the queued operations on each connection before processing any new operations on that connection.
3

Would it be fine to use indexes if I always pull the full documents out as opposed to particular fields? I don’t do any mass insertions, so I don’t think the associated slowdowns in insertions would matter…

4

Indexes will speed up locating the documents you are interested in.

Without a supporting index the query will execute a collection scan reading every single document to find ones that match, this is slow.

Use explain on a query, if you see COLLSCAN there is indexing to be done.

https://www.mongodb.com/docs/manual/applications/indexes/

5

OK, I created an index for a collection that has the fields “date” and “user_id”, which is what I use to get back objects filtering by those three fields. I didn’t see any change in performance however. Still around ~20 seconds.

Should I try M2?

6

I doubt the index matches the query then. As mentioned look at the explain output.

If you share the query shape then it is likely the community can suggest a performant index.

Its your money, but don’t expect any great increses.

7

I wrote it in Java but essentially I’m performing an operation that checks that the user id is “equal” to the one provided and that the “date” field is between the first and last day of one year (I used the lt and gte operators for this). I’ll try using explain to see if there is any useful feedback

8

I get this output:

Document{{explainVersion=1, queryPlanner=Document{{namespace=rumpel.bills, indexFilterSet=false, parsedQuery=Document{{$and=[Document{{user_id=Document{{$eq=65429159fa69510fb71ea91a}}}}, Document{{date=Document{{$lt=Mon Jan 01 00:00:00 CET 2024}}}}, Document{{date=Document{{$gte=Sun Jan 01 00:00:00 CET 2023}}}}]}}, queryHash=63D239C1, planCacheKey=BF30D454, maxIndexedOrSolutionsReached=false, maxIndexedAndSolutionsReached=false, maxScansToExplodeReached=false, winningPlan=Document{{stage=FETCH, inputStage=Document{{stage=IXSCAN, keyPattern=Document{{date=1, user_id=1}}, indexName=date_1_user_id_1, isMultiKey=false, multiKeyPaths=Document{{date=, user_id=}}, isUnique=false, isSparse=false, isPartial=false, indexVersion=2, direction=forward, indexBounds=Document{{date=[[new Date(1672527600000), new Date(1704063600000))], user_id=[[ObjectId(‘65429159fa69510fb71ea91a’), ObjectId(‘65429159fa69510fb71ea91a’)]]}}}}}}, rejectedPlans=}}, executionStats=Document{{executionSuccess=true, nReturned=78, executionTimeMillis=0, totalKeysExamined=78, totalDocsExamined=78, executionStages=Document{{stage=FETCH, nReturned=78, executionTimeMillisEstimate=0, works=79, advanced=78, needTime=0, needYield=0, saveState=0, restoreState=0, isEOF=1, docsExamined=78, alreadyHasObj=0, inputStage=Document{{stage=IXSCAN, nReturned=78, executionTimeMillisEstimate=0, works=79, advanced=78, needTime=0, needYield=0, saveState=0, restoreState=0, isEOF=1, keyPattern=Document{{date=1, user_id=1}}, indexName=date_1_user_id_1, isMultiKey=false, multiKeyPaths=Document{{date=, user_id=}}, isUnique=false, isSparse=false, isPartial=false, indexVersion=2, direction=forward, indexBounds=Document{{date=[[new Date(1672527600000), new Date(1704063600000))], user_id=[[ObjectId(‘65429159fa69510fb71ea91a’), ObjectId(‘65429159fa69510fb71ea91a’)]]}}, keysExamined=78, seeks=1, dupsTested=0, dupsDropped=0}}}}, allPlansExecution=}}, command=Document{{find=bills, filter=Document{{$and=[Document{{user_id=65429159fa69510fb71ea91a}}, Document{{$and=[Document{{date=Document{{$gte=Sun Jan 01 00:00:00 CET 2023}}}}, Document{{date=Document{{$lt=Mon Jan 01 00:00:00 CET 2024}}}}]}}]}}, $db=rumpel}}, serverInfo=Document{{host=ac-p0ypvb1-shard-00-02.t1nl1se.mongodb.net, port=27017, version=6.0.12, gitVersion=21e6e8e11a45dfbdb7ca6cf95fa8c5f859e2b118}}, serverParameters=Document{{internalQueryFacetBufferSizeBytes=104857600, internalQueryFacetMaxOutputDocSizeBytes=104857600, internalLookupStageIntermediateDocumentMaxSizeBytes=16793600, internalDocumentSourceGroupMaxMemoryBytes=104857600, internalQueryMaxBlockingSortMemoryUsageBytes=33554432, internalQueryProhibitBlockingMergeOnMongoS=0, internalQueryMaxAddToSetBytes=104857600, internalDocumentSourceSetWindowFieldsMaxMemoryBytes=104857600}}, ok=1.0, $clusterTime=Document{{clusterTime=Timestamp{value=7308432186141573123, seconds=1701626970, inc=3}, signature=Document{{hash=org.bson.types.Binary@98822d9d, keyId=7278309952702119939}}}}, operationTime=Timestamp{value=7308432186141573123, seconds=1701626970, inc=3}}}

9

This is really good, the execution time is 0 millis and keys:docs:returned ratio is perfect:

nReturned=78,
executionTimeMillis=0,
totalKeysExamined=78,
totalDocsExamined=78,

Following the ESR “rule” a better index would be {user_id:1, date:1}

I think this could be down to data transfer from server to client. You could try enabling a compressor on the connection string and see if that helps. &compressors=zstd,snappy,zlib for example.

https://www.mongodb.com/docs/manual/tutorial/equality-sort-range-rule/#the-esr–equality–sort–range–rule
https://www.mongodb.com/docs/drivers/java/sync/v4.3/fundamentals/connection/network-compression/#specify-compression-algorithms