Ch. 2, Ticket: Paging - unreliable test result caused by search ordering

Expected behaviour:
The pytest function ‘test_supports_paging_text(client)’ should reliably pass/fail for the same code.

Actual behaviour:
The test somtimes passes and sometimes fails for the same code

Workaround:
Continue to refresh status page in mflix application until I get a token for the paging test. This token was then accepted by the quiz.

Observations:
The test reliably fails at "assert movies2[0].get(‘title’) == “Ugetsu”

Diagnosis:
This is cause by search scoring being the same for certain results and therefore the ordering of results with the same score being random (see screenshot of tests in Jupyter)

As a little further evidence, here is a screenshot of the test passing 3 times and failing once. The reader will have to trust me that I did not change the code in between :slight_smile:

You are right. See https://www.mongodb.com/community/forums/t/single-fetch-with-skip-and-limit-returns-unexpected-result/119205/8?u=steevej for an explication and solution.

Thanks Steeve for the link with the fuller explanation. I had also thought about adding an additional sort field after the score. But of course for this to fix my issue with the pytest I would have to make some changes to the “build_query_sort_project()” code to incorporate this, which I think is a bit outside the scope of the original exercise, but could be done fairly easily.

I think too. I am sure the university team will adjust the test eventually.

Unfortunately it is still not fixed. I had exactly the same problem. Few times I re-run the test in UI and eventually I got validation code :D. But it does not seem very reliable test.