Ch. 2, Ticket: Paging - unreliable test result caused by search ordering

[John_Underwood]

Expected behaviour:
The pytest function ‘test_supports_paging_text(client)’ should reliably pass/fail for the same code.

Actual behaviour:
The test somtimes passes and sometimes fails for the same code

Workaround:
Continue to refresh status page in mflix application until I get a token for the paging test. This token was then accepted by the quiz.

Observations:
The test reliably fails at "assert movies2[0].get(‘title’) == “Ugetsu”

Diagnosis:
This is cause by search scoring being the same for certain results and therefore the ordering of results with the same score being random (see screenshot of tests in Jupyter)

[John_Underwood]

As a little further evidence, here is a screenshot of the test passing 3 times and failing once. The reader will have to trust me that I did not change the code in between

[steevej (Steeve Juneau)]

You are right. See https://www.mongodb.com/community/forums/t/single-fetch-with-skip-and-limit-returns-unexpected-result/119205/8?u=steevej for an explication and solution.

[John_Underwood]

Thanks Steeve for the link with the fuller explanation. I had also thought about adding an additional sort field after the score. But of course for this to fix my issue with the pytest I would have to make some changes to the “build_query_sort_project()” code to incorporate this, which I think is a bit outside the scope of the original exercise, but could be done fairly easily.

[steevej (Steeve Juneau)]

@John_Underwood

I think is a bit outside the scope of the original exercise

I think too. I am sure the university team will adjust the test eventually.

[Slawomir_Gozdzielewski]

Unfortunately it is still not fixed. I had exactly the same problem. Few times I re-run the test in UI and eventually I got validation code :D. But it does not seem very reliable test.