Hello everyone, Im currently working on a hybrid search implementation and one of the requirements is pagination. After looking into the documentation seems likes there’s no way to do it yet. Is there any workaround or suggestion you can make to solve this problem?
Hi @Francisco_Gonzalez_Jara! Thank you for posting, and sorry for the delay in responding.
I am assuming you are following along with the hybrid search tutorial here. May I ask what limit value you are planning on using within your $vectorSearch stage? We have seen <30ms latency for rather large limits (500+) for pure vector search, even against reasonably large datasets (100k-1M vectors) so I suspect that pagination might not strictly necessary. I want to get a better understanding for your anticipated query pattern first before I provide any suggestions.
Hi @Henry_Weller thanks for replying. Within the vector search stage we are planning to use a limit around and 100 to 150 . Sadly pagination is a strong requirement that I have for the hybrid search responses even if the latency is not a problem. Currently, we are trying to find workarounds like paginate the full response after get all the documents at the same time.
Hi Francisco,
You could look at how to use pagination on the $search stage.
yeah I reviewed the documentation, but if im not wrong that would only apply for the fulltext search part of the pipe. Or am I wrong?