OpenAI RAG with Mongo Atlas Max_Token error

Hi, I tried the code over in this tutorial: RAG with Atlas Vector Search, LangChain, and OpenAI | MongoDB But I keep on experiencing error whenever I run

openai.BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 4097 tokens, however you requested 7013 tokens (5013 in your prompt; 2000 for the completion). Please reduce your prompt; or completion length.", 'type': 'invalid_request_error', 'param': None, 'code': None}}``

Hey @Jonathan_Natakusuma - That error represents more no of tokens being passed to the LLM than its allowed context window. Can you share what query you are passing on to this?
cc @Harshad_Dhavale

1 Like

I’ve uploaded a total of 5 documents (2 extra documents to the one given in this example). What I found is that if the sum of all the documents’ words >1000ish it becomes problematic and I get this error. As soon as I delete some of these documents such that the number of words in aggregate is < 1000ish it works perfectly again. For my work I need 100s of documents. How can I do so while not triggering the max_token error?