hello everyone, had anyone faced this issue and how did you solve it?
:Request too large for gpt-4o-mini in organization org-Az40ITepvbTVuXvxbwmppFx9 on tokens per min (TPM): Limit 200000, Requested 557208. The input or output tokens must be reduced in order to run successfully.
Basically we have a data base with over 3000 products and when we ask about certain brand it just gives this error. is there a workaround this issue.
Hi @Nurmukhammed_Turlin! welcome to community!
You can use vector database and use OpenAI embedding to read the whole database instead of reading all over 3k products.
Anyway, the error was encountered because of limitations of OpenAI tokens per minute.
You can read here for find out about TPM and OpenAI related.
so by using vector database GPT can read the whole DB instantly instead of going through each row? Am i right?
Yes, the GPT will “Mapping your whole DB” and will give the “possible nearest” answer to your question.
Anyway, if it helps, please mark as solution (you can just like my replied messaage too^^)
LLMs have a context limit for the amount of data you want to pass in. Google’s Gemini can take up to a million tokens, but your current use case still sounds highly inefficient. You’ll probably want to implement a RAG system so your agent can pull only the information it needs using a concept called Semantic search. Have a look at the video below especially the demo section to see if this will resolve your use case
