I am building an AI agent using tools to answer questions about an ecommerce. The agent consumes diferent Excel files where the products are listed by category (one tool consumes the whisky Excel, another tool consumes the wine tool, etc…).
It turns out that the agent fail to respond easy questions like “What is the more expensive wine you have?” giving not always the right answers.
I switched to vector database using embeddings but no difference, the agent still failing with similar questions.
Is this just something expected since the AI is not deterministic? or am I doing something wrong in my setup?
INSTRUCTIONS:
Answer the users QUESTION using the DOCUMENT text above.
Keep your answer ground in the facts of the DOCUMENT.
If the DOCUMENT doesn’t contain the facts to answer the QUESTION return {NONE}
Of course there are many different prompts you could try… Prompting him be to take documents data as facts, or using words like exact, explicit, accurate, precise, specific etc.
Just in case somebody is intererested on this topic, I learnt that RAG is not helpful in use cases where we need to some calculations (min, max, sum, avg, etc…) along the data spread in diferent chunks.
For that purpose is better to use an hybrid approach, leaving RAG for the semantic searches and pure SQL queries for such calculations.