AI Agent + RAG = not accurate answers

Describe the problem/error/question

I am building an AI agent using tools to answer questions about an ecommerce. The agent consumes diferent Excel files where the products are listed by category (one tool consumes the whisky Excel, another tool consumes the wine tool, etc…).

It turns out that the agent fail to respond easy questions like “What is the more expensive wine you have?” giving not always the right answers.

I switched to vector database using embeddings but no difference, the agent still failing with similar questions.

Is this just something expected since the AI is not deterministic? or am I doing something wrong in my setup?

Information on your n8n setup

  • n8n version: 1.86
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Maybe your prompt needs fine tuning or maybe version of AI you are using is not powerful enough. For prompt try something like this:

DOCUMENT/(database):
(document text)
QUESTION:
(users question)

INSTRUCTIONS:
Answer the users QUESTION using the DOCUMENT text above.
Keep your answer ground in the facts of the DOCUMENT.
If the DOCUMENT doesn’t contain the facts to answer the QUESTION return {NONE}

Of course there are many different prompts you could try… Prompting him be to take documents data as facts, or using words like exact, explicit, accurate, precise, specific etc.

1 Like

Just in case somebody is intererested on this topic, I learnt that RAG is not helpful in use cases where we need to some calculations (min, max, sum, avg, etc…) along the data spread in diferent chunks.

For that purpose is better to use an hybrid approach, leaving RAG for the semantic searches and pure SQL queries for such calculations.

This is a very good example about how to achieve such hybrid solution: https://www.youtube.com/watch?v=mQt1hOjBH9o&t=1557s

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.