I’m building a chatbot that uses RAG (retrieval-augmented generation) to answer user questions based on vector database documents. I noticed that even though my system prompt instructs the LLM to always search the vector DB before answering, sometimes the tool is not executed, and the assistant responds without using the relevant data.
Is there a way to enforce a vector DB tool execution (like a RAG step) when using an LLM in n8n?
Or should I always include the vector search as an explicit node in the workflow, and pass the query to it, to guarantee it’s executed 100% of the time?
The system prompt should be able to force it to always use the tool. Generative AI is still pretty new, so it can be difficult to get it to work in practice. Here’s a few thoughts.
Use a section called ##rules. Tell the agent to always use the tool.
Make sure you include the tool messaging as close to the beginning of the prompt as you can. So it doesn’t try to do something else, and never gets to the tool.
Is there a way to enforce a vector DB tool execution (like a RAG step) when using an LLM in n8n?