Occasionally, my AI agent (RAG Agent) will ignore running my “Query Search Tool”. This will produce bad outputs. If it doesn’t decide to run the tool, it will only look in it’s memory to answer the question. How can I ensure that my agent will look into the memory and also always run the search tool.
I’ve explicitly put in the prompt to always use that tool and is currently using GPT-4O as the LLM model. I was previously using GPT 4.1 (which seemed to ignore my instructions more often)
There is no error message, my workflow will always run. The issue is that it neglects using the tool.
Please share your workflow
(Please ignore the if statement)
Share the output returned by the last node
The last node is typically outputting my company’s information (thus I cannot share it). The output is fine except the information pulled is wrong because it does not call the proper search tool.
Hey @bensont_Norooz , GPT-5 models are specially good at using tools and their prices are super competitive. Take a look at gpt-5, gpt-5-mini and gpt-5-nano.
They are also excellent at following instructions.
I’ve designed a framework for troubleshooting/improving AI Agents that I teach for my students:
Hello my friend, I hope my answer is useful and helps you! I do it in a very simple way: in the prompt I specify that it should check information from the tool (I modify the tool’s name with its function), and I instruct the agent to perform that action always by checking information from the node (tool). Hugs!
well these problems usually gets solved by system prompt tweaking.
try simplifying system prompt:
You are an internal document expert answering staff queries about policies, protocols, insurance, and procedures.
**MANDATORY**:
1. ALWAYS call the "Query Search Tool" FIRST for EVERY query, without exception.
2. NEVER answer from memory, prior conversations, or general knowledge.
3. Only use results from the Query Search Tool. If needed, call "List All Documents" or "Get File Content" AFTER the Query Search Tool.
4. If Query Search Tool returns no results, respond EXACTLY: "Sorry, I couldn’t find any relevant information to answer that."
**Output Format**:
- Summarize or quote the Query Search Tool results.
- Cite the source as: Source: [document_name] ([section, if available]).
- If no relevant information, use the exact apology message above.
