Describe the problem/error/question
I am running the “Demo: RAG in n8n” workflow. When I start running,
-
It show file upload screen and I uploaded PDF file.
-
“Load Data Flow” part seems to be executing fine.
-
When I ask a question, it does not use the Data Store.
It gives some random result or sometimes say the information is not available.
Can someone please help me and let me know what I am doing wrong.
What is the error message (if any)?
It gives a result like “[get_weather] {“location”:“Paris”} [END_TOOL_RESULT]”
Please share your workflow
Share the output returned by the last node
[get_weather]
{“location”:“Paris”}
[END_TOOL_RESULT]
Information on your n8n setup
- n8n version: 1.110.1
- Database (default: SQLite): Default
- n8n EXECUTIONS_PROCESS setting (default: own, main): (Not sure where to find this)
- Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
- Operating system: MacOS Sequoia 15.5
Hey!
I think that you have to specify in the system prompt, the rules on when/what the agent should do with the information 
Thank you for the reply. I tried giving some System Instructions like “You are a helpful assistant.
Answer questions using data from Data Tool.” But still the same result.
When making a RAG Agent in n8n, you should focus on the system prompt of the agent node as it is the essential node in the workflow. Having a detailed and clear system prompt will allow the AI model to work as you intended. Here is a starter system prompt that you can customize and tailor to your needs afterwards:
# RAG Agent System Prompt
You are a RAG Agent that retrieves information using the 'Data Query Tool' before responding to queries.
## Core Behavior
- **Always query first**: Use Data Query Tool for any factual question before generating responses
- **Prioritize retrieved data**: Retrieved information takes precedence over pre-trained knowledge
- **Be transparent**: Clearly indicate when information comes from queries vs. general knowledge
## Tool Usage
**Data Query Tool Protocol:**
1. Identify what information you need
2. Create specific, targeted queries
3. Use multiple queries for complex topics
4. Refine queries if initial results are poor
**When to Query:**
- Factual questions, current events, statistics
- Domain-specific information (technical, business, scientific)
- Before making claims about specific entities
- When users request current/specific data
## Response Structure
1. Query relevant data using Data Query Tool
2. Lead response with retrieved information
3. Add analysis and context
4. Note any data limitations or gaps
## Quality Standards
- Cross-reference through multiple queries when possible
- Prioritize recent information
- Acknowledge when queries fail or return insufficient data
- Use general knowledge only as fallback with clear disclaimers
**Template**: "Based on retrieved data: [findings]. [Analysis]. [Limitations if any]"
Remember: Query first, then reason and respond with the most current available information.
@Mookie_Lian Thank you for the prompt. I will use this and see.
Here you can find more details about prompting:
It helped me a lot to understand the underlying structure of AI prompting.
Have fun!
Thank you @Parintele_Damaskin I will check the Prompting guide, it will be very useful.
Thank you all for your help. With your guidance I managed to find out the issue I faced with my workflow. Since I am running locally I used google/gemma-3-1b model. Looks like it does not use the tool to get the data properly. I downloaded openai/gpt-oss-29b model and tested with proper prompts as you have guided and it worked. Then I downloaded google/gemma-3n-e4b model, and it works with that too.
Thank you again for the guidance, it helped me to find out the solution.
Glad that you make it work.
BTW…. assigningsolution to yourself is a bit greedy 
.
Thank you @Parintele_Damaskin. Sorry about marking as solution, I didn’t think it that way. I marked it because it solved my issue and anyone having the issue can fix it that way. I will not do that in future. Thanks.
