AI Agent giving wrong answears

Questions
, , ,
1

Hello n8n Community,

I’m new to n8n and exploring its capabilities—it seems like an incredible tool! However, I’ve run into some challenges while using the AI Agent for querying my data.

Here’s what I’m trying to do:
I have an AI Agent (using the OpenAI gpt-4-o-mini model) set up in a workflow triggered by a Chat Input. The AI Agent is connected to a database tool (I’ve tested both BigQuery and Supabase Postgres). My goal is to query a table and retrieve two columns: event_date and user_id.

The AI Agent successfully queries the database. However, when I ask questions like “What’s the count of unique users identified?”, it provides incorrect and inconsistent answers.

Here’s what I’ve tried so far:

  • Switching models (DeepSeek, Gemini, etc.)
  • Adjusting the system prompt in various ways
  • Experimenting with different invocation methods
  • Using community templates

Unfortunately, none of these approaches have resolved the issue. Sometimes, even with a small dataset, the responses remain inaccurate.

Here’s an image of the workflow setup I’m using:

simple_workflow_n8n
simple_workflow_n8n1295×747 83.7 KB

Since I’m still learning both n8n and working with LLMs, I’m unsure if I’m approaching this the right way. Should I avoid using the AI Agent for this type of task, given that n8n has built-in aggregation nodes for such operations?

Any guidance or advice on what I might be doing wrong—or how to get accurate results—would be greatly appreciated.

Thank you for your help!

Setup Details:

  • n8n version: 1.95.3
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app): n8n cloud
  • Operating system: GNU/Linux 6.11.0-26 x86_64
2

Generally LLMs are really bad at math. This stems from technical implementation of transformer based language models. If you want to learn more, here’s amazing (albeit long) video on how it’s implemented technically: https://www.youtube.com/watch?v=7xTGNNLPyMI

I don’t know how your executeQuery input looks like. Does LLM have full right to generate query? If so, can you paste what queries it generated for each of differing results? Because if it has full access to the query, it created wrong query. If query is rather constant, it could just hallucinate the answer. Especially if you give it task “count X in text” - this is too generic for LLM, especially if you give high temperature - it gives more room for non-deterministic answer.

3

Thanks for the response, raph_kai.

The query is fixed and returns the same data every run. I am using model temperature 0.2. Here is the tool settings:

query-postgres
query-postgres397×849 38.7 KB

This should return the columns: event_date, user_pseudo_id.

Is there a way to avoid hallucination for this task or should I use other node to perform this operation?

4

Oh, so yeah, you ask LLM to do math, which it really sucks at. To avoid hallucinations, you have to be more specific. If you want to count all data, execute:

SELECT COUNT(1) FROM ga_data;

COUNT(1) instead of COUNT(*) is just a best practice concerning DB data retrieval - it’s more efficient (it’s interesting topic how the SQL internals work, maybe for another day).

if you want to add some conditions:

SELECT COUNT(1) FROM ga_data WHERE user_pseudo_id = 1; -- or other id you wish

No matter what you do: please, please don’t allow user of your chatbot define queries, the same with AI. It can assist with some parts, but there’s always a risk of injecting SQL query and unauthorized data access/modification.