Mistral AI Model Retrieves Correct Answer but Does Not Forward Output in RAG Chatbot

Questions
,
1

Introduction

We built a custom RAG chatbot using Mistral based on this n8n blog post and the provided workflow template.

Workflow Overview

Our chatbot follows these steps:

  1. Retrieves files from a folder.
  2. Stores the extracted information in a Pinecone Vector Store.
  3. Uses an AI Agent to process queries and return relevant answers.

File Processing Stage:

image
image1016Ă—604 51.3 KB

Chat Stage:

image
image706Ă—734 62.3 KB

Configuration Details

Disclaimer: Only the French part is put inside the n8n config.

System Message (AI Agent):

FR
Tu es un assistant virtuel qui donne des réponses aux questions concernant le fonctionnement du logiciel SOFTWARE_NAME développé par COMPANY_NAME en te basant exclusivement sur la documentation interne auquel tu as accès par le biais d'un outil.

EN
You are a virtual assistant who provides answers to questions about the operation of the SOFTWARE_NAME software developed by COMPANY_NAME, based exclusively on the internal documentation to which you have access via a tool.

Vector Store Tool - Description of Data:

FR
Utilise cet outil pour obtenir des informations Ă  propos de la documentation interne de SOFTWARE_NAME.
Cette base de données contient la documentation interne de SOFTWARE_NAME.

EN
Use this tool to obtain information about SOFTWARE_NAME's internal documentation.
This database contains SOFTWARE_NAME's internal documentation.

Issue

Expected Behavior

The retrieved answer should be returned to the user as output, but it appears to be lost within the process.

Questions & Debugging Attempts

  • We’ve checked the AI agent and vector store tool configurations.
  • No errors appear in the logs.
  • The retrieved answer is visible within the execution flow, but it’s not passed forward. As if the Tool: part of the input was ignored ?

Similar issues

Request for Support

Why is the response not being forwarded to the output? Are there any known issues with how n8n processes AI agent outputs in this scenario?

Workflow

Debug info

core

  • n8nVersion: 1.85.4
  • platform: docker (self-hosted)
  • nodeJsVersion: 20.18.3
  • database: postgres
  • executionMode: scaling
  • concurrency: -1
  • license: community
  • consumerId: unknown

storage

  • success: all
  • error: all
  • progress: false
  • manual: true
  • binaryMode: memory

pruning

  • enabled: true
  • maxAge: 336 hours
  • maxCount: 10000 executions

client

  • userAgent: mozilla/5.0 (windows nt 10.0; win64; x64; rv:136.0) gecko/20100101 firefox/136.0
  • isTouchDevice: false

Generated at: 2025-04-01T13:23:50.344Z

2

seems a reply based on system prompt. Check user prompt field on ai node, it think it gets an empty user prompt.

In logs you should see 1 system prompt (you have) AND user prompt. In your case is a chatInput “Quelles sont les 3 operacions de productions à reinsegner?”
To check if the error is what i’m saying, try to hardcode the user prompt adding it on AI Agent node in place of ChatInput

3

Hey there, I don’t follow you, I can see the chatInput in my two last screenshots of the initial AI Cloud Chat Model and in the one that retrieves the informations from the database.

I just tried hard-coding it with no changes :

image
image507Ă—372 22.3 KB

I tried without system message, same issue.

4

add your workflow here using </> on the wysiwyg editor, i’ll try to understand the problem

5

Done, I edited the original post. Thanks for taking the time!

6

it works for me… I added a single file on pinecone store bypassing splitout and batch upload, but no matter becouse the problen is in the chat.

I used mistral-12b not the pixtral-small. You have tried other models? I can’t say more :frowning:
your workflow is ok

1 Like
7

I’m using the mistral-small-latest model which supposedly have tools working.

I will try another model tomorrow.

It works with pixtral-12b… I don’t get it. Why. There must be a bug on n8n for the model to not use the answer appropriately because this is exactly the same input he gets and it works fine as per Mistral’s documentation.

1 Like
8

I have the exact same issue. A working workaround is to append this to your system prompt:

IMPORTANT: Your full answer needs to be prefixed with "AI:" - otherwise it will be rejected.

My thought process was that the LLM runs multiple times - on the last execution, the input is your prompt plus the tool output appended (e.g. “Tool: ”).

The LLM then feels like this is a sufficient answer for the question and that it’s unlikely for more information to be added on top of that, but is also forced to generate more than 0 tokens, so it adds some stupid stuff like “let me know if there is anything else I can do to help”.

Still feels a bit like it’s a bug in n8n that it supplies the input in this way to mistral since this is bound to lead to errors and apparently works more often than not unless you specifically circumvent it in your prompt.

Disadvantage of the workaround is that you need to remove the AI prefix afterwards for example by using a regex replace.

I assume it might work without the workaround in more complex calls where the rag call isn’t the only necessary tool use and the RAG response isn’t the whole response to the user query.

1 Like
9

Hi, thanks for the workaround !

Is there a different behavior using ChatGPT ? Maybe they should redirect the AI output the same way it’s done using ChatGPT if it works better.

I’ll take a look, but if anyone find an issue or pull-request on n8n repository, let’s link it here. Otherwise it might be usefull to create one on our own !

10

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.