Hi
Describe the problem/error/question
I created a chat AI agent to answer users about a school’s courses & everything related to the school operations.
The AI agent is connecting to a supabase DB which have only 4 vectors (3 vectors for course’s information, 1 for Q&A - FAQ stuffs)
The chat bot work perfectly and answer all questions of user for the first 6,7 chats. With each execution I can see the agent get information from supabase and load them to the buffer memory. The buffer memory is set to get only 5 message as it’s context.
Howerver after about 7 chats back and forth, the AI agent stop query the supabase DB, and use infromation from the buffer memory to answer user’s question (Which is expected behavior i think). But now when user ask something new, the AI agent answers “I don’t have that information from our database” even when I checked the logs it get information in it’s buffer memory and that piece of information it need is right there.
I tested it multiple times with multiples questions, but right after about 7 chats back & forth, the agent response “We don’t have that information in our DB, would you like to chat with live agent?”
Things I have tried to solve the problem:
- I tried changed the buffer memory with supabase postgree, nothing changed.
- I tried changed the LLM model from Open AI 4o - mini to 4o, didn’t work
- I tried remove the buffer memory, now it worked because each time user asking something the Agent must query the Vector DB for informations. However it can’t have the context of the conversation now so the conversation soon become a mess after a few chat back & forth.
- I thought the problem is our LLM context window limit, but turns out it’s not, it can handle about 14, 15 chats back & forth with 3000 tokens each from system message & user message without exceed the LLM context windows limit of 128k tokens
I thinks the problem here is the AI agent have loaded all information it need to the buffer memory but somehow can’t use that information to answer user. Hope you can help me find a solution for this.
Please share your workflow
Information on your n8n setup
- n8n version: 1.91.2
- Database: Supabase Vector Database
- n8n EXECUTIONS_PROCESS setting (default: own, main): Selfhosted n8n
- Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
- Operating system: AWS ubuntu instance