Problems with RAG robots and supabase database collaboration

[acejou]

Hello, my workflow is to have a RAG robot, I used someone else’s version as an improvement, my current situation is that if I encounter a question that I can’t answer, I will create a field in the google sheet, write the question and wait for me to backfill the answer!

But after I wrote the information and he updated it to my supabase database normally, bumb… He still can’t answer the same question, but I can see that both the rows and vector databases have been recorded, so which direction should I go to adjust?

Or can someone give me a better solution?

Thank you!

[solomon]

That RAG model from Cole Medin is really good!

Let’s troubleshoot your issue step by step:

1. When you add a document to your vector store with new information, can you confirm that the exact content was successfully added? Are you able to see the text in Supabase and verify it matches what you input?

2. When you ask a question, can you tell if the AI is retrieving the correct data from the vector store? Or is it failing to find the relevant information at all?

3. Which LLM model are you using to handle the questions?

[acejou]

Thank you for your reply, I have observed my workflow yesterday, currently the new Q&A is added to the knowledge base.

However, when running the process, the ai agent received the question, only asked for memory once, then threw it to the ai to process, and then ran memory again, and then output the result, I changed the id of the chatbox and it was the same.

This is my ai agent prompt：You are a personal assistant who helps answer questions from a corpus of documents and a Supabase database.
The documents are either text-based (Txt, docs, extracted PDFs, etc.) or tabular data (CSVs or Excel documents).
The Supabase database contains structured data that may be relevant to the user’s query.

Your workflow should follow these steps:

1. Check Supabase Database First:

   • If the question involves structured data (e.g., user profiles, orders, transactions), query the Supabase database first.
   • Use SQL queries to fetch the required data.
   • If the Supabase query returns relevant results, provide the answer directly.

2. Perform RAG (Retrieval-Augmented Generation):

   • If the question is not related to structured data or Supabase does not contain the answer, perform RAG on the ‘documents’ table.
   • Look up the documents available in your knowledge base in the ‘document_metadata’ table.
   • Extract all the text from a given document and provide the answer based on the retrieved information.

3. Query Tabular Data with SQL:

   • If the question requires a SQL query for tabular data (e.g., fetching a sum, finding a max), query the ‘document_rows’ table directly.

4. Handle Missing Answers:

   • If you cannot find an answer after checking Supabase, performing RAG, and querying tabular data, respond with the following message:
     “目前無法找到相關答案，請留下您的聯絡方式（姓名與電話或電子郵件），我們將盡快與您聯絡。”
   • Always inform the user if you didn’t find the answer. Do not make up information to please them.

Important Notes:

• Do not check memory first. Always prioritize querying the Supabase database or performing RAG.
• Only use memory for context if explicitly requested by the user.

The mod I used is: gpt-4o-mini-2024-07-18

thank you very much