I am setting up a website communications agent to answer FAQs and also steer the user to the best suited product/ service for them.
The problem I am getting is that no matter the system instruction the Agent will go beyond its connect vector store tools/knowledge base and get general information from it’s own LLM knowledge. Instead of simply saying it doesn’t have access to that information.
Sometimes if will explicitly say it doesn’t have that information in it’s knowledge base. other times it will give a full on ChatGPT response with knowledge definitely not in the vector store.
I have opted for large embeddings for future proofing against larger more complex docs and information. I am using supabase for vector store and memory. I have opted using separate vector tables to things like, FAQ’s, Products ect… these work correctly and information is taken from both. But I get the same issue using 4o-mini and 4o-latest of the LLM using it’s own GPT knowledge base.
Has anyone manged to overcome the LLM going beyond the connected tools?
This will reduce creativity and make the model respond in a more direct way.
2. Better prompt
I see you’re frustrated with your trial and error, but please take a look at this example and see if it helps:
You are a personal assistant who helps answer questions from a corpus of documents. The documents are either text based (Txt, docs, extracted PDFs, etc.) or tabular data (CSVs or Excel documents).
You are given tools to perform RAG in the ‘documents’ table, look up the documents available in your knowledge base in the ‘document_metadata’ table, extract all the text from a given document, and query the tabular files with SQL in the ‘document_rows’ table.
Always start by performing RAG unless the question requires a SQL query for tabular data (fetching a sum, finding a max, something a RAG lookup would be unreliable for). If RAG doesn’t help, then look at the documents that are available to you, find a few that you think would contain the answer, and then analyze those.
Always tell the user if you didn’t find the answer. Don’t make something up just to please them.
3. Change the model to a more capable one
Some models are really more capable and should be used when the situation calls for them
I’ve had a situation where OpenAI 4o-mini would always give a wrong answer from the database, because the information was complex.
The only solution was to change the model to 4o. Then everything started falling into place.
.
If my reply answers your question, please remember to mark it as a solution.
If my reply answers your question, please remember to mark it as a solution.