I am getting some strange responses that include details of the tool execution. What could be the reason and how can i solve this? It is not clear to me why the AI Agent is not processing the response of the tool into a useful answer.
For the models I am using llama3.1 and nomic-embed-text. Can it have to do something with the models I am using? How can the Agent be instructed to properly consume and output the information (like, no strange tags or code snippets and raw data of the tools)?
For the Vector DB I created some dummy meeting notes for testing. Those are properly fed into Qdrant. If I use the Question and Answer Chain node in an alternative workflow, the same thing is working as expexcted:
Just gave this a go myself and I think it’s likely the llama 3.1 model - and possibly also trying to run the model on lowend hardware? - which might be the issue.
I tested with:
ollama llama-3.1-8b and got “I don’t know” response even though the answer came back in from the vector store and was in the prompt itself.
groq llama-3.1-8b and got into a endless tools call loop.
groq llama-3.1-70b and got the correct response some of the time.
This is the screenshot of the llama-3.1-70b where it failed.
Hello @Jim_Le , thank you for trying and investigating. I don’t think it is an hardware issue. I also think it is an compatibility issue between the AI Agent and ollama. However, what is curios is, that ollama is working fine in combination with the QA Chain node and llama3.1. Here is the working sample (feel free to try it):
For me the QA Chain is working wonderful. However, it has the drawback, that it does not support a chat memory and therefore is not as powerful as the AI Agent.
It would be interesting to know why the two nodes are behaving so differently.
Example output of the QA Chain with attached Vector Store and context:
(still refering to the “Space Cookies” meeting notes document)
I have found the same issue. The QA Chain is good, but we need the more powerful AI agent to work, because it has memory…I’m using AI tools agent, and also in my case it does NOT retrieve correct data from the tools, even if the tool retrieves them. It just hallucinates or tells me he doesn’t have the information…look here an example . Also, like for the QA chain, we should be able to customize the prompt, like it has been done here . Are these feature in the working?
