Trouble with AI Agents

Hey @Anthony_Lee

Some observations on your tool descriptions:

Example 1:

Call this tool to get context from a vector database that will assist in writing the book The Evolved Man.

For me, this is a classic case of focusing too much on the “how” (see my previous response to this). What response should the agent expect from this tool? How can it determine that the response was a success? If it can’t, it’ll just keep retrying/asking and you’ll get stuck.
In this scenario, I try to put myself in the virtual shoes of the agent:

  • Why should I use this tool? it helps the “user” write their book.
  • When should I use this tool? when user is mentioning/asking about something that was discussed previously/in the past.

Try this or some variation of:

Call this tool to search for past context that was saved whilst helping the user write their book.

Example 2:

Call this tool to make an API call to Perplexity AI that will do a real-time web lookup for research papers on a topic.

Becareful of keywords in your tool description as these also act as trigger words for tool use:

  • “Perplexity AI” - did the user specifically request to use Perplexity?
  • “real-time web lookup” - did the user specifically request for real-time or does it matter if it’s slightly delayed?
  • “research papers” - did the user specifically ask for research papers or just wants the summary of the research?

Sometimes being overly specific or service-dependent means the tool won’t be used or used incorrectly.

Try this instead:

Call this tool to research a topic suggested by the user.

  • “research a topic” - doesn’t care about what service is being used, focuses on purpose; just send the topic to be researched.
  • “suggested by the user” - only trigger this when the user mentions or requests for it or when it is a good time to do so. Good for limiting ways extraneous calls.

Overall I think you have quite an ambitious project on your hands but don’t give up, I’ll be cheering you on!

I cooked up this example workflow to hopefully help you debug your tool problem.



7 Likes