AI Agent that has 3 to 4 different types of output, combine or split up?

[miguste]

Describe the problem/error/question

Hi! I’m trying to test-build a chatbot for a pharmacy, but it has different types of tools it uses, for example

• The general_info tool which returns information from my pinecone store.
• The product_info tool which checks my database (airtable) for products, when a user requests product info
• The global_availability tool, which does an API call.

2 of these 3 should return JSON, the other one (general_info) should just respond like a “human”, saying things like opening hours, but for the products I would like a JSON response.

What’s the best practise in terms of structuring this? If I have one AI Agent, with 3 tools, I can’t use the “structured output node”, because the output is different for each of the 3 tools. How does one go about this? Thanks!

[solomon]

The output of the tools will only be read by the AI Agent.
If you need the Agent to output a certain format, you can specify using the output parser on the Agent and not on the tools.

You don’t need to change the output of the tools, as long as the AI Agent can understand it and interpret it for you.

If my reply answers your question, please remember to mark it as a solution.

[miguste]

This still leaves me with my original problem, the AI Agent has 3 output methods, depending on the tool it uses,

If it uses the general_info tool it should output in text, but if it uses the product_tool it should output in structured JSON format.

[solomon]

Oh, I see. You need different outputs from the AI.
Can you share your workflow with us?

[miguste]

Sure thing @solomon

Main Flow

• GetGeneralInformation looks up information in my vector store (which has info parsed from a PDF, such as opening hours, store information), This should answer in human language
• LookupExternalProduct call another flow (see below) This should answer in a strict JSON response

LookupExternalProduct Flow

This flow performs a GET request, and parses the correct data, this returns the following data,

{
  "availability": "{{ $json.notARR[0].notificationStatus }}",
  "name": "{{ $json.prescriptionName }}"
}

[solomon]

Nice. So in this case I believe you can add instructions to the prompt that strictly say to return raw data from specific tools.

You can do that both in the main prompt, but also in the tools description.
Try first in the main prompt, and if you still see inconsistencies, add an additional note to the tool description.

For example:

Main prompt:

When using the following tools, only return their raw output in JSON, without any additional text:
- LookupExternalProduct
- YourOtherTool

Tool description:

When using this tool, respond only with its raw output, without any extra text.

[miguste]

Thanks Solomon! This feels right, I will try this as soon as I can.

[solomon]

Let me know how it goes!

If the AI outputs the JSON as a string, you might need to turn it into a JSON object. Something like this: