Multi Main Agent & Multi Sub agent architecture workflow

Help me Build my Workflow
1

Hi all! My first post here so If I’m missing something, let me know :slight_smile:

I’m struggling a bit with my personal assistant setup. What’s the idea:

A personal assistant that has access to multiple sub agents for specific tasks. The main agent is my personal assistant, says good morning and it talks to me and gaters info about the task to do and delegates. Sounds simple right?

Most of the videos on youtube are not actually usable in prod environments, just funny demos.

Main problem:

  • The whole workflow, make it strict, back and forth feedback, approval. Universal.

Wishes:

  • Human in the loop
  • Strict step by step workflow when handling choices or approvals or feedback
  • Multiple main agent can use the sub agents (PA can search for contact, but marketing agent can also search for contacts in same workflow)
  • Telegram connected to main agent (chat input)

Tools:

  • Telegram (for talking)
  • Airtable (for contacts)
  • OpenAi (for LLM, but open for others)
  • Redis (long ish memory)

What didn’t work out or gave me headaches:

  • using sub agents as tools (agent swarn) doesn’t work because you can’t built a workflow around the sub agent tool or create multi step workflows, only 1 tool with more sub tools.
  • So, for above, human in the loop is not possible. and not scalable to other main agents. Except when you add all the main agents in 1 workflow.
  • defining workflows in the prompt of the main agent kept going wrong, the agent was unreliable and sometimes skips steps or said it had a contact while it didn’t. Even when I made the step explicitly mandatory.
  • talking back and forth towards main agent about choices, what’s the best way to do this?
    • via memory? pushing everything back to memory and let main and sub agent use same memory?
    • just push choice back to telegram only?

The first use case I couldn’t get working:

  1. Me via telegram: I’m looking for Pieter
  2. Main agent: search for Pieter
  3. Sub agent: I found 2:
    1. Pieter 1 - [email protected]
    2. Pieter 2 - [email protected]
  4. Main agent: Great I’ll pick 1
  5. Main to me: what would you like to do with Pieter 1 - [email protected]? Send an email?
  6. Me via telegram towards main agent: Yes, send him a message that I’m 20 minutes later today, thanks!

From here, but maybe out of scope for this one…

  1. Bill uses this info to send the request towards email_agent: Send email to Pieter 1 - [email protected] with the text “Yes, send him a message that I’m 20 minutes later today, thanks!”
  2. email_agent will rewrite the email and create a subject based on the message.
  3. using a message for approval, is this the email you would like to send now:
    To: [email protected]
    Subject: I’m late
    Message: Hi Pieter 1, I just want to let you know that I’m a bit later today, apologies. Kind regards Hans.
    [signature]
  4. approval via chat? or do we need telegram human in the loop here? which are both fine. but the prompt is never strict enough, so I assume human in the loop via Telegram.

I hope this is a bit clear since it’s a long message but the best way is to provide some context, right?

Edit:

  • n8n version: 1.119.1

  • Database (default: SQLite): don’t know

  • n8n EXECUTIONS_PROCESS setting (default: own, main): no idea

  • Running n8n via (Docker, npm, n8n cloud, desktop app): cloud

  • Operating system: windows

1 Like
2

Sometginh similar?

brain
brain1833×787 292 KB

I can say that system prompts has to be very strict, and clear detailed instructions about tools use (when,why,examples,guardrail like never to do etc.. )
If you expect strict output, then as well you have to provide context, format etc.
You can achieve a fully strict flow, if is that your wish.

2 Likes
4

haha, yeah that looks like it, assuming that the sub workflows can also be used by other agents too. But the reason that I think it’s needed, is that I noticed that when my prompt got to big, things got a bit less strict and you have a load a very big prompt everytime you ask for just one small thing. It just sounds logical to put as much as possible in the sub workflows. How does this one works for you?

1 Like
5

If in your main workflow you have several agent nodes , and you want that they share same memory(content key , and all agents share same memory node), I just update the memory with the context and additional intermediateSteps, so others can be “aware”.

What can I say, I have 2 projects…

One is very strict(description, parameters, etc) to follow a strict pattern, even is not responding more than x phrases (or set top_k so it s very deterministic).

2nd is a bit more “free” since it can adapt for user input(and even reformulation of initial user prompt as it may lack info).

Summary, yes you can achieve a “strict” flow, or a “free” one(here hallucinations may intervene).

I didn’t find yet the “perfect” combination for all the jobs.

6

P.S @Bart_Sch I think that s how a LLM works… you say “A”, the llm respond “A is etc”, next prompt(when memory attached so it s a “conversation”), will be again human: “A” - ai: “A is etc” -user: “B” - ai: “B is etc” … An so on adding the previous meesages into context. Thats why using as well a vector db with crucial informations may help as well. Faster and you can updated it only with relevant content.

The topic will never end soo… I think you got the idea.

Here a good example as well with redis memory.

1 Like