I’ve been building an AI workflow in n8n for internal operations, and I’m hoping to get some feedback from people who have dealt with similar issues.
The workflow handles different requests depending on the context. It pulls data, routes tasks, updates systems, and hands work off to the right process. It was working well at first, but as we’ve expanded the workflow, the agent has started making poor decisions about which tool or workflow to call.
Sometimes it skips retrieval when it should query a knowledge base. Other times it calls the wrong branch or answers from memory instead of using the latest data. The prompt has grown over time, so I’m wondering if that’s part of the problem or if the overall architecture needs to change.
Right now we’re using n8n with OpenAI, a vector database for retrieval, and a few internal APIs. We’ve also been testing Workbeaver for desktop tasks that aren’t exposed through APIs, while keeping n8n as the main orchestration layer.
For those building larger AI agents in n8n:
Do you keep a single orchestrator, or split responsibilities across smaller agents?
How do you reduce hallucinations when multiple tools are available?
Do you rely mostly on prompt engineering, or have you found architectural changes to be more effective?
Any best practices for deciding when an agent should retrieve data versus answering directly?
I’d really appreciate hearing what has worked for your projects. Thanks in advance.
this smells less like a prompt problem and more like youve outgrown the single orchestrator. when one agent holds every tool its picking from a big menu every turn, and the tool descriptions plus your grown prompt are all fighting for the same attention budget, so tool selection gets mushy exactly as you add branches. thats the pattern youre describing.
what fixed it for me was splitting it. keep a thin router agent whose only job is pick the lane, then hand off to small sub agents that each own 2 or 3 tools. smaller decision space per agent, shorter prompt each, way more reliable tool choice. the single fat orchestrator looks elegant but it degrades with every tool you bolt on.
for the skipping retrieval thing, stop letting retrieval be a tool the agent can choose to skip. on the branches where the answer has to be grounded, run retrieval as a fixed step before the agent even sees the request, so it always has fresh context and cant answer from memory. take that decision away from the model where correctness matters.
prompt eng vs architecture, past a certain size architecture wins and its not close. prompt tweaks have a ceiling, routing plus scoped sub agents is what holds when the surface keeps growing.
ive built a handful of these multi agent setups in production, happy to go deeper on the routing structure if it helps.
You have hit the wall that almost every growing n8n agent hits: one orchestrator with a fat prompt and a dozen tools. The model’s tool-selection accuracy falls off a cliff as both the tool count and the prompt length climb, so the symptoms you are seeing (skipped retrieval, wrong branch, answering from memory) are the expected failure mode, not a tuning problem you can prompt your way out of.
Taking your questions in order.
Single orchestrator vs split. Split. Turn the one or two big workflows into four or five smaller ones, each owning a single responsibility, and have the orchestrator call them via webhook or Execute Workflow. The orchestrator’s only job becomes routing: classify the request, hand it to the right sub-workflow, collect the result. It should not retrieve, update, or touch APIs itself. Each sub-agent then sees only the 3 to 5 tools relevant to its job instead of the whole menu, and the model chooses correctly far more often when the menu is short. As a bonus you can actually debug it, because a failure is now isolated to one small workflow instead of buried in a monolith.
Reducing hallucinations with multiple tools available. Two levers. First, shrink the tool set per agent, per above. Second, take decisions away from the model wherever the decision is actually deterministic. If a branch can be chosen from a known field with a Switch node, do that and do not make the LLM decide. Reserve the agent’s judgment for the genuinely ambiguous cases. Most “it called the wrong branch” problems are really “I asked the model to make a choice that a rule could have made.”
Prompt engineering vs architecture. Prompt tuning has a ceiling and you sound like you are hitting it. Past a certain size the prompt is the problem, not the fix, because you are pushing the context window far enough that the model loses the earlier instructions. A tight 3-tool agent with a short prompt beats a 15-tool agent with a giant prompt every time. Fix the architecture first, then tune prompts inside the smaller pieces.
Deciding retrieve vs answer directly. Do not leave this to the model’s discretion, because “answered from memory instead of latest data” means you gave it the option to skip retrieval. In the branch that needs fresh data, make retrieval a required step in the flow, not a tool the agent may or may not call. If it must go through the vector DB or the API to reach the answer, it cannot shortcut to memory. Force the path rather than hoping the prompt convinces it.
One more thing worth naming since you are running this in production: once you split into webhook-linked sub-workflows, watch for silent failures across them. A sub-agent can return green while quietly dropping work, and the orchestrator will happily continue on stale or empty data. Worth keeping an eye on item counts at each handoff.
Happy to go deeper on the routing pattern if useful.
@James198 One more point to back up splitting, now you can tune each job (sub workflow) and it’s tools to the model that works best for that part of the job. Haiku is terrible at tool calling, so is GPT 4 mini, Sonnet and GPT 5 work amazingly well with it. And segregating parts of your workflow allow you to really dial in and fine tune your output.