Disclosure up front: I run NoCrash, so I see a lot of agent setups, and the most common thing people show us right now is an AI agent that finished green and confident while the tool it called underneath had quietly failed. The run closes successful, the agent reports the task is done, and only later does someone notice the answer was made up on top of nothing. A search tool returned zero results, or an HTTP node came back empty, or a database lookup timed out and gave back nothing, and the model just wrote a fluent, confident answer over that hole. No error was thrown, so the whole run shows green.
What people only learn the hard way is that a green run means the agent finished talking, not that the tool it leaned on actually returned real data. An LLM does not stop when a tool comes back empty. It fills the gap. So the failure mode is not a crash, it is a confident wrong answer sitting on top of a failed tool call, and nothing in the execution log looks wrong because technically nothing threw.
The fix people use, no tool required: stop trusting the agent’s own “done”. Check the tool output before the model is allowed to answer on it. After each tool call that matters, verify the result is actually present and non-empty, and if it came back empty, make the agent say it could not retrieve the data instead of guessing. Treat an empty tool result as a stop condition, not as something to narrate around. (For the case where the run still closes green on an empty tool result, we built a free grader that flags it from the API side: ) n8n Workflow Grader — is your workflow safe to run? | NoCrash
If your agent talks to anything it does not control, that one check is worth more than a better prompt. How are you all catching the empty-tool case, a validation step after each call, a guard in the system prompt, or something that watches the tool outputs for you?
The pattern I’ve settled on: every tool sub-workflow ends with a Code node that normalizes the output into a fixed shape - { ok: true/false, data: ..., reason: '' }. If the original call returns empty or throws, the Code node forces ok: false with a reason string. The agent’s system prompt says: “If any tool returns ok=false, stop and tell the user which step failed and why. Do not fill in missing data.”
This way the guard lives in the tool itself, not in the main agent prompt. It’s portable - you can reuse the same pattern across different agents without rewriting the system prompt each time, and the main agent never gets a chance to narrate over a hole.
1 Like
The fixed-shape ok/data/reason wrapper is where I landed too, and stamping the reason at the tool means the main agent never has to guess what broke. The one spot I would still watch is that “if ok=false, stop” lives in the system prompt, so you are trusting the model to honor it. Under a long context or a conflicting instruction it can still narrate over an ok=false. What held up better for me was routing on ok=false in the workflow itself, an IF node right after the tool, so a false result physically cannot reach the model as something answerable. Same shape you built, just enforced outside the prompt instead of inside it.
The other thing that bit me was a newer tool quietly not getting the wrapper, so the guard held on the tools that had it and was just missing on the one that did not. How do you make sure every new tool actually gets the shape, a shared sub-workflow, a lint step, or just discipline?
The workflow-level enforcement point is the key one for me. If ok=false only lives in the prompt, the model is still part of the control system. I would rather make the workflow decide that a failed tool result cannot reach the final answer path at all.
The pattern I like is:
- Every tool returns the same envelope:
ok, data, reason, source, checked_at.
- An IF/Switch node routes
ok=false to a failure branch before the model sees it.
- Empty-but-valid responses get their own category, because “no rows found” and “database timed out” should not be treated the same.
- The failure branch creates an issue with the tool name, input summary, reason, and owner.
- The user-facing answer says the tool result could not be verified instead of letting the model improvise.
For client work, the issue record matters as much as the guard. A green-looking agent run that answered from missing data is not just a prompt problem; it is a support incident that needs a resolution note and probably a client-facing explanation if it affected an outcome.
This is one of the silent-failure categories Maintain Flow needs to handle well: successful execution, failed business evidence.
1 Like
The empty-but-valid category is the one most people miss. “No rows found” and “database timed out” landing in the same bucket is how a real gap gets narrated as a normal result, so splitting them at the envelope is the right call. Same with source and checked_at living in there, that is what lets the failure branch write a useful issue instead of just “something broke”.
And you named the part I care about most: for client work the issue record is the deliverable, not the guard. A green run that answered from missing data is a support incident, and if it touched a client outcome it needs a resolution note and an honest explanation, not a silent retry. “Successful execution, failed business evidence” is a cleaner name for that category than anything I have used.
That last one is exactly the failure class we watch for from outside the run, the execution closes fine and the business thing never happened. Sounds like you are deep in this already. What does your failure branch do for the client-facing side, template the explanation or leave it to a human?