I keep seeing workflows where an LLM has full access to Stripe refunds, email sending, and database writes — with zero approval gate between the AI deciding to act and the action executing. If the model hallucinates a refund for the wrong customer, or gets prompt-injected via an email body, it just… goes through. Who’s actually gating this?
This is something I think gets overlooked in a lot of AIAgent demos
I prefer treating the LLM as the decision/recommendation layer not the final authority for high impact actions.
AI decision - deterministic validation - human approval - action
For things like refunds, payments and destructive DB writes or sensitive outbound messages I would not give the agent unrestricted execution rights in my opinion!
@StefanGwiggner — exactly right. “AI decision → deterministic validation → human approval → action” is the correct architecture. The problem is most people stop at step 1 and never build steps 2-3 because it’s tedious.
The part that gets overlooked even in your model: deterministic validation. Most people jump straight from AI decision to human approval, which means the human is the only check. And humans rubber-stamp things that look reasonable — a well-written email draft or a refund that “looks correct” passes review because nothing flags it as wrong. The bug is often upstream, invisible to a human reviewer.
What works in production: deterministic policy checks that auto-approve low-risk actions (internal email, template reply, small refund) and only escalate high-risk ones to human approval. That way the human isn’t reviewing every action — they’re reviewing the ones that didn’t pass deterministic checks. Fewer reviews, higher scrutiny per review, less rubber-stamping.
The other thing: audit trail. Not just “who approved what” — but “what did the AI decide, what did the policy check, what did the human see, and what actually happened.” If a bad action slips through, you need to trace back to which layer failed. Without that, you’re guessing.
Yes good point. Especially about the human becoming the only safety layer.
If every single action needs approval, sooner or later people will just start clicking approve without really checking it.
I like the idea of separating it by risk
AI decision - deterministic policy check - auto-approve low risk / human approval high risk- action - audit log
And agreed on the audit trail. For production I think storing the AI decision, policy result, approval and final action should basically be part of the workflow design from the beginning not something added later.
This actually makes me want to build a reusable pattern for this ![]()
Funny timing XD I actually built exactly this as a reusable pattern a while back,
It’s a community n8n node
n8n-nodes-guardian) and applying for a native package, that does the deterministic policy check + auto-approve/escalate + audit trail as a drop-in step — so instead of wiring Sheets + Slack + custom routing every time, you point your agent’s action at it and it returns ALLOW / DENY / REQUIRE_APPROVAL with the reasoning attached. The audit trail is append-only with a hash chain, so you get the “what did the AI decide, what did the policy check, what actually happened” traceability from day one instead of bolting it on later — exactly what you said matters.
There’s a free tier if you want to poke at it before building your own version from scratch — might save you a few days, or at least give you ideas to steal for your own implementation if you’d rather build it yourself. Happy to share the link if useful, or just compare notes on the architecture — genuinely curious how you’d approach the risk-tiering logic differently.
LG, Boris
Sure, send me the link. I’d be interested to have a look at how you implemented it, especially the risk-tiering and audit side.
I’m still tempted to build my own version though, mostly because I’d like to learn, understand and control the whole policy layer myself.
The part that worries me is that the tool being available can quietly become the same thing as the agent being authorized to use it.Those are very different states. For consequential actions like refunds, email sends, or database writes, I would want a clear chain:
capability exists → authority is established → policy is checked → action executes → downstream outcome is verified.
Otherwise the workflow can be technically functional while still having no real control boundary.
Stefan — DM’d you the link.
And seriously, if the reason you want to build it is to understand the policy layer yourself, build it. That’s a good reason. I’d rather talk to someone who’s actually built one anyway.
@Mcpherson_AI — yeah, this is the thing. Tool gets wired into the agent and that’s it, that’s the whole authorization step. Nobody ever sat down and decided that this agent may issue refunds, up to X, for this client. Connecting the node just quietly became permission.
The human version is worse I think, because it feels like you’ve solved it. If I approve something because it looks fine, I haven’t authorized anything. I’ve assumed it was already authorized and clicked. A well-written refund for the wrong amount gets approved every time.
Your first four links are all doable without too much pain. Capability is just wiring. Authority is scoping, which agent, which project, which client, which action types, decided somewhere outside the workflow so that adding a node doesn’t grant anything by itself. The policy check has to look at the payload and not just the action type, and it has to default to deny, because what gets you is always an action nobody wrote a rule for. Execution is the boring part.
The fifth one is where I’ve only got half an answer.
Three things get mushed together a lot. The decision (policy said yes), the claim (something told you it ran), and the actual outcome (some system other than the agent says it happened). Most setups only have the first and treat the second like it’s the third.
What I did do is bind the decision to the payload. It gets canonicalised and hashed when the decision is made, and at execute time you have to send a payload that hashes to the same thing or it’s a 403 before anything runs. Approved for 50, called with 500, it just refuses. If a human edited the payload while approving it, it compares against what they approved, not what was originally sent. And execution is once only, enforced by a unique constraint in the database rather than an if statement, so replays don’t do anything.
Still not verification though. The caller is the one telling me it ran.
What it does kill is the gap between what was approved and what was claimed, and honestly I think that’s where most of the real damage is. It’s rarely an agent lying about executing. It’s an agent executing something slightly different from what got approved and reporting success completely honestly.
The third one I just don’t have. Only thing that works is going out of band, webhook from Stripe instead of the agent telling me, reading the record back from the CRM instead of trusting the report. Fine if the system emits events, useless if it doesn’t.
Have you seen anyone actually solve that last part? Genuinely asking, it’s been bugging me for months.
@BorkoB Yes, this is the exact problem I’ve been working on. I’m building Observa around the idea that the caller cannot be the final authority on whether the business outcome happened. The approach is the same one you’re describing: keep the execution claim, but separately perform an authoritative read-back from the downstream system and bind that result back to the original intent and execution. So for Stripe, the tool call saying “refund succeeded” is evidence of execution. The stronger verification is an independent Stripe read-back showing the intended refund actually exists with the expected amount/customer/state. For systems that expose a usable API or event stream, that can be automated. For systems that don’t, the honest answer has to remain unresolved rather than pretending the execution claim is verification. I’m testing this in shadow mode with n8n right now. The harder part has actually been maintaining continuity across intent → authority → execution → downstream evidence without letting any one layer grade itself. Your decision/claim/outcome distinction is exactly how I think about it.
Observa — right, so you’re coming at it from the other end. That makes sense.
One thing I’d push on, because it took me a while to see and it might save you some pain. A read-back that finds a matching refund isn’t the same as a read-back that proves your intent caused it.
If you query Stripe and find a refund for the right amount, to the right customer, in the right state, that’s a matching record. It isn’t necessarily your record. Two refunds of 50 to the same customer in the same window. A retry that created a second one. Someone doing it manually in the dashboard while the workflow was mid-flight. All of those give you a green read-back for the wrong reason.
What you probably want is correlation, not matching. Put the intent ID into the idempotency key, or into metadata on the object at creation time, so the read-back can say “this specific intent produced this specific object” rather than “something consistent with what we expected exists”. Then the downstream evidence is bound to the intent instead of just resembling it.
Which is the same problem you already named, not letting a layer grade itself, just one level further down. The read-back is only independent if the thing it’s reading was tagged by a layer that isn’t the one doing the grading.
Other thing, more framing than criticism. Verification is always after the fact. By the time the read-back lands, the money moved. So it’s a detection control, not a prevention control. That’s not a weakness, it’s a different job — you find out what happened and you can prove it, but you can’t stop it. Gating before execution is the mirror image, you can stop it but you never really know what happened downstream. I don’t think either one is the whole answer on its own.
I’ve had the gate side working for a while and the verification side has been the open question the entire time, so it’s good to see someone actually going at it properly.
What are you using for the shadow comparison, out of interest? Storing expected state and diffing against the read-back, or something looser?
Scott Aerators team: AI automation can be powerful, but financial workflows definitely need clear permissions and safeguards. Adding spending limits, approval steps, and monitoring could help reduce the risk of an automated process making unexpected transactions.
@BorkoB That correlation point is important. A matching downstream state is not enough if you can’t bind it back to the specific intent/execution that produced it.
For the shadow comparison, I’m keeping it more structured than a loose “looks right” check.
The basic model is:
bound intent / expected state
→ observed execution
→ authoritative downstream read-back
→ comparison result
Where the downstream system gives us a strong correlation primitive, I want that bound into the lifecycle. Where it doesn’t, I don’t want Observa promoting a plausible match into VERIFIED just because the values line up. That should stay uncertain or require another evidence path.
And I agree on prevention vs detection. Gating can stop an unauthorized action, but it can’t by itself prove what happened afterward. Verification can prove the downstream state, but it arrives after execution.
That’s why I’m treating them as two halves of the same lifecycle rather than substitutes for each other.
Yeah, two halves of the same lifecycle feels right to me.
One thing on the correlation problem. You said where the downstream system gives you a primitive you bind it, and where it doesn’t you stay uncertain. There’s maybe a third option — if you can’t get a primitive from the system, you can sometimes mint one at the gate.
The decision already has a unique id and a hash of the payload it approved. Push that id out with the call — idempotency key, a metadata field, a reference on the message — and now the downstream record carries it. Your read-back gets a hard primitive even though the vendor never gave you one.
Only works if something actually controls the outbound call though. If the agent hits Stripe directly and the gate just says yes or no, there’s nothing to inject into. You’d need the gate to proxy it, or hand back a token the caller has to attach. Most setups are advisory, which is probably why nobody bothers.
And I’d give your uncertain state its own name rather than treating it as not-verified. Everything that’s ever bitten me in this area looked fine because it didn’t fire.
What do you do when the read-back is late? Eventual consistency, a webhook that shows up twenty minutes after.
@BorkoB That’s a really useful distinction. Minting the correlation primitive at the gate is probably the cleanest path when the runtime is actually under governance. We can carry the decision ID/token through the outbound action and bind the downstream receipt back to it rather than infer correlation later.
And agreed on uncertainty. “Not verified yet” and “could not be verified” shouldn’t collapse into the same state. A pending/reconciliation state with webhook or delayed read-back transition makes much more sense, especially once enforcement is controlling the execution path.
The boundary you called out is the key one: if Observa only returns ALLOW and the caller goes around it, we can’t create that binding. If the runtime bridge/proxy owns the handoff, we can.
Yep, that’s the line. If the runtime owns the handoff you can bind it. If the caller can go around you, you’re inferring. Everything else follows from which side of that you’re on.
Good luck
The “runtime proxy owns the handoff” boundary is where most advisory agent architectures quietly fall apart under failure conditions.
If an LLM or sub-agent handles the actual transport to Stripe directly, any intermediate drop—network socket reset, 504 gateway timeout, or client-side retry—instantly breaks the lifecycle. The model assumes a drop is an unexecuted failure, regenerates the payload without the original idempotency key, and executes a second charge at 3 AM.
In a benchmark run across ~12,000 autonomous tool-calling traces we analyzed, tool-call failures hovered around 18.4%. More importantly, nearly 28% of those failures were connection/timeout drops where the downstream API actually accepted the request, but the agent runtime dropped the response context. Without an enforced proxy wrapping the request with a pre-minted deterministic key, the retry mechanism becomes an unmonitored money drain.
For delayed read-backs (eventual consistency / late webhooks), splitting state into three distinct buckets rather than binary verified/unverified saved us a lot of operational headache:
ACKNOWLEDGED_PENDING(outbound call left the proxy with an injected correlation ID, waiting for downstream receipt)RECONCILED(authoritative webhook or polling read-back returned and matched the decision hash)ORPHANED_UNCONFIRMED(read-back timeout TTL expired—typically 15-30m—requiring automated hold/freeze before any further tools can be invoked on that account)
If you treat `ACKNOWLEDGED_PENDING` as a hard circuit breaker that blocks subsequent state-mutating tool calls for the same tenant, you neutralize the runaway loop entirely, even if Stripe webhooks arrive 20 minutes late.
I documented the failure mode taxonomy and FinOps leakage metrics from those traces here if you want to inspect the breakdown: AI Agent FinOps Stats [12,400 Prod Workflows + Failure Rates]
This is a real concern, Borko. The biggest risk isn’t just that an AI agent can make a wrong decision — it’s that the workflow may execute that decision immediately through Stripe, email, or a database before anyone has a chance to catch it.
A practical approach is to put a risk-based validation layer between the agent and the final action. Low-risk actions can continue automatically, while refunds, customer communications, record changes, or other sensitive actions can be checked against defined rules and routed for approval when something falls outside those rules.
I’d also keep a clear audit trail of the agent’s requested action, the data behind the decision, the validation result, and the final action taken. That would help with both troubleshooting and giving clients visibility into what their AI agents are actually doing.
The prompt-injection case makes this especially interesting, because an API request can look completely valid even when the AI was influenced by untrusted input. I’d be interested to see how you’re addressing that part in your policy layer.
That’s the part I find hardest too, because you can’t solve it at the request level. An injected instruction and a real one look identical by the time they reach the Stripe node. Anything trying to tell them apart is guessing.
So the layer doesn’t look at the prompt at all. It looks at the action. You decide in advance what’s allowed and it gets enforced whatever reason the agent gave. Refund over X goes to approval. Destination not on the allowlist is denied. More than N refunds in an hour is denied. An injected “refund 5,000 to this account” fails all three, and it fails without anyone having to work out that the email was poisoned.
The honest limit: if the injected action sits inside policy, it’s allowed. Injection doesn’t get defeated, it gets bounded, and how tight that bound is depends on how narrow the rules are. Which is an argument for writing them narrow.
One other piece. The payload gets hashed when it’s checked. If it changes between the check and the execution, the execution is refused, because it’s not the thing the policy saw anymore. That closes the window where the agent gets influenced after the decision.
And when something does get through, the log shows the payload, the rule that allowed it and the timestamp. That turns “the agent did something strange” into a specific rule you go fix.
I think @StefanGwiggner, @BorkoB and @Mcpherson_AI have already covered the policy, approval, audit and downstream verification sides pretty thoroughly.
One n8n-specific point I’d add is that I probably wouldn’t expose the Stripe refund operation to the AI Agent at all.
n8n already has human review for AI tool calls, so you can gate a sensitive tool before it executes. But for something like refunds, I’d go one step further and make the Agent’s tool a Call n8n Workflow Tool that calls a dedicated refund workflow.
The Agent would only be allowed to submit a constrained request: customer/order, requested amount, reason, etc. The child workflow would own the Stripe credential and the actual authority to perform the refund.
Inside that workflow I’d then:
-
retrieve the charge and existing refund state directly from Stripe;
-
apply the deterministic rules using normal n8n logic;
-
reject anything outside policy;
-
use human approval where the policy requires it;
-
execute the refund with an idempotency/correlation value;
-
store the result and reconcile it against Stripe afterwards.
That creates quite a different security boundary from attaching a Stripe node directly to the Agent.
The Agent can ask for a refund, but it can’t perform one. It also can’t choose to bypass the validation because the Stripe credentials and mutation node only exist inside the controlled workflow.
That seems particularly relevant to @BorkoB’s point about prompt injection. You don’t have to reliably detect that an email poisoned the Agent. Even a completely compromised Agent is still limited to making requests against the same deterministic interface.
It also ties into @Mcpherson_AI’s point about the runtime owning the handoff. If the Agent only gets the governed sub-workflow as its tool, rather than direct access to Stripe, n8n itself becomes the place where that handoff is enforced rather than advisory.
For me that is probably the cleanest n8n pattern here:
Don’t just put a gate in front of a powerful Agent tool; make the powerful operation inaccessible to the Agent in the first place.
That distinction makes sense, Borko. Trying to identify whether every action came from a clean or injected prompt can become unreliable, while controlling what the agent is actually allowed to execute gives you a much clearer boundary.
I think the limitation you mentioned is the important part — if a manipulated action still falls within the allowed policy, it can get through. That makes narrow rules, thresholds, and approval conditions especially important rather than giving the agent broad permissions.
The audit trail also adds value here because when something unexpected does pass, you’re not left guessing what happened. You can see the action, the rule that allowed it, and then adjust that specific policy.
Have you considered adding a risk score or anomaly check alongside the fixed rules, so an action that is technically within policy but unusual for that customer or workflow could still be sent for approval?