Different angle to the answers above, because everyone so far is gating writes: money, emails, records. The action I had to learn to gate is the one that writes nothing, the AI answering a customer directly.
A chatbot replying to a real person is an irreversible external action too. Once it has said something wrong you cannot roll it back. But none of the usual instincts fire, because nothing was inserted, nothing hit an external API, no row changed. It does not look like the dangerous class of action, so it usually does not get a gate at all.
What I use instead of a human approval step, since you cannot put a human in front of a live chat reply without killing the product: a confidence gate between retrieval and answer. If the retrieved evidence is too thin or the similarity is too weak, the model does not get to answer. It says it does not have that information and hands off to a human. Refusal is the default and answering is the thing that has to be earned. Same shape as an allowlist, just applied to whether the model may speak rather than whether it may act.
Two things I got wrong that are probably worth passing on.
On your auditing question: I log the retrieved chunks and their similarity scores next to every answer, not just the final text. Without that, “why did it say that” is unanswerable, because the answer text alone tells you nothing about what the model was actually looking at. It is the read side version of what nathan3 said about storing the agent’s reasoning alongside the decision.
Second, and this one caught me two days ago: the gate itself can fail silently, and it fails in the safe looking direction. A stale condition in a filter node downstream of retrieval was dropping every row, so the gate saw zero evidence and correctly refused. Every execution green, no error thrown. The bot politely told people it did not know things it demonstrably did know, and it would have carried on doing that indefinitely, because a refusal never looks like a failure. I only found it by asking a question I already knew the source documents answered.
So the thing I would add to the propose then execute pattern described above: whatever component decides “do not proceed”, monitor how often it fires. A deny path that quietly moves from firing 5 percent of the time to 100 percent of the time is a broken system that looks exactly like a cautious one.