@enzosoftware’s post is the one I’d build on, because “have I ever actually looked at what this sends?” is the right question, and pointing an integration at a local server that logs the request body is the cheapest way to answer it. Same experience here, it has never once come back clean.
There’s a question after that one, and I don’t think it has come up yet in the thread: once you have found the break and written a fix, how do you know the fix works?
The default answer is deploy and watch. Which makes production the test rig, for a failure class that by definition doesn’t announce itself, so “no alerts since Tuesday” is indistinguishable from “still broken, still quiet”. The cadence and error-rate baselines Miguel_Ruilope described will eventually surface it, but eventually is carrying a lot of weight there.
What made this tractable for me was keeping the recorded execution from when it broke, the actual request and response bodies rather than a summary, and replaying a candidate fix against that exact recording with egress blocked. Two properties do the work.
Sealed network: if the replay can reach the live API, you are testing today’s API, not the one that broke you, and a green result means nothing.
Raw bytes rather than a summary: the whole failure class here is shape, and the empty JSON body example above is exactly what a summarised log throws away.
Then the question is narrow enough to actually answer: does this fix turn that specific recorded failure into the right output, yes or no. Far easier to be certain about than “is this workflow healthy”.
The honest limit of it is that this only works for failures you already recorded. For anything not yet caught, @ali.alsamraay’s static read of the workflow JSON, and reading the bytes once before trusting a new workflow, are strictly better, because neither needs the failure to have happened first. Outside-in, inside-out and replay are answering three different questions, and I don’t think any one of them substitutes for the others.
Disclosure, in the same spirit as ali’s: I built a tool that does the replay-and-verify part, and its main design rule is that it refuses to claim a fix works when it cannot prove it. It is free and has no users yet, so please treat it as an approach worth stealing rather than a recommendation. The sealed-replay idea is worth more than my implementation of it.