7 failure-path questions before trusting an external API workflow

I keep seeing external-API workflows tested only on the happy path. That proves a workflow can run, but not that it can fail safely.

Here is the compact checklist I now use before calling a webhook/API workflow production-ready:

1. What happens if the provider accepts the request but the client times out?

Look for a durable operation record created before the call, a stable idempotency key, and reconciliation before any resubmit.

2. Can the same webhook arrive twice—or five times—without repeating the business effect?

Event deduplication is useful, but the stronger guard is a unique effect record around charging, publishing, sending, or provisioning.

3. Can polling and webhooks race safely?

Both paths should use one atomic state-transition route. The losing handler should become a no-op.

4. Which errors are actually retryable?

Handle connection failures, 408, 429, and selected 5xx responses explicitly. Honor Retry-After, add jitter, and bound both attempt count and elapsed time.

5. Can a late event move state backward?

Transitions should be monotonic. A late “running” event must not overwrite success; a late success may need to reconcile a local timeout.

6. Where do unknown or invalid events go?

Keep a dead-letter record with reason, payload hash, first-seen time, and replay status instead of silently discarding it.

7. Can an operator answer “what happened?” without reading raw logs?

Expose the operation ID, provider reference, attempts, state history, last error, next action, and a safe manual-replay control.

A minimum test set is: accepted request + client timeout, duplicate webhooks, webhook/poller race, 429 with Retry-After, late success, unknown operation, and invalid signature.

I also put a formatted copy here for easy sharing: 7 questions before trusting an API automation – Telegraph

What failure case would you add?

1 Like