Handle Rate Limits and Retries for External APIs

[Greg_John]

Hi everyone,
I’m building n8n workflows that depend heavily on external APIs, and I’m trying to find a reliable way to handle rate limits without slowing down the entire workflow.
For example, an API might return: {
“status”: 429,
“message”: “Too Many Requests”,
“retry_after”: 30
}
My main concerns are:Avoiding unnecessary retries
Respecting API rate limits
Preventing one slow API from blocking other workflows
Handling temporary failures without losing data
I’m considering exponential backoff, request queues, and limiting concurrency.
For those running n8n in production:
How do you normally handle 429 responses?
Do you use a queue or simply retry with delays?
How do you decide the maximum number of retries?
Have you found a good way to prevent one API from becoming a bottleneck for the entire n8n instance?

Describe the problem/error/question

What is the error message (if any)?

Please share your workflow

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Information on your n8n setup

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[n8n]

Hey @Greg_John, while you wait for a response, here are some things that might help:

Suggested resources

Automatically matched to your question.

Docs:

• Handle rate limits
• Common Issues
• Handle rate limits

Forum:

• Handling Rate Limits Across Parallel Workflows Without Losing Throughput
• N8n HTTP Request Failing Due to Rate Limits – Possible Solutions?
• Way to Handle API Rate Limits Across Hundreds of Tenants in n8n

@Niffzy, @Yo_its_prakash - you’ve helped with similar issues before, can you take a look?

Automatically suggested by n8n’s community bot. It’s a pilot - please share feedback here.

[Niffzy]

Hi @Greg_John A good production approach is to treat rate limits as expected behavior rather than simply retrying everything immediately.

API Request

↓

429 / Temporary Error?

↓ Yes

Wait + Exponential Backoff

↓

Retry

↓

Maximum Attempts Reached?

↓ Yes

Dead-Letter / Error Workflow

Recommended approach

Respect the API’s Retry-After value when provided

Use exponential backoff for temporary failures

Set a maximum retry count

Limit concurrent requests to the API

Send permanently failed jobs to an error/recovery workflow

For example: Retry 1 → 2 seconds

Retry 2 → 4 seconds

Retry 3 → 8 seconds

Retry 4 → 16 seconds

The exact delays should depend on the API’s limits.

The key is to avoid retry storms. If hundreds of executions receive a 429 at the same time and all retry immediately, you can make the problem even worse.

For high-volume systems, combining concurrency limits + exponential backoff + a recovery path usually provides a much more reliable setup.

[achamm]

@Greg_John HTTPS requests have retry times with a wait, you could use that to help you, but you can’t really make rate limits faster unless you have a higher plan. You can set it up so they do batching, and retry on fail, + wait nodes will help!

setting up error workflows can also help with monitoring it easier!

Heres a json i just made for anothr similar post that applys here also!

[pirateprentice]

Good answers above. A few things that specifically address your concern about one slow API blocking other workflows:

**Isolate per-API execution with sub-workflows**

The key insight: don’t handle rate limits inside your main workflow. Move each external API call into its own sub-workflow (Execute Workflow node). Your main flow stays fast; the sub-workflow handles its own retry logic and can run independently without blocking other parallel executions.

**Track quota state across executions**

The built-in HTTP retry only handles transient failures. For true rate limit management across concurrent executions, you need state that persists between runs:

1. Before the API call, read remaining quota from a key-value store (Airtable row, Redis via HTTP, or a Google Sheet)

2. If quota is close to empty, write a “cooldown until” timestamp and skip

3. After a successful call, decrement the counter

This prevents multiple concurrent workflow executions from hammering the same API quota simultaneously.

**Circuit breaker pattern**

Add a circuit breaker check early in your API sub-workflow:

- Read a “circuit state” flag from your KV store

- If `open` (tripped after N consecutive 429s), return a structured error immediately without touching the API

- A separate scheduled workflow resets the flag after the cooldown expires

This is what stops a rate-limited API from causing cascading failures across everything that depends on it.

**Dead-letter queue**

After max retries, don’t just log the error — write the failed payload to an Airtable table or a webhook queue. A separate recovery workflow retries these on a schedule, or flags them for Slack review.

The exponential backoff + Wait node approach @Niffzy outlined is solid for the retry layer. The circuit breaker + external state layer is what makes it production-safe when multiple workflows share the same API quota.

-–

If this is for a system where failures have real cost (lost orders, missed triggers), this is the kind of architecture we handle in done-for-you builds at [occelatus.io/automate](Automation Services — Occelatus Labs). Happy to answer questions here either way.

[bozad]

Good answers above on backoff and sub-workflow isolation. Three things that
bite in production and have not come up yet.

Do not retry every failure. Only 429 and 5xx are worth retrying. A 400, 401
or 403 will fail identically on every attempt, so retrying them just burns your
retry budget and makes a bad credential look like a rate-limit problem. Branch
on the status code before the retry logic, not after. This is the single most
common cause of “my retries are not helping”.

Add jitter to the backoff. Plain exponential backoff means every parallel
item that got throttled at the same moment retries at the same moment, and you
re-trigger the limit in lockstep. Randomise each delay by something like plus or
minus 30 percent. With ten concurrent items this is the difference between
recovering on the second attempt and thrashing for a minute.

Make retried writes idempotent. This is the one that actually loses money.
If a POST times out or returns 502, you often do not know whether the server
processed it. Retrying can create the record twice. If the API supports an
idempotency key, send one and reuse it across retries of the same logical
operation. If it does not, check for the record before writing on any attempt
after the first. Retrying a read is free; retrying a write is not.

One more thing on your retry_after: 30 example: honour that value rather than
your own backoff curve when the API sends it. It is also worth knowing there are
two different situations behind a 429, and they need different responses. A
per-endpoint window limit means the quota is exhausted until a fixed reset time,
so the only fix is waiting and pacing your request rate. General service
throttling means you are simply sending too fast right now, so backoff plus
lower concurrency clears it. If the response carries a reset timestamp you are
in the first case; if it only carries Retry-After you are usually in the
second.