Handle rate limits when an n8n workflow processes thousands of API requests

Questions
1

Hello
I’m building an n8n workflow that processes a large number of records and makes an API request for each one. It works fine with smaller batches, but when the volume increases, I start getting 429 Too Many Requests responses.
I don’t want to simply add a long fixed delay because the API has different rate limits depending on the endpoint.
I’m currently considering something like: const retryAfter = $json.retryAfter || 2;
return [{
json: {
retryAfter,
nextAttempt: new Date(Date.now() + retryAfter * 1000).toISOString()
}
}];
What’s the best production approach for handling API rate limits in n8n?

Describe the problem/error/question

What is the error message (if any)?

Please share your workflow

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Information on your n8n setup

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
2

Hey @anon20761130, while you wait for a response, here are some things that might help:

Suggested resources

Automatically matched to your question.

Docs:

Forum:

@Niffzy - you’ve helped with similar issues before, can you take a look?

Automatically suggested by n8n’s community bot. It’s a pilot - please share feedback here.

3

Hi @anon20761130 I would use a combination of batching, concurrency control, and exponential backoff, rather than relying on fixed delays.

First, limit how many requests n8n sends at once. Then, when the API returns 429, respect the Retry-After header if it exists.

For retries, use something like:

const attempt = $json.attempt || 0;
const retryAfter = Number($json.retryAfter) || Math.pow(2, attempt);

return [{
json: {
...$json,
attempt: attempt + 1,
retryAfter
}
}];

I would also set a maximum retry count and move repeatedly failing requests into a separate error/dead-letter workflow.

The important part is to control the request rate before the API starts rejecting you, rather than trying to recover from thousands of 429 errors afterward.

4

@anon20761130

Since your API provides specific instructions on when to retry, you should treat the 429 response as a piece of data rather than a workflow failure.

Use the following workflow structure:

  1. HTTP Request Node:
    • Settings > On Error: Set to Continue (using error output). This is crucial; it prevents the workflow from stopping when the 429 occurs.
  2. If Node:
    • Check if the response status is 429.
  3. Code Node (The “Parser”):
    • If it’s a 429, extract the Retry-After header.
    • Note: Some APIs return seconds, others return a timestamp. Your code should normalize this to a number of milliseconds.
const response = $json.error || $json; // Depending on how n8n returns error output
const retryAfterHeader = response.headers?.['retry-after'];

// Convert header to milliseconds
let waitMs = 5000; // Default fallback
if (retryAfterHeader) {
    // If header is a number (seconds), convert to ms. 
    // If it's a date string, calculate the difference.
    waitMs = isNaN(retryAfterHeader) 
        ? new Date(retryAfterHeader).getTime() - Date.now() 
        : parseInt(retryAfterHeader) * 1000;
}

return {
    retry: true,
    waitMs: Math.max(waitMs, 1000), // Safety minimum of 1s
    data: $json // Pass original data along to retry
};
  1. Wait Node:
    • Set the Wait Amount to an expression: {{ $json.waitMs }}.
  2. Loop Back:
    • Connect the Wait node back to the HTTP Request node to retry the specific record.

For looping, these safeguards must be implemented:

  • Maximum Retry Count: In your Code node, increment a retryCount property. If retryCount > 5, route the item to an error branch instead of looping again. This prevents infinite loops that consume your worker resources.
  • Dead Letter Queue (DLQ): Instead of letting the workflow simply “fail,” route items that exceed the max retries to a separate “Error Handling” branch. This branch should log the failure to a database (like Supabase or Airtable) or send a Slack alert. This allows you to manually inspect and re-run only the failed records later.
5

Hi @anon20761130 Welcome!
Different limits per endpoint is an argument for pacing each node separately rather than building one shared retry path. The HTTP Request node has Batching under Add Option, with Items per Batch and Batch Interval, which the docs describe as the equivalent of Loop Over Items plus Wait. Being a per-node setting, the node calling your strict endpoint carries its own interval and the node calling the generous one carries another, with no surrounding logic in either.
Retry On Fail in that node’s Settings catches what still slips through. Its Wait Between Tries is a fixed number rather than anything read from the response, so parsing the header by hand only earns its complexity if the value the server sends actually varies from run to run.