Question on Dynamic Rate Limits with Scheduled Workflows

Hi everyone,

I’m currently working on a workflow in n8n where I am integrating Airtable with Apollo.io to retrieve emails for my Ideal Client Profile (ICP). However, I’m running into some challenges due to the rate limits imposed by Apollo.io.

Context:

  • Apollo.io has three levels of rate limits: daily, hourly, and minute. (it has enrichment api rate limit and contacts api rate limit )
  • I have set up two dynamic rate limit checkers in my workflow to monitor these limits.
  • The workflow starts with a Schedule Trigger that runs every day at 9 AM.

Problem:

The core issue arises when one of the rate limit checkers hits its daily limit at 11 AM. When this happens, it should wait for 24 hours before proceeding, meaning that the rate limit will be refreshed the next day at 11 AM. However, since the schedule trigger runs at 9 AM, it starts the workflow again at that time, which could lead to a 429 error because the daily limit hasn’t been reset yet.

Current Limitations:

  • As I am on a self-hosted version of n8n (version 1.61.0), I cannot use persistent variables or workflow static data to check for refresh times.
  • I need a way to ensure that my workflow does not attempt to run tasks that would exceed the rate limits.

Question:

  1. How can I implement a solution that dynamically checks the rate limits before the scheduled trigger runs at 9 AM?
  2. What strategies can I use to prevent the workflow from executing if the rate limit has been hit, given that I cannot store or use persistent variables?
  3. Is there a way to adjust the workflow logic to either delay the next execution or modify the schedule based on when the rate limit resets?
  4. Are there any best practices for managing multiple rate limit checkers within a single workflow that can help avoid the 429 error?

Any insights or suggestions from the community would be greatly appreciated! Thank you in advance for your help.

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Hey @Rohit_Gurav , I would start with understanding which limit you actually hit, daily, hourly, and minute?

If a single execution flow causes this issue, you need to introduce delays between HTTP requests you make to Apollo. I suspect you might have many items in a set of items and each trigger individual call to the API. If that is the case, add Loop node with Wait to ensure each item triggers the HTTP request with some delay ensuring the “minute” limit threshold is not broken.

Another approach (if supported by Apollo API) is making a single call that would include all the items (or some of them) rather than individual item call. This would significantly reduce the number of requests sent to the API.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.