I have a sequence of workflows that fire when a user signs up or creates and account for the first time. One of these workflows does a high volume of API calls (can be 1000+) through loops to an API that allows 240 calls per min. I have tested the process with creating 5 users back to back and I ended up exceeding my API call limit and erroring the workflow. I’m thinking of scalability. I have played with timing between calls but that can double or triple what can already be a long process (it happens on the back end so I guess that is ok) But I was hoping someone could recommend a way to build a queue system. I have looked into queue mode but since my limitations are API related and not system based I wasn’t sure if that would be a good route. As I was brain storming I was thinking of instead of actually starting the workflow with the signup maybe add the user to a list, and then always have a process that it checking and processing that list (executing the workflow) one at a time?!? Could that be done? Thank you for any insight.
- n8n version: Version 0.227.1
- Database (default: SQLite): MariaDB
- n8n EXECUTIONS_PROCESS setting (default: own, main):
- Running n8n via (Docker, npm, n8n cloud, desktop app): npm
- Operating system: Ubuntu Server 22.04