I have a queue instance with a worker who is configured to have a concurrency of 30.
My understanding of concurrency is this worker can run up to 30 simultaneous tasks at the same time.
In my daily usage, I’m 100% happy with the performance and how good it is working.
Recently, I had to add a database query operation that require to run repetitive tasks (4 actions) to compare data.
So I built this with the idea ok 30 concurrency is like let’s say 20 actions every 3–4 seconds, that would make it in less an hour to complete the 15k entries.
But the reality is it runs aprox 10 tasks per minute… this is an image from one minute:
As you can see, most of the time, the “executions of the workflow” are 1-3 sec each one. I don’t understand why it’s just creating a few per minute… at this rate it would need a day or two to complete.
Any ideas? Should I try increasing the concurrency? Add another worker to the instance? I still don’t understand what is the difference between a worker with 30 concurrency or 10 workers with 3 concurrencies each…
My worker has access up to 32vcpu and 32GB ram. I had set up a timeout for the main workflow of 1 hour, and it hit the hour, this is the usage of the worker for that hour:
I did expect a very high usage for like 30–45 minutes, maybe 5 or 6gb ram usage and cpu but to perform this execution task at the earliest…
- n8n version: 1.16.0
- Database (default: SQLite): Postgresql
- n8n EXECUTIONS_PROCESS setting (default: own, main): queue
- Running n8n via (Docker, npm, n8n cloud, desktop app): docker image + railway