Describe the problem/error/question
This is a starting point for testing concurrency issues in worker nodes. The expected results below match my observations, but… I have seen comments where different combinations of webhook calls, responding immediately or not, etc. seems to produce different behavior. For example, similar tests in some circumstances might run workflows serially, even on different workers, perhaps because they are blocking each other on some other resource (memory? webhook handlers? other?) inside or outside of n8n.
What is the error message (if any)?
No error. Just posting a “head start” for setting things up to observe queue mode behavior.
There is a third party utility called n8nworkerview which may also help for watching worker node behavior. (However, note that it needs a slight tweak to work on recent versions of n8n.)
Please share your workflow
These will need small adjustments to the webhook paths / endpoints to re-hook them together in another environment.
Indirect starter for the split-out driver workflow, to make it run on a worker. This keeps the “main/parent” workflow from running on the “main” instance of n8n.
Driver / generator workflow to initiate enough “child” workflows to determine if they’re running serially or concurrently.
Child workflow that should be queued up and processed by one of the workers. Each call runs this and does not respond until the delay has passed. Note: There seems to be a 60 second threshold for a Wait that causes the process to “dehydrate” instead of remaining “inline”/“active” while it waits.
Share the output returned by the last node
Expected behavior: Workers grab a somewhat equal share of the sub-workflows and run them concurrently, independently of the “parent/driver” workflow that started all 100 of them via webhook.
Example 1: Three (3) workers set to concurrency of 40 each - all workflows run at the same time (~33 per worker), and the total time the driver workflow runs is ~10 seconds.
Example 2: Five (5) workers set to concurrency of 11 each - 50 workflows can run at the same time (~10/worker), and the total time the driver workflow runs is ~20 seconds.
Example 3: Two (2) workers set to concurrency of 5 each - 20 workflows can run at the same time, and the total time the driver workflow runs is ~50 seconds.
This might also be useful for testing whether additional webhook processing nodes make any difference.
Information on your n8n setup
- n8n version: 1.71.3
- Database (default: SQLite): Postgres
- n8n EXECUTIONS_PROCESS setting (default: own, main): queue
- Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
- Operating system: (Host) MacOS
