N8N Queue mode load testing

I’m working on running N8N in Kubernetes, with 1 main container and 3 queue worker containers.

While I struggle with getting everything in K8s configured (I’m still pretty new to it), I was wondering if anyone had a good workflow (or idea for workflow) to test performance across the distributed workers. That is, I’d like to fire off “Workflow 1” with 3 queue workers, get a processing time, and then scale it up to 4 or 5 workers and see a shorter time.

Right now, I’m using a function node to create 1000 simple items, break those into batches, and send them off to a sub-Workflow which just adds a new field to each item. I use the Respond to Webhook node after the batching is done to get an overall time benchmark.

Does that seem reasonable? Are there better ways to construct workflows to take advantage of parallel processing across the workers? Any advice for constructing a solid test workflow would be gratefully received.

1 Like

Hi @Ian_Walls, I hope you’re well?

Just wanted to let you know this has already been flagged to @krynble internally who’d be best placed to provide more in-depth insights into the queue mode :slight_smile:

1 Like

Hey @Ian_Walls

This question you raised is something we are also working on internally to crunch some numbers.

There is an internal project for this and we intend to publish the results as soon as it is finished.

On a quick glimpse, our goal is to better understand how n8n will behave according to different hardware specs and types of workloads - IO intensive, CPU intensive, data intensive, etc.

I don’t have any details to share right now as we’re working on other projects but stay connected to our blog and social channels for updates.

1 Like

So, after some testing, I modified my workflow to fire off HTTP Requests, rather than included Workflows. The ‘subtask’ workflow is webhook triggered, and I have some initial numbers that make sense to me:

With 5000 records, in batches of 100, and 4 workers with concurrency 10 each:

  • ~40s to process with ‘respond on completion’ option for the subtask Webhook node
  • ~8s to process with ‘respond immediately’, and another 3-5 seconds for all the subtasks to wrap up

The next trick I want to figure out is how to get the main process to notify me of the results of the complete run while still having the subtask’s Webhook node respond immediately. I’m guessing I should have the subtask fire off an HTTP request somewhere on completion, but that listener would need to wait for all the worker processes to return either success or failure, and then act. Does anyone have any ideas how to set something like this up?