Scaling strategy for n8n workers

Questions
,
1

I’ve got a self-hosted (or, rather, AWS-hosted) n8n running in queue mode. My workers are now scaling out and in depending just on CPU and memory utilization. I wonder if there’s a better approach/strategy for this scaling process?

Thinking about it, checking the length of the job queue (i.e. how many jobs are waiting for an available worker to pick it up) seems like a sensible approach. But I’m not sure how to do that. Is this something that n8n provides as a metric that I can use for the autoscaling strategy? Any common way to do this (or similar) with n8n?

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Hi @Mykra, thank you for reaching out and being a part of the community!

There have been a few conversations in our forum and others regarding autoscaling n8n’s queue mode.

To answer your question, we don’t provide an out-of-the-box method for monitoring queue length. That said, since we use Redis for our queue mode then you could in theory monitor Redis to get the length of the queues.

Then you could publish that number to CloudWatch as a custom metric and create an autoscaling policy that adjusts the number of workers.

I hope that helps guide you in the right direction! Happy building!

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.