My self hosted n8n instance deployed in queue mode using Google Cloud Run doesn't scale

[Gustavo_Trivelatto]

Describe the problem/error/question

I have a self-hosted instance of n8n running in my Google Cloud as a container in Cloud Run. Until recently I only used the basic mode with a single instance and never had a problem, but some weeks ago I started to identify the necessity of scaling it up. I found out about queue mode and implemented it, having a main instance (not scalable) and a worker instance (with scaling enabled in the revision settings).

Everything seems to be working fine, the UI shows me jobs getting in the queue, the jobs are being processed and everything works. However, it doesn’t seem to scale correctly. I’m currently facing a problem where many jobs are getting stuck in the queue and my processing time is sky rocketing due to it, and even with the queue being so full I never see my worker scaling to more than one instance.

What am I doint wrong? It doesn’t seem like my configs are wrong, do anyone know what could I be doing wrong?

What is the error message (if any)?

Please share your workflow

Since I don’t have a workflow I’ll be sharing the most relevant settings for both instances.

Main:

4 GB and 4 vCPUs
Fixed to 1 instance

Worker:

2 GB and 2 vCPUs
Min. 1 instance, Max. 100 instances

My worker recent metrics:

This is from the last 24h, where the queue got completely stuck and the worker refused to go beyond a single instance.

Information on your n8n setup

• n8n version: 1.80.5
• Database (default: SQLite): PostgreSQL
• n8n EXECUTIONS_PROCESS setting (default: own, main): own/queue
• Running n8n via (Docker, npm, n8n cloud, desktop app): Official Docker images on Google Cloud Run

[hubschrauber]

This is just a wild guess, but Google Cloud Run targets 60% utilization to scale up, and you have 2 vCPUs allocated for each worker container. If that is presented to the container as 2 separate cores, and the n8n process (node.js) doesn’t ever use more than 1 (node.js is single threaded), maybe you’re never going over 50% and never triggering the autoscaling.

Just as an experiment, you could see what happens if you allocate only 1 vCPU per container.

[Gustavo_Trivelatto]

It really worked! I didn’t try but I imagine that putting the concurrency limit higher would also lead to a similar effect, because I connected the Redis DB to a Bullboard instance and my queue seemed to be always stuck at 10 jobs a time. Maybe that’s why it was never going over the processing threshold.