Autoscaling Queue Workers on Google Cloud Run (Redis/BullMQ) – How do you solve the HTTP-blindness?

Feedback
,
1

Hey everyone,

My team and I are currently deploying n8n in Queue Mode (Main + Workers via Redis) on Google Cloud Run. We aim for a serverless, scale-to-zero (ornear-zero) architecture to keep idle costs low.

However, we hit a fundamental architectural wall:
Google Cloud Run’s native autoscaler works by monitoring incoming HTTP requests. But the n8n-worker instances don’t receive HTTP traffic (except maybe a health check ping); they actively pull jobs from the Redis queue.

Because Cloud Run is completely blind to the Redis queue depth, it never automatically scales out the worker instances, even if 100 heavy jobs are piling up in Redis. Cloud Run just sees “0 HTTP requests” and keeps the worker at 1 instance (or scales it to 0).

My Questions to the community:

  1. How are you horizontally scaling your n8n workers on Cloud Run?

2. Are you using external metric autoscalers (like CREMA / KEDA) to ping the Google Cloud API based on Redis queue depth? (Seems very complex for a serverless setup).
3. Or did you give up on Queue Mode for Cloud Run and went back to standard mode?

Note: We currently found a workaround by strictly limiting the worker to 1 vCPU and using a continuous health-ping to keep the container awake. By doing this, heavy jobs force the single CPU to 100%, which successfully tricks the Cloud Run CPU-autoscaler into spinning up more instances. It works well, but I’m curious if there’s a more “official” or elegant solution out there!

Would love to hear your deployment strategies!

1 « J'aime »
2

Hey Yoel!
Not sure if you already solved this but i ran into your post while looking into Cloud Run with queue-based workers and your diagnosis looks right to me!
Google now has a simpler solution for this with Cloud Run Worker Pools + CREMA that can adjust the instance count based on an external metric/signal, (even though worker pools still don’t autoscale).
So yeah, your workaround is basically compensating for this and it’s not crazy at all!
I haven’t tested CREMA against n8n’s Redis queue yet so i can’t say how hard the integration is, but it seems closer to what you wanted! Let me know if you found another (easier) solution!