N8n scaling - queue mode

Trying the Queue Execution Mode

  • n8n version: 1.57
  • Database (default: SQLite): Postgres
  • n8n EXECUTIONS_PROCESS setting (default: own, main): Main
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • Operating system: Ubuntu

Have setup a Main container, Worker container and Webhook container
Have a simple test flow XYZ with a Webhook and a Respond to Webhook node.

If we call the test flow through the main instance - it works.
If we call the test flow through the Webhook instance it hangs. It runs in the back and and completes successfully, but no data is returned, it just spins for a while and then times out. What are we doing wrong ?

Below are the docker commands for Main, Worker and Webhook

Main instance
sudo docker run -d --restart always --name n8n -p 5678:5678
-e DB_TYPE=postgresdb
-e DB_POSTGRESDB_DATABASE=n8ndb
-e DB_POSTGRESDB_HOST=postgresdb
-e DB_POSTGRESDB_PORT=5432
-e DB_POSTGRESDB_USER=postgres
-e DB_POSTGRESDB_SCHEMA=public
-e DB_POSTGRESDB_PASSWORD=password123
-e QUEUE_BULL_REDIS_HOST=redisthost
-e QUEUE_BULL_REDIS_PORT=6379
-e QUEUE_BULL_REDIS_PASSWORD=password123
-e QUEUE_BULL_REDIS_DB=5
-e N8N_ENCRYPTION_KEY=“encryptionKey”
-e EXECUTIONS_MODE=queue
-e N8N_METRICS=true
-e QUEUE_HEALTH_CHECK_ACTIVE=true
-e N8N_LOG_LEVEL=debug
-e N8N_LOG_OUTPUT=console,file
-e N8N_LOG_FILE_MAXSIZE=50
-e N8N_LOG_FILE_MAXCOUNT=60
-v n8n_main:/home/node/.n8n
docker.n8n.io/n8nio/n8n


Worker instance
sudo docker run -d --name n8n-worker -p 5679:5678
–restart always
-e DB_TYPE=postgresdb
-e DB_POSTGRESDB_DATABASE=n8ndb
-e DB_POSTGRESDB_HOST=postgresdb
-e DB_POSTGRESDB_PORT=5432
-e DB_POSTGRESDB_USER=postgres
-e DB_POSTGRESDB_SCHEMA=public
-e DB_POSTGRESDB_PASSWORD=password123
-e QUEUE_BULL_REDIS_HOST=redishost
-e QUEUE_BULL_REDIS_PORT=6379
-e QUEUE_BULL_REDIS_PASSWORD=password123
-e QUEUE_BULL_REDIS_DB=5
-e N8N_ENCRYPTION_KEY=“encryptionKey”
-e EXECUTIONS_MODE=queue
-e N8N_METRICS=true
-e QUEUE_HEALTH_CHECK_ACTIVE=true
-e N8N_LOG_LEVEL=debug
-e N8N_LOG_OUTPUT=console,file
-e N8N_LOG_FILE_MAXSIZE=50
-e N8N_LOG_FILE_MAXCOUNT=60
docker.n8n.io/n8nio/n8n worker


WEBHOOK instance (this is where the trouble starts)
sudo docker run -d --name n8n-webhook -p 5680:5678
–restart always
-e DB_TYPE=postgresdb
-e DB_POSTGRESDB_DATABASE=n8ndb
-e DB_POSTGRESDB_HOST=postgresdb
-e DB_POSTGRESDB_PORT=5432
-e DB_POSTGRESDB_USER=postgres
-e DB_POSTGRESDB_SCHEMA=public
-e DB_POSTGRESDB_PASSWORD=password123
-e QUEUE_BULL_REDIS_HOST=redishost
-e QUEUE_BULL_REDIS_PORT=6379
-e QUEUE_BULL_REDIS_PASSWORD=password123
-e QUEUE_BULL_REDIS_DB=5
-e N8N_ENCRYPTION_KEY=“encryptionKey”
-e EXECUTIONS_MODE=queue
-e N8N_METRICS=true
-e QUEUE_HEALTH_CHECK_ACTIVE=true
-e N8N_LOG_LEVEL=debug
-e N8N_LOG_OUTPUT=console,file
-e N8N_LOG_FILE_MAXSIZE=50
-e N8N_LOG_FILE_MAXCOUNT=60
docker.n8n.io/n8nio/n8n webhook

This has been answered on email, The first thought was maybe the load balancer for the webhook workers is not correctly configured.