Worker fails to start with Redis Unavailable message

[phpguru]

Describe the issue/error/question

I am attempting to create a 3-droplet setup at Digital Ocean with a MySQL database. I am using the n8n-digital-ocean github repo as a starting point. One is the master node, one is the worker node, and one is the redis server.

I have installed redis-tools package on both the master and the worker nodes for testing, and can confirm that I can connect to the redis server from both servers.

Redis starts up fine.
The master starts up fine.
The worker fails to start.

I am using this line from your docs to start the worker

docker run --name n8n-queue -p 5679:5678 n8nio/n8n n8n worker

Initially I tried with a Redis password, but disabled it and the result is the same.

What is the error message (if any)?

Here is the complete output

root@ubuntu-s-4vcpu-8gb-intel-nyc3-01:~/n8n-digital-ocean# docker run --name n8n-queue -p 5679:5678 n8nio/n8n n8n worker
Starting n8n worker...
UserSettings were generated and saved to: /home/node/.n8n/config
Migrations in progress, please do NOT stop the process.
query is slow: CREATE TABLE IF NOT EXISTS "credentials_entity" ("id" integer PRIMARY KEY AUTOINCREMENT NOT NULL, "name" varchar(128) NOT NULL, "data" text NOT NULL, "type" varchar(128) NOT NULL, "nodesAccess" text NOT NULL, "createdAt" datetime NOT NULL, "updatedAt" datetime NOT NULL)
execution time: 3455

n8n worker is now ready
 * Version: 0.194.0
 * Concurrency: 10

Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Migrations finished.
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Redis unavailable - trying to reconnect...
Unable to connect to Redis after 10000. Exiting process.

Here is a redis test on the same worker node

root@ubuntu-s-4vcpu-8gb-intel-nyc3-01:~/n8n-digital-ocean# redis-cli -h 10.108.0.6
10.108.0.6:6379> KEYS *
(empty array)
10.108.0.6:6379> SET foo bar
OK
10.108.0.6:6379> GET foo
"bar"

Information on your n8n setup

• 0.194.0
• MySQ
• Master node with 4 vCPU 8GB Memory @ Digital Ocean
• Worker node with 4 vCPU 8GB Memory @ Digital Ocean

[MutedJam]

Hey @phpguru, this sounds like n8n can’t talk to your Redis instance. Did you set QUEUE_BULL_REDIS_HOST and QUEUE_BULL_REDIS_PORT on your worker instance?

[phpguru]

Yes. I ended up figuring out that when you start the worker with docker run you must pass -e environment variables. I made the mistake of thinking that the worker is a complete copy of n8n but when you docker run the worker, it’s a complete instance all on its own and has no environment.

So my guess is that by default it was just expecting redis to be at localhost.

After passing all the -e flags it worked.

But I am still not 100% sure how to easily verify that the system is running in queue mode and the worker is functioning as it should.

Also, how are people “autoscaling” workers? Clearly monitoring my system and logging in via SSH to docker run more workers is not the way.

[doomer]

Hi !

Thanks for the above answers, helped me.
If anyone is wondering, do not hesitate to pass almost all the docker environment variables that you may use in your main n8n master.

To help with the debugging, you might want to instanciate your worker with something like this and then progressively cleanup/user a docker file :

docker run -it --rm -e DB_TYPE=postgresdb -e DB_POSTGRESDB_HOST=<IP/HOST> -e DB_POSTGRESDB_PORT=5432 -e DB_POSTGRESDB_DATABASE=n8n -e DB_POSTGRESDB_USER=changeUser -e DB_POSTGRESDB_PASSWORD=changePassword -e N8N_BASIC_AUTH_ACTIVE=true -e EXECUTIONS_MODE=queue -e QUEUE_BULL_REDIS_HOST=<IP/HOST> -e QUEUE_BULL_REDIS_PORT=6379 -e N8N_ENCRYPTION_KEY=<ENCRYPTIONKEY> -v "~/yourpath:/home/node/.n8n" n8nio/n8n n8n worker