QUEUE MODE is not working ("Starting Soon")

Questions
1

Describe the problem/error/question

Hi,

My issue is the same as here. We have setup a queue mode with redis but as soon as we turn on EXECUTION_MODE=queue, our webhook keep saying “Starting Soon”. When we pass it again on “regular”, everything goes well.

We see in the main logs that job is added to the queue but nothing happen on the worker level.

I have posted yesterday about an issue with our startup licence, this is not clear for me if these two are related. Or should i say if the redis service can not work without a Startup/Entreprise licence (this is not clear for me yet).

Here is bellow our config

  • DB_TYPE=postgresdb

    postgres instance details

    • DB_POSTGRESDB_HOST=${DB_HOST}
    • DB_POSTGRESDB_PORT=${DB_PORT}
    • DB_POSTGRESDB_DATABASE=${DB_NAME}
    • DB_POSTGRESDB_SCHEMA=public

    redis instance details

    • EXECUTIONS_MODE=queue
    • QUEUE_MODE=redis
    • QUEUE_BULL_REDIS_HOST=${REDIS_HOST}
    • QUEUE_BULL_REDIS_PORT=${REDIS_PORT}
      #- QUEUE_BULL_REDIS_CONNECTION_TIMEOUT=30000
      #- QUEUE_BULL_REDIS_TIMEOUT=30000
      #- QUEUE_BULL_REDIS_KEEP_ALIVE_DELAY=15000
      #- QUEUE_BULL_REDIS_PREFIX=n8n
      #- QUEUE_BULL_PREFIX=jobs
    • QUEUE_BULL_REDIS_DB=0

    n8n configuration

    • N8N_SECURE_COOKIE=false
    • N8N_BASIC_AUTH_ACTIVE=true
    • N8N_BASIC_AUTH_USER=${N8N_USER}
    • N8N_HOST=n8n.NAME.fr
      #- N8N_DISABLE_PRODUCTION_MAIN_PROCESS=true
    • WEBHOOK_URL=${URL}
    • N8N_PORT=5678
    • N8N_BINARY_DATA_STORAGE=s3
      #- N8N_DEFAULT_BINARY_DATA_MODE=s3
    • N8N_AVAILABLE_BINARY_DATA_MODES=s3
      #- N8N_EXTERNAL_STORAGE_S3_HOST=${N8N_S3_HOST}
    • N8N_EXTERNAL_STORAGE_S3_HOST=10.10.0.68:9000
      #- N8N_EXTERNAL_STORAGE_S3_BUCKET_REGION=${N8N_S3_REGION}
    • N8N_EXTERNAL_STORAGE_S3_BUCKET_REGION=IAS-PRD-STNS
    • N8N_EXTERNAL_STORAGE_S3_PROTOCOL=http
    • N8N_EXTERNAL_STORAGE_S3_BUCKET_NAME=n8n-bucket
    • N8N_EXTERNAL_STORAGE_S3_ACCESS_KEY=n8nuser
    • N8N_EXTERNAL_STORAGE_S3_ACCESS_SECRET=secret

    Secrets data

    • N8N_BASIC_AUTH_PASSWORD_FILE=/run/secrets/N8N_PASSWORD
    • EXECUTIONS_DATA_SAVE_ON_ERROR=all
    • EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
    • EXECUTIONS_DATA_SAVE_ON_PROGRESS=true
    • EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=false
    • EXECUTIONS_DATA_PRUNE=true
    • EXECUTIONS_DATA_MAX_AGE=168
    • EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000
    • NODE_ENV=production
    • N8N_LOG_LEVEL=debug

Information on your n8n setup

  • n8n version: 1.91.3
  • **Database (default: SQLite):POSTGRES
  • **n8n EXECUTIONS_PROCESS setting (default: own, main):main
  • **Running n8n via (Docker, npm, n8n cloud, desktop app):Docker

THank you so much !

2

Change the following or review your current settings:

  • EXECUTIONS_MODE=queue

  • N8N_RUNNERS_ENABLED=true

  • OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS=true

  • N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true

  • QUEUE_BULL_REDIS_HOST=redis

  • QUEUE_BULL_REDIS_PORT=6379

  • QUEUE_BULL_REDIS_DB=2

  • QUEUE_BULL_REDIS_HOST=redis

  • QUEUE_BULL_REDIS_PORT=6379

  • QUEUE_BULL_REDIS_DB=2

  • EXECUTIONS_MAX=50

3

Hi @mpst

Just to share what I know—storing n8n’s binary data in S3 is currently an enterprise-tier feature. You can find the details in their documentation here: External storage for binary data | n8n Docs

On the other hand, Redis can be used without a license, but S3 support for binary data unfortunately requires the enterprise plan.

4

Thank you. Will try the solution.

But concerning s3, i don’t get it : i still get the execution data when redis is disabled. So i guess it is working even without the licence ? Or maybe this is the combo of these two that do not work. At least we could get any kind of notification… I don’t konw.

5

FYI the solution provided has not worked. We finaly got a new startup key and the issue still sticks. I see that many people have posted about this but no one provided a clear solution :frowning:

6

How are you deploying your worker node? Can you share ure dockerfile, it needs to run ‘n8n worker’ command.

7

Here you go

# Swarm-compatible docker-compose file for n8n

# Environment variables:

# - DB_HOST: Hostname of the PostgreSQL database

# - DB_PORT: Port of the PostgreSQL database

# - DB_NAME: Name of the PostgreSQL database

# - REDIS_HOST: Hostname of the Redis instance

# - REDIS_PORT: Port of the Redis instance

# - N8N_USER: Username for basic authentication

# - N8N_WORKER_REPLICAS: Number of worker replicas

# - N8N_S3_HOST: Hostname of the S3-compatible storage

# - N8N_S3_BUCKET: Name of the S3 bucket

# - N8N_S3_REGION: Region of the S3 bucket

# - ENV_DOMAIN: Domain name for the n8n instance

# Docker host secrets:

# - POSTGRES_USER: PostgreSQL username

# - POSTGRES_PASSWORD: PostgreSQL password

# - N8N_PASSWORD: n8n basic authentication password

# - N8N_ENCRYPTION_KEY: n8n encryption key

# - S3_ACCESS_KEY: S3-compatible storage access key

# - S3_SECRET_KEY: S3-compatible storage secret key

# - REDIS_PASSWORD: Redis password

#export N8N_EXTERNAL_STORAGE_S3_BUCKET_NAME=$(cat /run/secrets/N8N_S3_BUCKET) &&
#export N8N_EXTERNAL_STORAGE_S3_ACCESS_KEY=$(cat /run/secrets/N8N_S3_ACCESS_KEY) &&
#export N8N_EXTERNAL_STORAGE_S3_SECRET_KEY=$(cat /run/secrets/N8N_S3_SECRET_KEY) &&
#env | grep -i "n8n|key|bucket|s3" &&


version: ‘3.8’
x-shared: &shared
image: {url}/n8n-with-packages

# As certain variables are not designed to be read from secret files, we need to override the default entrypoint

entrypoint: >
/bin/sh -c "
export N8N_ENCRYPTION_KEY=$$(cat /run/secrets/N8N_ENCRYPTION_KEY) &&
export N8N_BASIC_AUTH_PASSWORD=$$(cat /run/secrets/N8N_PASSWORD) &&
export DB_POSTGRESDB_USER=$$(cat /run/secrets/N8N_POSTGRES_USER) &&
export DB_POSTGRESDB_PASSWORD=$$(cat /run/secrets/N8N_POSTGRES_PASSWORD) &&
export QUEUE_BULL_REDIS_PASSWORD=$$(cat /run/secrets/REDIS_PASSWORD) &&
export N8N_LICENSE_ACTIVATION_KEY=$$(cat /run/secrets/N8N_LICENSE_KEY) &&
exec n8n"
environment:
- N8N_EDITION=ee
- DB_TYPE=postgresdb
# postgres instance details
- DB_POSTGRESDB_HOST=${DB_HOST}
- DB_POSTGRESDB_PORT=${DB_PORT}
- DB_POSTGRESDB_DATABASE=${DB_NAME}
- DB_POSTGRESDB_SCHEMA=public
# redis instance details
- EXECUTIONS_MODE=queue
- QUEUE_MODE=redis
- QUEUE_BULL_REDIS_HOST=${REDIS_HOST}
- QUEUE_BULL_REDIS_PORT=${REDIS_PORT}
#- QUEUE_BULL_REDIS_CONNECTION_TIMEOUT=30000
#- QUEUE_BULL_REDIS_TIMEOUT=30000
#- QUEUE_BULL_REDIS_KEEP_ALIVE_DELAY=15000
#- QUEUE_BULL_REDIS_PREFIX=n8n
#- QUEUE_BULL_PREFIX=jobs
- QUEUE_BULL_REDIS_DB=0
# n8n configuration
- N8N_SECURE_COOKIE=false
- N8N_BASIC_AUTH_ACTIVE=true
- N8N_BASIC_AUTH_USER=${N8N_USER}
- N8N_HOST=[n8n.NAME.fr](http://n8n.name.fr/)
#- N8N_DISABLE_PRODUCTION_MAIN_PROCESS=true
- WEBHOOK_URL=[https://n8n.NAME.fr](https://n8n.name.fr/)
- N8N_PORT=5678
- N8N_BINARY_DATA_STORAGE=s3
#- N8N_DEFAULT_BINARY_DATA_MODE=s3
- N8N_AVAILABLE_BINARY_DATA_MODES=s3
#- N8N_EXTERNAL_STORAGE_S3_HOST=${N8N_S3_HOST}
- N8N_EXTERNAL_STORAGE_S3_HOST=10.10.0.68:9000
#- N8N_EXTERNAL_STORAGE_S3_BUCKET_REGION=${N8N_S3_REGION}
- N8N_EXTERNAL_STORAGE_S3_BUCKET_REGION=IAS-PRD-STNS
- N8N_EXTERNAL_STORAGE_S3_PROTOCOL=http
- N8N_EXTERNAL_STORAGE_S3_BUCKET_NAME=n8n-bucket
- N8N_EXTERNAL_STORAGE_S3_ACCESS_KEY=n8nuser
- N8N_EXTERNAL_STORAGE_S3_ACCESS_SECRET=[secret]
# Secrets data
- N8N_BASIC_AUTH_PASSWORD_FILE=/run/secrets/N8N_PASSWORD
- EXECUTIONS_DATA_SAVE_ON_ERROR=all
- EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
- EXECUTIONS_DATA_SAVE_ON_PROGRESS=true
- EXECUTIONS_DATA_SAVE_MANUAL_EXECUTIONS=false
- EXECUTIONS_DATA_PRUNE=true
- EXECUTIONS_DATA_MAX_AGE=168
- EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000
- NODE_ENV=production
- N8N_LOG_LEVEL=debug
- N8N_RUNNERS_ENABLED=true
- OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS=true
#- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- QUEUE_BULL_REDIS_DB=2
- N8N_COMMUNITY_PACKAGES_ENABLED=true
- N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true
#- NODE_FUNCTION_ALLOW_BUILTIN=crypto
#- NODE_FUNCTION_ALLOW_EXTERNAL=googleapis,axios,lodash,bwip-js,minio
networks:
- n8n
- traefik-public
secrets:
- N8N_POSTGRES_USER
- N8N_POSTGRES_PASSWORD
- N8N_PASSWORD
- N8N_ENCRYPTION_KEY
- N8N_S3_ACCESS_KEY
- N8N_S3_SECRET_KEY
- N8N_S3_BUCKET
- N8N_LICENSE_KEY
- REDIS_PASSWORD
services:
n8n-main:
<<: *shared
#ports:
# - 5678:5678
deploy:
replicas: 1
restart_policy:
condition: on-failure
placement:
constraints:
- node.role == manager
- node.labels.role != database
labels:
- “traefik.enable=true”
- “traefik.http.routers.n8n-main.rule=Host(`n8n.NAME.fr`)”
- “traefik.http.routers.n8n-main.entrypoints=web”
- “traefik.http.services.n8n-main.loadbalancer.server.port=5678”
- “traefik.tags=traefik-public”
- “traefik.docker.network=traefik-public”

n8n-worker:
<<: *shared
deploy:
replicas: ${N8N_WORKER_REPLICAS}
restart_policy:
condition: on-failure
placement:
constraints:
- node.role == worker
- node.labels.role != database
command: worker
depends_on:
- n8n

redis:
image: redis:6
deploy:
placement:
constraints:
- node.role == worker
- node.labels.role != database
restart: always
networks:
- n8n

networks:
n8n:
external: true
traefik-public:
external: true

secrets:
N8N_POSTGRES_USER:
external: true
N8N_POSTGRES_PASSWORD:
external: true
N8N_PASSWORD:
external: true
N8N_ENCRYPTION_KEY:
external: true
N8N_S3_ACCESS_KEY:
external: true
N8N_S3_BUCKET:
external: true
N8N_S3_SECRET_KEY:
external: true
N8N_LICENSE_KEY:
external: true
REDIS_PASSWORD:
external: true
8

Hey @mpst

Applogies for the deplayed reply, but did you try checking

  • Is your n8n-worker container starting correctly?
docker service logs n8n-worker
  • Inside the container, check if Redis is reachable:
docker exec -it <worker_container> ping redis

It seems like a network issue, as the worker only need to access redis and the db. redis just posts some details for the wokers pickup, it very minimal see image

Screenshot 2025-05-27 at 23.46.48
Screenshot 2025-05-27 at 23.46.482354×1490 193 KB

After the worker receives this, it calls the db for everything it needs.

Best regards,

Samuel