N8n queue mode on docker swarm with traefik: endless execution

Questions
, ,
1

Describe the problem/error/question

The workflows that ran correctly in the previous installation are now running endlessly without showing relevant information in the workers’ logs.

Context: I switched from a VPS with n8n v0.219.1 in standalone mode (which was functioning properly) to a new installation with n8n v1.27.3 in queue mode on Docker Swarm with Traefik. I imported both workflows and credentials using the CLI and am accessing them through the UI.

The issue is that, apparently, the worker receives the job, starts executing it (or not), but fails to complete the execution and does not display any pertinent logs.

Worker Logs:

2024-02-21T09:42:31.569Z | debug | Starting n8n worker… “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:31.575Z | debug | Queue mode id: worker-08InHfq7Vu1sdWjB “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:32.571Z | debug | Lazy Loading credentials and nodes from n8n-nodes-base “{\n credentials: 352,\n nodes: 443,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”
2024-02-21T09:42:32.883Z | debug | Lazy Loading credentials and nodes from @n8n/n8n-nodes-langchain “{\n credentials: 13,\n nodes: 66,\n file: ‘LoggerProxy.js’,\n function: ‘exports.debug’\n}”
2024-02-21T09:42:33.507Z | debug | [license] initializing for deviceFingerprint 50627b8c2e69ea19b70eda4c07f57d31d20400c4a024bf186921fdca2efe59a7 “{ file: ‘LicenseManager.js’, function: ‘log’ }”
2024-02-21T09:42:33.514Z | debug | License init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.566Z | debug | Binary data service init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.567Z | debug | External hooks init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.571Z | debug | External secrets init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.572Z | debug | Initializing event bus… “{ file: ‘MessageEventBus.js’, function: ‘initialize’ }”
2024-02-21T09:42:33.574Z | debug | Initializing event writer “{ file: ‘MessageEventBus.js’, function: ‘initialize’ }”
2024-02-21T09:42:33.578Z | debug | Checking for unsent event messages “{ file: ‘MessageEventBus.js’, function: ‘initialize’ }”
2024-02-21T09:42:33.579Z | debug | Start logging into /home/node/.n8n/n8nEventLog-worker.log “{ file: ‘MessageEventBus.js’, function: ‘initialize’ }”
2024-02-21T09:42:33.579Z | debug | MessageEventBus initialized “{ file: ‘MessageEventBus.js’, function: ‘initialize’ }”
2024-02-21T09:42:33.580Z | debug | Event bus init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.586Z | debug | Opening Redis connection to listen to messages with timeout 10000 “{ file: ‘worker.js’, function: ‘initQueue’ }”
2024-02-21T09:42:33.976Z | debug | Initialising Redis client of type client(bull) connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:42:33.983Z | debug | Initialising Redis client of type subscriber(bull) connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:42:33.989Z | debug | Queue singleton ready “{ file: ‘worker.js’, function: ‘initQueue’ }”
2024-02-21T09:42:33.991Z | debug | Queue init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:33.992Z | debug | Initialising Redis client of type publisher connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:42:34.066Z | debug | Initialising Redis client of type subscriber connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:42:34.078Z | debug | Initialising Redis client of type bclient(bull) connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:42:34.083Z | debug | Subscribed Redis PubSub client to channel: n8n.commands “{ file: ‘RedisServicePubSubSubscriber.js’ }”
2024-02-21T09:42:34.083Z | debug | Orchestration init complete “{ file: ‘worker.js’, function: ‘init’ }”
2024-02-21T09:42:34.182Z | info |
n8n worker is now ready “{ file: ‘worker.js’, function: ‘run’ }”
2024-02-21T09:42:34.183Z | info | * Version: 1.27.3 “{ file: ‘worker.js’, function: ‘run’ }”
2024-02-21T09:42:34.183Z | info | * Concurrency: 10 “{ file: ‘worker.js’, function: ‘run’ }”
2024-02-21T09:42:34.184Z | info | “{ file: ‘worker.js’, function: ‘run’ }”
2024-02-21T09:51:15.093Z | info | Start job: 87 (Workflow ID: 8 | Execution: 91) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T09:51:15.099Z | debug | Initialising Redis client of type client(cache) connection with host: redis and port: 6379 “{ file: ‘RedisServiceHelper.js’, function: ‘getRedisStandardClient’ }”
2024-02-21T09:52:15.100Z | info | Start job: 88 (Workflow ID: 8 | Execution: 92) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T09:53:15.041Z | info | Start job: 89 (Workflow ID: 8 | Execution: 93) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T09:54:15.101Z | info | Start job: 90 (Workflow ID: 8 | Execution: 94) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T09:55:15.095Z | info | Start job: 91 (Workflow ID: 8 | Execution: 95) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T09:56:15.110Z | info | Start job: 92 (Workflow ID: 8 | Execution: 96) “{ file: ‘worker.js’, function: ‘runJob’ }”
2024-02-21T10:11:44.176Z | info | Start job: 93 (Workflow ID: 8 | Execution: 97) “{ file: ‘worker.js’, function: ‘runJob’ }”

Please share your workflow

Portainer Stack

version: "3.7"

services:
  n8n_editor:
    image: n8nio/n8n:latest
    command: start
    networks:
      - web
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_DATABASE=n8n_queue
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_USER=postgres
      - DB_POSTGRESDB_PASSWORD=XXXXXX
      - N8N_ENCRYPTION_KEY=XXXXXXXXXXX
      - N8N_HOST=n8n.XXXXXX.com
      - N8N_EDITOR_BASE_URL=https://n8n.XXXXXX.com/
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - N8N_LOG_LEVEL=debug
      - WEBHOOK_URL=https://webhook.XXXXXX.com/
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - QUEUE_BULL_REDIS_DB=2
      - NODE_FUNCTION_ALLOW_EXTERNAL=moment,lodash,moment-with-locales
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - GENERIC_TIMEZONE=America/Sao_Paulo
      - TZ=America/Sao_Paulo
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == manager
      resources:
        limits:
          cpus: "0.5"
          memory: 1024M
      labels:
        - traefik.enable=true
        - traefik.http.routers.n8n_editor.rule=Host(`n8n.XXXXXX.com`)
        - traefik.http.routers.n8n_editor.entrypoints=websecure
        - traefik.http.routers.n8n_editor.priority=1
        - traefik.http.routers.n8n_editor.tls.certresolver=letsencryptresolver
        - traefik.http.routers.n8n_editor.service=n8n_editor
        - traefik.http.services.n8n_editor.loadbalancer.server.port=5678
        - traefik.http.services.n8n_editor.loadbalancer.passHostHeader=1

  n8n_webhook:
    image: n8nio/n8n:latest
    command: webhook
    networks:
      - web
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_DATABASE=n8n_queue
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_USER=postgres
      - DB_POSTGRESDB_PASSWORD=XXXXXX
      - N8N_ENCRYPTION_KEY=XXXXXXXXXXX
      - N8N_HOST=n8n.XXXXXX.com
      - N8N_EDITOR_BASE_URL=https://n8n.XXXXXX.com/
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - N8N_LOG_LEVEL=debug
      - WEBHOOK_URL=https://webhook.XXXXXX.com/
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - QUEUE_BULL_REDIS_DB=2
      - NODE_FUNCTION_ALLOW_EXTERNAL=moment,lodash,moment-with-locales
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - GENERIC_TIMEZONE=America/Sao_Paulo
      - TZ=America/Sao_Paulo
    deploy:
      mode: replicated
      replicas: 2
      placement:
        constraints:
          - node.role == manager
      resources:
        limits:
          cpus: "0.5"
          memory: 1024M
      labels:
        - traefik.enable=true
        - traefik.http.routers.n8n_webhook.rule=(Host(`webhook.XXXXXX.com`))
        - traefik.http.routers.n8n_webhook.entrypoints=websecure
        - traefik.http.routers.n8n_webhook.priority=1
        - traefik.http.routers.n8n_webhook.tls.certresolver=letsencryptresolver
        - traefik.http.routers.n8n_webhook.service=n8n_webhook
        - traefik.http.services.n8n_webhook.loadbalancer.server.port=5678
        - traefik.http.services.n8n_webhook.loadbalancer.passHostHeader=1

  n8n_worker:
    image: n8nio/n8n:latest
    command: worker --concurrency=10
    networks:
      - web
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_DATABASE=n8n_queue
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_USER=postgres
      - DB_POSTGRESDB_PASSWORD=XXXXXX
      - N8N_ENCRYPTION_KEY=XXXXXXXXXXX
      - N8N_HOST=n8n.XXXXXX.com
      - N8N_EDITOR_BASE_URL=https://n8n.XXXXXX.com/
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - N8N_LOG_LEVEL=debug
      - WEBHOOK_URL=https://webhook.XXXXXX.com/
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_BULL_REDIS_PORT=6379
      - QUEUE_BULL_REDIS_DB=2
      - NODE_FUNCTION_ALLOW_EXTERNAL=moment,lodash,moment-with-locales
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - GENERIC_TIMEZONE=America/Sao_Paulo
      - TZ=America/Sao_Paulo
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == manager
      resources:
        limits:
          cpus: "0.5"
          memory: 1024M

networks:
  web:
    name: web
    external: true

Information on your n8n setup

  • n8n version: 1.27.3
  • Database: Postgres
  • n8n EXECUTIONS_PROCESS setting: main / queue
  • Running n8n via: Docker Swarm Portainer
  • Operating system: Debian 12
2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Hey @Lucas_Couto,

Welcome to the community :cake:

I would recommend starting off without the worker to make sure your n8n installation is working then toggle queue mode on and add one worker and take it from there.

4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.