Install Community Nodes in Queue Mode on n8n 1.X.X

[automatio]

Describe the problem/error/question

Currently we cannot install Community nodes when running n8n in queue mode.

It seems that the changes brought by n8n 1.X.X to the custom nodes folder might have solved this issue.

My setup

I’m running n8n 1.2.0 in queue mode and use custom nodes without any issue. The only problem is that the installation of nodes is currently blocked through the UI when running in queue mode. (I need to run in normal mode, install nodes, then back to queue mode).

My Docker Compose

version: '3.8'

volumes:
  db_storage_temp:
  n8n_storage:
  redis_storage:

x-shared: &shared
    image: n8nio/n8n:latest
    restart: always
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
      - DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
      - N8N_EMAIL_MODE=smtp
      - N8N_SMTP_HOST=${N8N_SMTP_HOST}
      - N8N_SMTP_USER=${N8N_SMTP_USER}
      - N8N_SMTP_PASS=${N8N_SMTP_PASS}
      - N8N_SMTP_SENDER= "Acme [email protected]"
      - EXECUTIONS_MODE=queue
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_HEALTH_CHECK_ACTIVE=true
      - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=dev
      - GENERIC_TIMEZONE=${GENERIC_TIMEZONE}
    links:
      - postgres
    networks:
      - app-network
    volumes:
      - ./data:/files
      - n8n_storage:/home/node/.n8n
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy

services:

  postgres:
    image: postgres:11
    restart: always
    ports:
      - "5432:5432"
    networks:
      - app-network
    environment:
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - POSTGRES_DB
      - POSTGRES_NON_ROOT_USER
      - POSTGRES_NON_ROOT_PASSWORD
    volumes:
      - db_storage_temp:/var/lib/postgresql/data
      - ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
    healthcheck:
      test:
        [
          'CMD-SHELL',
          'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}'
        ]
      interval: 5s
      timeout: 5s
      retries: 10

  redis:
    image: redis:6-alpine
    restart: always
    networks:
      - app-network
    volumes:
      - redis_storage:/data
    healthcheck:
      test: [ 'CMD', 'redis-cli', 'ping' ]
      interval: 5s
      timeout: 5s
      retries: 10

  n8n:
    <<: *shared
    ports:
      - 5678:5678

  n8n-worker:
    <<: *shared
    command: worker
    depends_on:
      - n8n

networks:
  app-network:
    driver: bridge

Potential solution

Let user install community nodes from the UI when running n8n in queue mode; or at least make it possible to enable this by exception using a new environment variable.

[Jon]

Hey @automatio,

Welcome to the community

You could also manually install the nodes outside of n8n into the custom or ~/.n8n/nodes directory for each worker instance. This has been something other queue mode users have been doing for a while and it is also how private nodes were loaded before we added the UI feature, It would of course be great to enable this in the UI and have a process for the workers to also download the node so they can use them.

[automatio]

Hi Jon, my point is that it works easily in a queue mode setup (see my current stack), so the “locking out the installation of custom nodes through the UI in queue mode” does not make sense anymore (or at least should be something we can override)

[Jon]

Hey @automatio,

You are not wrong, But we have multple things to take into consideration as not everyone will be using the same stack so if we were to do it the best option would be to make sure the workers know they need to download a node.

[svenjanssen]

Try running the main (n8n) in regular mode (which unlocks community nodes install feature). Add webhook nodes (just like worker but command “webhook”), both in queue mode. Make sure that routes / and /webhook-test go to main (n8n) and /webhook goes to webhook node. Then you can install nodes and configure workflows using n8n, while the “queue work” is being done by worker / webhook (which are in queue mode).

After installing a new node make sure to restart worker and webhook, too.