How to add custom python package to n8n task runner

[MehranZ]

Describe the problem/error/question

I followed the instructions to add a custom package to the python but it did not work.

Here’s how I’ve tried:

$ git clone https://github.com/n8n-io/n8n.git
$ cd ./n8n
$ git checkout [email protected]
$ echo "pypdf" >>  docker/images/runners/extras.txt
$ nano docker/images/runners/n8n-task-runners.json

With editing the file, I added “N8N_RUNNERS_EXTERNAL_ALLOW”: “pypdf”. Then I built the docker file:

$ docker buildx build \
  -f docker/images/runners/Dockerfile \
  -t n8nio/runners:custom \
  .

This resulted in an error:

 => ERROR [javascript-runner-builder  2/10] COPY ./dist/task-runner-javascript /app/task-runner-javascript 0.0s
...
ERROR: failed to build: failed to solve: failed to compute cache key: failed to calculate checksum of ref 541ed8ae-efc1-4cc9-818e-37be482f2116::j4d4wtpi1bezx7g44056xw077: "/dist/task-runner-javascript": not found

This was because there is not dist folder and thus no ./dist/task-runner-javascript/package.json. I fixed this issue by adding a dummy file.

$ mkdir -p ./dist/task-runner-javascript/
$ echo '{ "name": "dummy_package", "version": "1.0.0" }' >>  ./dist/task-runner-javascript/package.json

[MehranZ]

Post continued…

This was because there is not dist folder and thus no ./dist/task-runner-javascript/package.json. I fixed this issue by adding a dummy file.

$ mkdir -p ./dist/task-runner-javascript/
$ echo '{ "name": "dummy_package", "version": "1.0.0" }' >>  ./dist/task-runner-javascript/package.json

$ docker buildx build \                                                                           
  -f docker/images/runners/Dockerfile \
  -t n8nio/runners:custom \
  .

This time, the docker image was created. So, I spun up an n8n instance using a docker compose:

services:
  n8n:
    image: n8nio/n8n:1.111.0
    restart: always
    ports:
      - "5678:5678"
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=n8n
      - N8N_BASIC_AUTH_PASSWORD=your_secure_password
      - N8N_ENCRYPTION_KEY=your_encryption_key
      - N8N_HOST=localhost
      - N8N_PROTOCOL=http
      - N8N_RUNNERS_ENABLED=true
      - N8N_RUNNERS_MODE=external
      - N8N_RUNNERS_BROKER_LISTEN_ADDRESS=0.0.0.0
      - N8N_RUNNERS_AUTH_TOKEN=your-secret-here
    volumes:
      - ./n8n_data:/home/node/.n8n
      # - ./init:/init

  task-runners:
    image: n8nio/runners:custom
    container_name: n8n-runners
    environment:
      - N8N_RUNNERS_TASK_BROKER_URI=http://n8n:5679
      - N8N_RUNNERS_AUTH_TOKEN=your-secret-here
      - N8N_RUNNERS_EXTERNAL_ALLOW=*
      - N8N_RUNNERS_LAUNCHER_LOG_LEVEL=debug
      # etc.
    depends_on:
      - n8n

But when I create a workflow and add a “code” node, and try to import the pypdf package:

from pypdf import PdfReader

I get the error message

ModuleNotFoundError: No module named 'pypdf'

Even if I go into the container’s terminal and run python, the package cannot be imported there either. I would like to also add that I can confirm the pypdf was added to the extras.txt file and it did end up in the task runner image. I confirmed this by going into the container’s terminal and:

$ cat /opt/runners/task-runner-python/extras.txt 
# Runtime-only extra Requirements File for installing dependencies in the Python task runner image.
# Installed at Docker image build time. Allow usage in the Code node
# via 'N8N_RUNNERS_EXTERNAL_ALLOW' env variable on n8n-task-runners.json.

# numpy==2.3.2

pypdf

But the package is no where to find. Has anyone managed to add a python package to the task runner?

What is the error message (if any)?

ModuleNotFoundError: No module named 'pypdf'

Information on your n8n setup

• n8n version: [email protected]
• Database (default: SQLite): SQLite
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
• Operating system: Linux

[MehranZ]

The documentation is complete. And also, I made a mistake.

First the mistake, make sure you task runner docker image and the n8n main docker image are of the same version.

Second, before building the task runner image, run these commands:

$ pnpm i

$ pnpm run build:n8n

Then, build the docker image for the task runner like this (the same as above):

$ docker buildx build \                                                                           
  -f docker/images/runners/Dockerfile \
  -t n8nio/runners:custom \
  .

No need to create a dummy package.json. It will be created by the second command.