N8n Docker Compose Setup on Portainer with Traefik - Gateway timeout error

prononext · August 18, 2022, 8:57pm

Dear n8n Community,

I am setting up n8n exactly as per Docker Compose - n8n Documentation in the official docs on a Portainer stack.

There is a really strange behavior as it is showing me a “gateway timeout” on the frontend.

All lets-encrypt certificates are generated without any problems but the request is ending in a gateway timeout.

When I restart the service, it randomly lets me into the GUI but when restarting container, stack, service or server again it is showing the gateway timeout again.

I have enabled debug logging for traefik and n8n service but there are logs on n8n it just says that the GUI is available and it is setup correctly.

In traefik the only log that is there for the gateway error seems to be:
time="2022-08-18T20:47:08Z" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 10.0.0.46:5678: i/o timeout"

I suppose as there is no network set in the docker compose, it does not know where to send the request correctly, but I do not know how to solve it.

I have replicated that procedure on new servers and setups about 10 times with always the same results.

It would be really nice to get a solution for this.

Jon · August 19, 2022, 4:21am

Hey @prononext,

If you check the log for n8n in portainer is it running? Is the IP in the log you have correct as well?

tight-crow-1218 · August 19, 2022, 6:08am

Hi @prononext , Where did you put traefik’s docker-compose file? Is it same stack with N8N or it is a different stack in portainer?
And which host is 10.0.0.46? you portainer host ?

prononext · August 19, 2022, 1:03pm

Yes traefik is inside the docker-compose file just like shown in the official docs, so it is on the same stack inside portainer. The host 10.0.0.46 is some docker ip, that seems to change on every restart.

The n8n container is running, here the debug log:

2022-08-19T12:58:04.463Z | info     | Initializing n8n process "{ file: 'start.js' }"
2022-08-19T12:58:06.081Z | debug    | No codex available for: N8nTrainingCustomerDatastore.node.js "{ file: 'LoadNodesAndCredentials.js', function: 'addCodex' }"
2022-08-19T12:58:06.082Z | debug    | No codex available for: N8nTrainingCustomerMessenger.node.js "{ file: 'LoadNodesAndCredentials.js', function: 'addCodex' }"
2022-08-19T12:58:07.263Z | debug    | Wait tracker querying database for waiting executions "{ file: 'WaitTracker.js', function: 'getwaitingExecutions' }"
n8n ready on 0.0.0.0, port 5678
Version: 0.191.1
2022-08-19T12:58:07.738Z [Rudder] debug: in flush
2022-08-19T12:58:07.738Z [Rudder] debug: batch size is 1
2022-08-19T12:58:07.740Z [Rudder] debug: no existing flush timer, creating new one
Editor is now accessible via: https://n8n.example.com

tight-crow-1218 · August 20, 2022, 9:33am

Traefik had ping and nc command builtin, you need make sure that traefik can connect to n8n container’s expose port. like this

/ # ping n8n
PING n8n (172.21.0.7): 56 data bytes
64 bytes from 172.21.0.7: seq=0 ttl=64 time=0.086 ms
64 bytes from 172.21.0.7: seq=1 ttl=64 time=0.103 ms
^C
--- n8n ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.086/0.094/0.103 ms
/ # nc -znv 172.21.0.7 5678
172.21.0.7 (172.21.0.7:5678) open

prononext · August 25, 2022, 8:15pm

Yes treafik and n8n where on the same network and reachable to eachother. I got a real headache from Traefik now and switched to Caddy, which was working on the first try.

tight-crow-1218 · August 26, 2022, 1:44am

Maybe you can paste your docker-compose file , I have 4 stack on different machine , they all working fine

prononext · August 26, 2022, 2:29am

Of course here it is:

version: '3.9'

services:
  caddy:
    image: caddy:${CADDY_VERSION_TAG}
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ${CADDY_DATA_FOLDER}/Caddyfile:/etc/caddy/Caddyfile
      - ${CADDY_DATA_FOLDER}/data:/data
      - ${CADDY_DATA_FOLDER}/config:/config
    links:
      - n8n

  n8n:
    container_name: n8n
    image: n8nio/n8n:${N8N_VERSION_TAG}
    restart: always
    ports:
      - "5678"
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER=${N8N_BASIC_AUTH_USER}
      - N8N_BASIC_AUTH_PASSWORD=${N8N_BASIC_AUTH_PASSWORD}
      - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - WEBHOOK_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/
      - GENERIC_TIMEZONE=${GENERIC_TIMEZONE}
      - N8N_EMAIL_MODE=${N8N_EMAIL_MODE}
      - N8N_SMTP_HOST=${N8N_SMTP_HOST}
      - N8N_SMTP_PORT=${N8N_SMTP_PORT}
      - N8N_SMTP_USER=${N8N_SMTP_USER}
      - N8N_SMTP_PASS=${N8N_SMTP_PASS}
      - N8N_SMTP_SENDER=${N8N_SMTP_SENDER}
      - N8N_SMTP_SSL=${N8N_SMTP_SSL}
      - N8N_PERSONALIZATION_ENABLED=${N8N_PERSONALIZATION_ENABLED}
      - N8N_HIRING_BANNER_ENABLED=${N8N_HIRING_BANNER_ENABLED}
      - N8N_DEFAULT_BINARY_DATA_MODE=${N8N_DEFAULT_BINARY_DATA_MODE}
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
      - N8N_LOG_LEVEL=${N8N_LOG_LEVEL}
    volumes:
      - ${N8N_DATA_FOLDER}/:/home/node/.n8n
      - /local-files:/files

tight-crow-1218 · August 26, 2022, 3:00am

Well, I mean docker compose file with treafik . I think new version of docker-compose files don’t need links , one of compose I use is like this

version: "3.3"
services:
  traefik:
    image: "traefik:v2.8"
    restart: unless-stopped
    container_name: "traefik"
    command:
      #- "--log.level=DEBUG"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.resolvers=223.5.5.5:53,223.6.6.6:53"
      - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=alidns"
      - "[email protected]"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    environment:
      - "ALICLOUD_ACCESS_KEY"
      - "ALICLOUD_SECRET_KEY"
      - "ALICLOUD_REGION_ID=cn-hangzhou"
    volumes:
      - "${DATA_FOLDER}/letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  n8n:
    restart: unless-stopped
    image: n8nio/n8n
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.n8n.rule=Host(`${SUBDOMAIN}.${DOMAIN_NAME}`)"
      - "traefik.http.routers.n8n.entrypoints=websecure"
      - "traefik.http.routers.n8n.tls.certresolver=myresolver"
      - "traefik.http.middlewares.n8n.headers.SSLRedirect=true"
      - traefik.http.middlewares.n8n.headers.STSSeconds=315360000
      - traefik.http.middlewares.n8n.headers.browserXSSFilter=true
      - traefik.http.middlewares.n8n.headers.contentTypeNosniff=true
      - traefik.http.middlewares.n8n.headers.forceSTSHeader=true
      - traefik.http.middlewares.n8n.headers.SSLHost=${DOMAIN_NAME}
      - traefik.http.middlewares.n8n.headers.STSIncludeSubdomains=true
      - traefik.http.middlewares.n8n.headers.STSPreload=true
    environment:
      - N8N_BASIC_AUTH_ACTIVE=true
      - N8N_BASIC_AUTH_USER
      - N8N_BASIC_AUTH_PASSWORD
      - N8N_HOST=${SUBDOMAIN}.${DOMAIN_NAME}
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_ENV=production
      - WEBHOOK_TUNNEL_URL=https://${SUBDOMAIN}.${DOMAIN_NAME}/
      - GENERIC_TIMEZONE=${GENERIC_TIMEZONE}
    volumes:
      - ${DATA_FOLDER}/.n8n:/home/node/.n8n

prononext · August 26, 2022, 3:22am

Traefik has never worked for me with n8n and was always very buggy on restarting stacks, server and container so I went with Caddy.