503 Service Unavailable and Unexpected EOF Errors with Proxy Configuration in n8n

  • n8n version: Version: 1.71.3
  • Database (default: SQLite): default
  • n8n EXECUTIONS_PROCESS setting (default: own, main): default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): docker
  • Operating system: Ubuntu server 22

Description of the Problem:

We have configured n8n behind a Traefik reverse proxy. Additionally, we use an HTTP proxy to forward all outbound requests from n8n through an SSH tunnel via Xray. The proxy itself works correctly when tested manually using curl within the n8n container.

However, as soon as we set the environment variable HTTPS_PROXY for n8n, all external HTTPS requests, particularly those to Google OAuth API, begin to fail with 503 Service Unavailable and unexpected EOF errors.

The environment works as follows:

  • Traefik handles reverse proxying to n8n.
  • Xray acts as an HTTP proxy, which forwards traffic through an SSH tunnel using a local SOCKS5 connection.
  • Requests to Google API work manually via curl within the n8n container but fail when triggered by n8n itself.

Infrastructure Configuration

Traefik Container

traefik:
  image: "traefik"
  container_name: traefik
  restart: always
  command:
    - "--api=true"
    - "--api.insecure=true"
    - "--providers.docker=true"
    - "--providers.docker.exposedbydefault=false"
    - "--entrypoints.web.address=:80"
    - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
    - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
    - "--entrypoints.websecure.address=:443"
    - "--certificatesresolvers.mytlschallenge.acme.tlschallenge=true"
    - "--certificatesresolvers.mytlschallenge.acme.email=${SSL_EMAIL}"
    - "--certificatesresolvers.mytlschallenge.acme.storage=/letsencrypt/acme.json"
  ports:
    - "80:80"
    - "443:443"
  volumes:
    - /hdd8tb/traefik:/letsencrypt
    - /var/run/docker.sock:/var/run/docker.sock:ro
  networks:
    custom_network:
      ipv4_address: 192.168.11.2

n8n Container

n8n:
  container_name: n8n
  build:
    context: ./n8n
    dockerfile: Dockerfile
  restart: always
  depends_on:
    - sshproxy
  ports:
    - "5678:5678"
  environment:
    - HTTP_PROXY=http://sshproxy:4424
    - HTTPS_PROXY=http://sshproxy:4424
    - NO_PROXY=localhost,127.0.0.1
    - N8N_HTTP_PROXY_TIMEOUT=10000
    - N8N_HTTPS_PROXY_TIMEOUT=10000
    - NODE_TLS_REJECT_UNAUTHORIZED=0
  networks:
    custom_network:
      ipv4_address: 192.168.11.3

Xray Container

sshproxy:
  image: sshproxy
  container_name: sshproxy
  build:
    context: ./xray
    dockerfile: Dockerfile
  environment:
    - SSH_REMOTE_HOST=${SSH_HOST}
    - SSH_REMOTE_PORT=22
    - SSH_USERNAME=${SSH_USER}
    - SSH_PASSWORD=${SSH_PASSWORD}
    - LOCAL_HTTP_PORT=4424
  ports:
    - "4424:4424"
    - "5535:5535"
  networks:
    custom_network:
      ipv4_address: 192.168.11.4

Proxy Initialization Script (Xray):

#!/bin/bash
sshpass -p "${SSH_PASSWORD}" ssh -o StrictHostKeyChecking=no -f -N -D 0.0.0.0:5535 -p ${SSH_REMOTE_PORT} ${SSH_USERNAME}@${SSH_REMOTE_HOST}
cat <<EOF > /etc/xray/config.json
{
  "inbounds": [
    {
      "port": 4424,
      "protocol": "http",
      "settings": {
        "allowTransparent": true
      }
    }
  ],
  "outbounds": [
    {
      "protocol": "socks",
      "settings": {
        "servers": [
          {
            "address": "127.0.0.1",
            "port": 5535,
            "udp": true
          }
        ]
      }
    }
  ]
}
EOF
xray -config /etc/xray/config.json

Manual curl Test within the n8n Container:

docker exec -it n8n curl -v -x http://sshproxy:4424 \
     -H "Content-Type: application/x-www-form-urlencoded" \
     -d "grant_type=refresh_token&client_id=your_client_id&client_secret=your_client_secret&refresh_token=your_refresh_token" \
     https://oauth2.googleapis.com/token

Result:
Request succeeds with an appropriate HTTP response (401 Unauthorized).

Logs and Errors

Xray Logs (From sshproxy Container):

2024/12/17 14:44:36 [Info] [1363944455] proxy/http: request to Method [CONNECT] Host [oauth2.googleapis.com:443]
2024/12/17 14:44:36 [Info] [1363944455] app/dispatcher: default route for tcp:oauth2.googleapis.com:443
2024/12/17 14:44:36 [Info] [1363944455] transport/internet/tcp: dialing TCP to tcp:127.0.0.1:5535
2024/12/17 14:44:36 [Warning] [1363944455] proxy/http: failed to read response from oauth2.googleapis.com > unexpected EOF

n8n Logs:

AxiosError: Request failed with status code 503
at IncomingMessage.handleStreamEnd (/usr/local/lib/node_modules/n8n/node_modules/axios/lib/adapters/http.js:599:11)
at IncomingMessage.emit (node:events:531:35)
at endReadableNT (node:internal/streams/readable:1696:12)
Error: Bad Request
at /usr/local/lib/node_modules/n8n/node_modules/@rudderstack/rudder-sdk-node/cjs/index.js:639:49

Key Observations:

  1. Manual requests using curl within the n8n container work correctly via the proxy.
  2. When HTTPS_PROXY is set for n8n, requests to Google API fail with 503 and unexpected EOF.
  3. Xray logs show failed reads while processing responses from Google API.

Question: How can we resolve this issue where n8n requests fail when HTTPS_PROXY is enabled, despite the proxy functioning correctly with manual curl tests? Is there a known compatibility issue with Axios, or specific configurations required to make HTTPS requests via a proxy work correctly in n8n?

Any insights or guidance on this issue would be greatly appreciated!

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
1 Like
  • n8n version: Version: 1.71.3
  • Database (default: SQLite): default
  • n8n EXECUTIONS_PROCESS setting (default: own, main): default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): docker
  • Operating system: Ubuntu server 22

Hey @itconsult70

Welcome to the community :raised_hands:

This is likely to be an issue we are aware of where Axios doesn’t support the CONNECT method when using https on an http proxy.

We are looking at fixing this but at the moment the options are…

  1. Don’t use an outbound proxy
  2. Make sure your proxy listens on HTTPS so https_proxy is set to https://sshproxy:port

Unfortunately, in my case, refusing the outgoing proxy = refusing N8N.

Is there a way to route all outgoing requests from n8n through a VPN container while maintaining traffic and domain binding to the main host?