After two months of uninterrupted work, the n8n began to skip requests very much, at times, just hangs.
In debug mode, I found that vorkers stop on error:
Proxying request to axios “{ file: ‘NodeExecuteFunctions.js’, function: ‘proxyRequestToAxios’ }”
After restarting postgres, or redis, the requests are halted.
The executions themselves hang for several hours.
Even if the request is executed successfully, it continues to hang in the “running” status.
It didn’t help:
- rolling back to previous versions.
- Complete reinstallation of n8n.
my docker compose:
version: '3.9'
volumes:
db_storage:
n8n_storage:
redis_storage:
x-shared: &shared
image: n8n_custom:0.11
restart: always
environment:
- GENERIC_TIMEZONE
- N8N_PORT
- N8N_USER_MANAGEMENT_DISABLED
- N8N_ENCRYPTION_KEY
- N8N_PERSONALIZATION_ENABLED
- N8N_HIRING_BANNER_ENABLED
- N8N_LOG_LEVEL
- N8N_DISABLE_PRODUCTION_MAIN_PROCESS
- EXECUTIONS_MODE
- EXECUTIONS_PROCESS
- EXECUTIONS_DATA_SAVE_ON_SUCCESS
- EXECUTIONS_DATA_SAVE_ON_ERROR
- EXECUTIONS_DATA_PRUNE
- EXECUTIONS_DATA_MAX_AGE
- N8N_DEFAULT_BINARY_DATA_MODE
- N8N_AVAILABLE_BINARY_DATA_MODES
- NODE_OPTIONS
- DB_TYPE
- DB_POSTGRESDB_PORT
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
- DB_POSTGRESDB_HOST
- DB_LOGGING_MAX_EXECUTION_TIME
- QUEUE_BULL_REDIS_PORT
- QUEUE_BULL_REDIS_HOST
- QUEUE_HEALTH_CHECK_ACTIVE
- WEBHOOK_URL
- N8N_ENDPOINT_WEBHOOK
- N8N_ENDPOINT_WEBHOOK_TEST
- N8N_ENDPOINT_WEBHOOK_WAIT
- N8N_SKIP_WEBHOOK_DEREGISTRATION_SHUTDOWN
- N8N_EDITOR_BASE_URL
- VUE_APP_URL_BASE_API
- N8N_AUTH_EXCLUDE_ENDPOINTS
- N8N_BASIC_AUTH_ACTIVE
- N8N_BASIC_AUTH_USER
- N8N_BASIC_AUTH_PASSWORD
- N8N_ONBOARDING_FLOW_DISABLED
- N8N_HIDE_USAGE_PAGE
- N8N_DIAGNOSTICS_ENABLED
- N8N_VERSION_NOTIFICATIONS_ENABLED
- N8N_TEMPLATES_ENABLED
- NODE_FUNCTION_ALLOW_BUILTIN
- NODE_FUNCTION_ALLOW_EXTERNAL
- N8N_COMMUNITY_PACKAGES_ENABLED
depends_on:
- postgres
- redis
volumes:
- n8n_storage:/home/node/.n8n
- ${N8N_LOCAL_STORAGE}:/files
services:
postgres:
image: postgres:${POSTGRESQL_VERSION_TAG}
restart: always
hostname: postgres1
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_DB
- POSTGRES_NON_ROOT_USER
- POSTGRES_NON_ROOT_PASSWORD
volumes:
- db_storage:/var/lib/postgresql/data
- ${DB_INIT_FILE}:/docker-entrypoint-initdb.d/init-data.sh
healthcheck:
test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
interval: 5s
timeout: 5s
retries: 10
redis:
image: redis:${REDIS_VERSION_TAG}
hostname: redis1
restart: always
# command: redis-server /usr/local/etc/redis/redis.conf
volumes:
- redis_storage:/data
- /opt/redis/redis_1.conf:/usr/local/etc/redis/redis.conf
healthcheck:
test: ['CMD', 'redis-cli', 'ping']
interval: 5s
timeout: 5s
retries: 10
n8n-main1:
<<: *shared
command: ${N8N_MAIN_COMMAND}
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n-main1.rule=Host(`${N8N_UI_SUBDOMAIN}.${DOMAIN_NAME}`)"
- "traefik.http.routers.n8n-main1.entrypoints=websecure"
- "traefik.http.routers.n8n-main1.tls.certresolver=leresolver"
- "traefik.http.services.n8n-main1.loadbalancer.server.port=5678"
####### - n8n Webhook
n8n-wh:
<<: *shared
command: ${N8N_WEBHOOK_COMMAND}
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n-wh.rule=Host(`${N8N_WEBHOOK_SUBDOMAIN}.${DOMAIN_NAME}`) && Method(`POST`)"
- "traefik.http.routers.n8n-wh.entrypoints=websecure"
- "traefik.http.routers.n8n-wh.tls.certresolver=leresolver"
- "traefik.http.services.n8n-wh.loadbalancer.server.port=5678"
n8n-worker:
<<: *shared
command: ${N8N_WORKER_COMMAND}
scale: 4
networks:
default:
name: traefik_portainer_default
and .env file:
DOMAIN_NAME=example.com
N8N_UI_SUBDOMAIN=n8n
N8N_WEBHOOK_SUBDOMAIN=api
WEBHOOK_URL=https://api.example.com/
N8N_EDITOR_BASE_URL=https://n8n.example.com/
VUE_APP_URL_BASE_API=https://n8n.example.com
DB_INIT_FILE=/opt/n8n/init-data.sh
N8N_LOCAL_STORAGE=/opt/n8n/local-files
TRAEFIK_VERSION_TAG=latest
POSTGRESQL_VERSION_TAG=latest
REDIS_VERSION_TAG=alpine
N8N_VERSION_TAG=latest
POSTGRES_USER=n8n
POSTGRES_PASSWORD=qweqweqwe
POSTGRES_DB=n8n
POSTGRES_NON_ROOT_USER=n8n
POSTGRES_NON_ROOT_PASSWORD=asdasdasd
GENERIC_TIMEZONE=Europe/Moscow
N8N_MAIN_COMMAND=/bin/sh -c "sleep 5; n8n start"
N8N_WORKER_COMMAND=/bin/sh -c "sleep 5; n8n worker --concurrency=5"
N8N_WEBHOOK_COMMAND=/bin/sh -c "sleep 5; n8n webhook"
N8N_PORT=5678
N8N_USER_MANAGEMENT_DISABLED=false
N8N_DIAGNOSTICS_ENABLED=false
N8N_PERSONALIZATION_ENABLED=false
N8N_HIRING_BANNER_ENABLED=false
N8N_LOG_LEVEL=error
N8N_DISABLE_PRODUCTION_MAIN_PROCESS=false
EXECUTIONS_MODE=queue
EXECUTIONS_PROCESS=main
EXECUTIONS_DATA_SAVE_ON_SUCCESS=none
EXECUTIONS_DATA_SAVE_ON_ERROR=all
EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=14
N8N_DEFAULT_BINARY_DATA_MODE=filesystem
N8N_AVAILABLE_BINARY_DATA_MODES=filesystem
NODE_OPTIONS="--max-old-space-size=4096"
DB_TYPE=postgresdb
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_HOST=postgres1
DB_LOGGING_MAX_EXECUTION_TIME=0
QUEUE_BULL_REDIS_PORT=6379
QUEUE_BULL_REDIS_HOST=redis1
QUEUE_HEALTH_CHECK_ACTIVE=true
N8N_SKIP_WEBHOOK_DEREGISTRATION_SHUTDOWN=true
N8N_AUTH_EXCLUDE_ENDPOINTS=true
N8N_ONBOARDING_FLOW_DISABLED=true
N8N_HIDE_USAGE_PAGE=false
N8N_DIAGNOSTICS_ENABLED=false
N8N_VERSION_NOTIFICATIONS_ENABLED=false
N8N_TEMPLATES_ENABLED=false
N8N_ENDPOINT_WEBHOOK=main
N8N_ENDPOINT_WEBHOOK_TEST=test
N8N_ENDPOINT_WEBHOOK_WAIT=wait
N8N_LOG_LEVEL=debug
N8N_ENCRYPTION_KEY=qqqwwweeerrrrttttyyy
NODE_FUNCTION_ALLOW_BUILTIN=*
NODE_FUNCTION_ALLOW_EXTERNAL=*
N8N_COMMUNITY_PACKAGES_ENABLED=true
Can you tell me how else I can debug correctly to find the error?
Information on my n8n setup
- **n8n version: 0.222.2 **
- Database: postgres
- n8n queue mode
- Running n8n via Docker
- Operating system: Ubuntu