I added N8N_RUNNERS_ENABLED=true back
- I switched the Node.js memory to 2048
Still had the same issue
Then removed the Node.js memory limit line from the .yml file (keeping N8N_RUNNERS_ENABLED=true)
Same problem, it crashes as soon as it starts, unless I manage to quickly turn off those 3 active workflows. [which in most cases only possible when I have NODE_OPTIONS=--max-old-space-size=512]
These are the logs of the n8n instance firing up:
root@[HOSTNAME]:/opt/n8n-docker-caddy# docker compose up
WARN[0000] /opt/n8n-docker-caddy/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Running 3/3
✔ Network n8n-docker-caddy_default Created 0.1s
✔ Container n8n-docker-caddy-n8n-1 Created 0.1s
✔ Container n8n-docker-caddy-caddy-1 Created 0.1s
Attaching to caddy-1, n8n-1
caddy-1 | {"level":"info","ts":1749413006.730384,"msg":"maxprocs: Leaving GOMAXPROCS=1: CPU quota undefined"}
caddy-1 | {"level":"info","ts":1749413006.7313514,"msg":"GOMEMLIMIT is updated","package":"github.com/KimMachineGun/automemlimit/memlimit","GOMEMLIMIT":1853374464,"previous":9223372036854775807}
caddy-1 | {"level":"info","ts":1749413006.7322245,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-1 | {"level":"info","ts":1749413006.7346182,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-1 | {"level":"warn","ts":1749413006.7348614,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy-1 | {"level":"info","ts":1749413006.7373714,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
caddy-1 | {"level":"info","ts":1749413006.738037,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
caddy-1 | {"level":"info","ts":1749413006.738272,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
caddy-1 | {"level":"info","ts":1749413006.739473,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
caddy-1 | {"level":"info","ts":1749413006.7402198,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
caddy-1 | {"level":"info","ts":1749413006.74172,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-1 | {"level":"warn","ts":1749413006.7419899,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-1 | {"level":"warn","ts":1749413006.742157,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-1 | {"level":"info","ts":1749413006.7423573,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
caddy-1 | {"level":"info","ts":1749413006.742572,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["[DOMAIN]"]}
caddy-1 | {"level":"warn","ts":1749413006.74493,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [[DOMAIN]]: no OCSP server specified in certificate","identifiers":["[DOMAIN]"]}
caddy-1 | {"level":"info","ts":1749413006.7459815,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-1 | {"level":"info","ts":1749413006.7461538,"msg":"serving initial configuration"}
caddy-1 | {"level":"info","ts":1749413006.749712,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"[INSTANCE_ID]","try_again":1749499406.7497106,"try_again_in":86399.99999963}
caddy-1 | {"level":"info","ts":1749413006.7497888,"logger":"tls","msg":"finished cleaning storage units"}
caddy-1 | {"level":"info","ts":1749413006.7502124,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"[MEMORY_ADDRESS]"}
n8n-1 | User settings loaded from: /home/node/.n8n/config
n8n-1 | Last session crashed
n8n-1 | Initializing n8n process
n8n-1 | n8n ready on ::, port 5678
n8n-1 | n8n Task Broker ready on 127.0.0.1, port 5679
n8n-1 | [license SDK] Skipping renewal on init because renewal is not due yet or cert is not initialized
n8n-1 | Registered runner "JS Task Runner" ([RUNNER_ID])
n8n-1 | Version: 1.97.1
n8n-1 | ================================
n8n-1 | Start Active Workflows:
n8n-1 | ================================
n8n-1 |
n8n-1 | <--- Last few GCs --->
n8n-1 |
n8n-1 | [7:[MEMORY_ADDRESS]] 114363 ms: Mark-Compact 955.7 (1002.2) -> 943.7 (1002.2) MB, 634.89 / 0.00 ms (average mu = 0.282, current mu = 0.255) allocation failure; scavenge might not succeed
n8n-1 | [7:[MEMORY_ADDRESS]] 115255 ms: Mark-Compact 957.5 (1004.0) -> 945.4 (1004.0) MB, 688.50 / 0.00 ms (average mu = 0.255, current mu = 0.228) allocation failure; scavenge might not succeed
n8n-1 |
n8n-1 |
n8n-1 | <--- JS stacktrace --->
n8n-1 |
n8n-1 | FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
n8n-1 | ----- Native stack trace -----
n8n-1 |
caddy-1 | {"level":"error","ts":1749413122.9765534,"logger":"http.log.error","msg":"read tcp [INTERNAL_IP]:[PORT]->[INTERNAL_IP]:[PORT]: read: connection reset by peer","request":{"remote_ip":"[CLIENT_IP]","remote_port":"[PORT]","client_ip":"[CLIENT_IP]","proto":"HTTP/2.0","method":"GET","host":"[DOMAIN]","uri":"/rest/workflows?includeScopes=true&includeFolders=true&filter=%7B%22projectId%22%3A%22[PROJECT_ID]%22%2C%22isArchived%22%3Afalse%7D&skip=0&take=1","headers":{"Sec-Fetch-Site":["same-origin"],"Accept-Language":["en"],"Sec-Fetch-Mode":["cors"],"Push-Ref":["[PUSH_REF]"],"Cookie":["[REDACTED]"],"Sec-Ch-Ua-Platform":["\"macOS\""],"Accept":["application/json, text/plain, */*"],"If-None-Match":["W/\"[ETAG]\""],"Priority":["u=1, i"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"],"Sec-Ch-Ua":["\"Not.A/Brand\";v=\"99\", \"Chromium\";v=\"136\""],"Browser-Id":["[BROWSER_ID]"],"Dnt":["1"],"Accept-Encoding":["gzip, deflate, br, zstd"],"Sec-Fetch-Dest":["empty"],"Sec-Ch-Ua-Mobile":["?0"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"[DOMAIN]"}},"duration":83.007847709,"status":502,"err_id":"[ERROR_ID]","err_trace":"reverseproxy.statusError (reverseproxy.go:1390)"}
[... additional error logs with similar pattern of redacted sensitive information ...]
n8n-1 exited with code 0
[... more connection refused errors with redacted sensitive information ...]
n8n-1 | User settings loaded from: /home/node/.n8n/config
n8n-1 | Last session crashed
n8n-1 | Initializing n8n process
n8n-1 | n8n ready on ::, port 5678
n8n-1 | n8n Task Broker ready on 127.0.0.1, port 5679
n8n-1 | [license SDK] Skipping renewal on init because renewal is not due yet or cert is not initialized
n8n-1 | Registered runner "JS Task Runner" ([RUNNER_ID])
n8n-1 | Version: 1.97.1
n8n-1 | ================================
n8n-1 | Start Active Workflows:
n8n-1 | ================================
More info:
Stats with all workflows off for 1.97.1 (2 GB Memory / 1 AMD vCPU / 25 GB Disk)
lmk if I can provide anything else to help identify the problem
