Hello! I have this weird error: “Cannot read properties of undefined (reading ‘node’)” starting directly at the webhook trigger, but the problem is that it’s not always the case, around 0.7% executions from my estimation. I analyzed logs from the tool which triggers this webhook, no problem with it.
As you can see in screens, the webhook is well triggered, but doesn’t seem to receive data according to the interface, while it does since a slack node is able to get an object from it.
{
"message": "Cannot read properties of undefined (reading 'node')",
"stack": "Error: Cannot read properties of undefined (reading 'node')\n at Queue.onFailed (/usr/local/lib/node_modules/n8n/node_modules/bull/lib/job.js:516:18)\n at Queue.emit (node:events:531:35)\n at Queue.emit (node:domain:488:12)\n at Object.module.exports.emitSafe (/usr/local/lib/node_modules/n8n/node_modules/bull/lib/utils.js:50:20)\n at EventEmitter.messageHandler (/usr/local/lib/node_modules/n8n/node_modules/bull/lib/queue.js:476:15)\n at EventEmitter.emit (node:events:519:28)\n at EventEmitter.emit (node:domain:488:12)\n at DataHandler.handleSubscriberReply (/usr/local/lib/node_modules/n8n/node_modules/ioredis/built/DataHandler.js:80:32)\n at DataHandler.returnReply (/usr/local/lib/node_modules/n8n/node_modules/ioredis/built/DataHandler.js:47:18)\n at JavascriptRedisParser.returnReply (/usr/local/lib/node_modules/n8n/node_modules/ioredis/built/DataHandler.js:21:22)"
}
Thank you for your help!
Information on your n8n setup
n8n version: 1.46.0
Database (default: SQLite): using RDS Postgres and EFS for storing n8n data
It looks like you are using queue mode, Are you running with multiple webhook hookers and have you checked your workers to make sure they are on the same version of n8n and that nothing odd is showing in their logs?
Without seeing the workflow JSON I can’t see what node versions you are using but it could be that the main instance is on a newer version of n8n that has a newer webhook version and the worker doesn’t have it available.
I’m still facing this issue, even though we regularly update our n8n version. What’s troubling is that it occurs intermittently for some executions, while most other executions work just fine.
Contrary to what I mentioned earlier, I don’t have the webhook content for some of the error executions. There’s simply no data available, making it difficult to debug.
@theo We have not tracked down the root cause yet, but we believe it might by caused by this old safeguard. Can you please set QUEUE_RECOVERY_INTERVAL to 0 to disable it and let us know if the issue persists?
Over the week we ruled out QUEUE_RECOVERY_INTERVAL as the root cause of this issue. We are monitoring our internal instance for it, but since it’s rare and intermittent, we will have more information once it happens again.
If you agree, I can create a custom Docker image based on your version 1.46.0 with an additional check for your instance to report this issue to our Sentry, so we can gather more information about it faster.
You can find the custom image here and you can review the debug change here.
Cannot read properties of undefined (reading 'node') comes from this line. This means the worker is receiving from the queue a job with a malformed execution, where nodeExecutionStack (i.e. the nodes to execute) is unexpectedly an empty array.
The debug change detects this malformed execution in the main instance, i.e. before the job is even enqueued, and throws Execution to enqueue has empty node execution stack and reports it to our Sentry. You’ll now see this error when browsing these affected executions. The stack trace and other details in Sentry should point us to where the malformed execution is coming from.
I’ve setup a temporary setup on our error tracker to track this.
Can you please set the env variable N8N_SENTRY_DSN to https://[email protected]/4504016528408576, restart the instance, and then try to reproduce the issue
When the issue happens again, we should hopefully get more of the stack-trace, that we can then use to get this fixed.
And we never know: I have this other post with strange webhooks errors, maybe you can trace these errors as well so we can understand a bit more what’s happening?
I just checked sentry, and did not see any errors so far. Did you see the error again after setting the env variable and after restarting the instance?
For “Cannot read properties of undefined (reading ‘node’)”: yes we had one at 12:46:13PM (Paris time), executionID = 7050410
Can you please try setting QUEUE_WORKER_MAX_STALLED_COUNT to 0 and restarting the instance to see if this error happens again? Scaling mode relies on Bull to move stalled jobs back to the queue. Setting the max count to 0 can help us check if Bull’s stalling mechanism is causing this.
If the original error “Cannot read properties of undefined (reading ‘node’)” persists even with QUEUE_WORKER_MAX_STALLED_COUNT=0 then feel free to revert this count to the original setting.
Yesterday we received your multiple job stalled more than maxStalledCount so Sentry is configured correctly, but we have no events yet for “Cannot read properties of undefined (reading ‘node’)” I’ll reach back once we have more information.
