N8n webhook empty

Questions
1

Describe the issue/error/question

I had a server changeover. Along the way I also had updates from n8n. Unfortunately, I can not analyze the reason.
But I have the following problem.
I have a workflow with a webhook. This webhook is called by another program. The other program works without errors.
But n8n starts the workflow but then shows that it failed.
When then look why, then the execution is completely empty and has no data.
grafik

grafik
grafik944Ă—427 6.04 KB

grafik
grafik567Ă—856 21.3 KB

My log file also looks very strange. The configuration is read in several times as if n8n had crashed but then not really.
I mean “Initializing n8n process” is logged only once.
Just like the start of the existing workflows.
But the import is always logged again afterwards.
If I then also do the manual execution again, something quite funny also comes up:
grafik
As soon as the workflow should theoretically start because data is coming in, this error occurs.
Is it perhaps also because it generally does not work?
And also why the config is read several times?

Share the output returned by the last node

Information on your n8n setup

  • n8n version: 0.212.1
  • **Database you’re using (default: SQLite):postgres **
  • Running n8n with the execution process [own(default), main]: main
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: npm
2

Sometimes it is executed sometimes not. Is there any way to disable the cache?

It’s really an imenses problem and so n8n can’t be used currently.

3

Hey @lublak,

That is an odd one, Can you share how you have n8n set up and the config you are using?

4

@Jon So when I have problems then they are always odd :stuck_out_tongue:
I use pm2 as a windows service which starts n8n.
The service is created via winsw.
The config:

N8N_CONFIG_FILES : 'D:/n8n/config.json',
WEBHOOK_TUNNEL_URL: 'http://n8n.intranet.local/',
N8N_USER_FOLDER: 'D:/n8n',

{
	"protocol": "http",
	"host": "localhost",
	"listen_address": "0.0.0.0",
	"port": 5678,
	"executions": {
		"process": "own",
		"saveDataOnSuccess": "all",
		"saveDataOnError": "all",
		"saveDataManualExecutions": true,
		"pruneData": true,
		"pruneDataMaxAge": 336,
		"pruneDataTimeout": 3600
	},
	"generic": {
		"timezone": "Europe/Berlin"
	},
	"database": {
		"type": "postgresdb",
		"postgresdb": {
			"database": "n8n",
			"host": "localhost",
			"port": 5432,
			"user": "n8n",
			"password": "",
			"schema": "n8n"
		}
	},
	"userManagement": {
		"emails": {
			"mode": "smtp",
			"smtp": {
				"host": "ip",
				"port": 25,
				"secure": false,
				"auth": {
					"user": "smtp@domain",
					"pass": ""
				},
				"sender": "smtp@domain"
			}
		}
	}
}
5

So one thing I have found on my machine while testing another issue is if you use a config file it seems to log the file on what looks to be every workflow execution, I am not sure why we do this maybe it is part of the database query process.

Looking at your console output it looks like you have more than just that set, There is also the warning about disabling node_tls_reject_unauthorized.

Does the one service always cause these empty log events? Does it also happen if you were to call the webhook with curl or whatever the powershell version is :slight_smile:

6

Oh this is then but new that the is logged with each execution.
But there are worse things.

Yes, node_tls_reject_unauthorized is set via the env variable because we used to have issues with certain SSL protocols. I need to double check if this is even needed anymore.

“have more than just that set”? What do you mean by that?

I tried now with curl. And it also has this empty windows.

7

Hey @lublak,

More than that set was referring to the env option for node_tls_reject so I was curious as to what else could be set.

What CURL command did you run? I am just wondering if some of this is caused by the way n8n is running as well. If you just run it from npm does that work? I would maybe do npm then pm2 and start moving through the options to see if anything specific does something funky.

8

Ah for nodejs? node_tls_reject and NODE_ENV=production

I used

curl -X POST http://n8n.intranet.local/webhook-test/24bb8606-1260-4842-9729-94213536846b
   -H 'Content-Type: application/json'
   -d 'big json!' (throw this EBUSY Error)

curl -X POST http://n8n.intranet.local/webhook/24bb8606-1260-4842-9729-94213536846b
   -H 'Content-Type: application/json'
   -d 'big json!'

but now i use Insomnia because its easier to test :slight_smile:
I now stopped pm2 and just started n8n in a console. Also i created a clean install of n8n with a different location (now with a sqlite db, because i was to lazy to setup another postgres database :stuck_out_tongue:).
Aaaaand! It doesn’t work. Here it is also empty

I found a eventlog:

{"__type":"$$EventMessageWorkflow","id":"7df56903-962c-47c9-8a77-c5ac9e3955d6","ts":"2023-01-27T15:14:28.515+01:00","eventName":"n8n.workflow.started","message":"n8n.workflow.started","payload":{"executionId":"4","userId":"4f10ffaf-1caf-40e9-9a88-8b692f368904","workflowId":"1","isManual":false,"workflowName":"My workflow"}}
{"__type":"$$EventMessageConfirm","confirm":"7df56903-962c-47c9-8a77-c5ac9e3955d6","ts":"2023-01-27T15:14:28.515+01:00","source":{"id":"0","name":"eventBus"}}
{"__type":"$$EventMessageWorkflow","id":"309ed921-db1f-41f7-a2b6-1e932e265bd6","ts":"2023-01-27T15:14:30.957+01:00","eventName":"n8n.workflow.failed","message":"n8n.workflow.failed","payload":{"executionId":"4","success":false,"userId":"4f10ffaf-1caf-40e9-9a88-8b692f368904","workflowId":"1","errorMessage":"EBUSY: resource busy or locked, copyfile 'C:\\ProgramData\\npm\\npm\\node_modules\\n8n\\node_modules\\n8n-nodes-base\\dist\\nodes\\Google\\Sheet\\googleSheets.svg' -> 'C:\\Users\\testadmin\\.cache\\n8n\\public\\icons\\nodes\\n8n-nodes-base.googleSheets.svg'","isManual":false,"workflowName":"My workflow"}}
{"__type":"$$EventMessageConfirm","confirm":"309ed921-db1f-41f7-a2b6-1e932e265bd6","ts":"2023-01-27T15:14:30.958+01:00","source":{"id":"0","name":"eventBus"}}

So here is the same issue with the cache

9

@Jon i executed it now multiple times and some times it works fine:

grafik
grafik305Ă—543 17.7 KB

So i think n8n loads some file and also tries to edit the file at the same time?
And there i currently now antivir enabled on this test system.

10

I now installed n8[email protected] which is before this commit and everything works fine :slight_smile: so i think its a issue with n8n.
This is the line: n8n/LoadNodesAndCredentials.ts at master · n8n-io/n8n · GitHub

And now it works stable again:

grafik
grafik262Ă—654 3.9 KB

i keep the old version for now

11

I created an issue for this: >= 0.204.0 EBUSY errors in cache · Issue #5274 · n8n-io/n8n · GitHub

12

@lublak The configuration file is read multiple times because you are running with executions.process as own, which runs every execution in a new process. And since we aren’t passing in the config from the parent process to child processes, these new processes need to read the config file.

13

The EBUSY error is likely happening because there is some process on your machine (possibly an antivirus) that’s preventing any other process from reading the source svg file.

Can you please try these steps ?

  1. delete C:\ProgramData\npm\npm\node_modules\n8n
  2. delete C:\Users\testadmin\.cache\\n8n
  3. install latest n8n again

An alternative solution would be switch over to running n8n in docker, which would prevent 3rd party software from messing with internal files of n8n.

14

Thanks for the info :slight_smile:

15

First of all: Currently there is no antivirus enabled on the pc. Even the defender is disabled.
It was a complete fresh installation.

  1. Already tested
  2. Already tested
  3. Already tested
  4. Also tested on my local windows 10 pc
  5. I test it if I’m at home on my private pc

The file is only blocked by n8n itself. There is no process blocking the file.

The issue with docker is that n8n needs access to all files of the system.
Even Network shares are needed.
Also we only allowed to use windows servers.

16

I am experiencing the same thing (have been for a little while now)

Looks like for some reason n8n is trying to copy svg files for cache purposes (or maybe having npm do this). It is very sporadic (different svg files not even related to the workflow that is running always from n8n-nodes-base\dist\nodes → .cache\n8n\public\icons\nodes).

As stated, was working perfectly fine on older versions (and I’ve been running n8n for years). Even tried a fresh clean isolated system, and errors still persist. I know windows has its quirks, but some environments are not able to use deployment methods like docker for reasons such as inability to perform virtualizing, or access to certain resources.

Ive commented on the GH issue. Possible culprit: cli/src/LoadNodesAndCredentials.ts#L412 - Merge By PR#4865 on December 12th, 2022

17

Yes it is this line.
I don’t quite understand why the SVG files have to be copied at all.
As I said: Even without antivirus software there are these errors.

18

After some testing I have confirmed this is the issue. Seems that this runs every time a workflow is run. The issue pops up if there are multiple workflows running at the same time, they will copy these cache files, but the files in the destination are still being used by the other operations (n8n is its own culprit xD). In reality, not sure the reason these need to be cache’d every time a workflow runs (maybe on workflow saves, or once on load would be a better choice?)

short of cloning the main repo commenting out the section and rebuilding, my temp solution is after this has run once (so the svgs are there) to go into file \AppData\Roaming\npm\node_modules\n8n\dist\LoadNodesAndCredentials.js (assuming you have it installed globally) and commenting out line 245, replacing it with a return true;

I would submit a PR with a fix, but not sure how core team would want to handle this. If they are gonna copy the files with every execution, should only need to copy file if it doesn’t exist or is outdated, or possibly just silent squash the error so the workflow can continue.

19

I think at each start should actually be sufficient. The updates gibts rather only when you update n8n.
Evneutll if here also the custom nodes are copied: When installing nodes run again. Or execute only for these nodes?

20

now that Im thinking about it, i guess it goes back to what @netroy said where, it runs per every workflow because they run in their own instances, so it needs to load the nodes and credentials for it. Though that still doesn’t seem right that it needs to cache icons every run (feel like there could be a better place to handle that).

I think at least for the time being, tossing a try/catch in there and just silent fail so the workflow still runs might be at least a starting point. Just doing a lil testing on it and I’ll look into tossing a PR in shortly.