Main proccess randomly starting webhook jobs

Apparently my main process is receiving incoming Webhook requests at very random rate(1 o 2 request every 3 minutes). While my Webhooks processors continuously receiving incoming requests at 60+ requests per seconds.

I already checked the main process with tcpdump, and I don’t see any HTTP request coming from the load balancer. The Workflow started in the main process are Production Webhook Triggered and aren’t duplicates in any of the Webhook processors.

So I’m wondering where are those requests coming from?

Setup:
I’m using Traefik routing with 1 main process, 2 Webhook processors and 4 workers.

Main process routing

      - traefik.http.routers.n8n.rule=Host(`${DOMAIN_NAME}`) && !PathPrefix(`/webhook/`)

I already disabled main process in my main n8n node

      - N8N_DISABLE_PRODUCTION_MAIN_PROCESS=true

Webhook processor routing

      - traefik.http.routers.n8n_webhook_worker.rule=Host(`${DOMAIN_NAME}`) && PathPrefix(`/webhook/`)

Are those not your test workflow executions when you develope and test workflows?

no, they are not from /webhook-test. I don’t even see a single HTTP traffic with tcpdump in the main n8n node.

as you can see in this screenshoot. all these execution id are the same workflow, but randomly 1 will be called in the main n8n process.

Then it will be a workflow that does not use a Webhook based trigger, rather any other trigger that exists like Interval, Cron, RabbitMQ, all the ones that use polling,…

I only have 1 trigger of type webhook in this workflow. no interval, cron or rabbitmq nodes in this workflow.

The execution id 70438196 you see previously that was picked up by the main n8n process:

hm super strange. Then I am sadly out of ideas for now.
I am only certain that n8n does not make up API requests by itself. So would still expect that it receives something. If you say that tcpdump does not show anything, then it must receive it via a different network or something like that.

Does looking at the data the Trigger received help to clear that up?

After more testing and debugging, I think I found something interesting

I stopped the high traffic Webhook call(let say its the workflow 20), and the main process with N8N_LOG_LEVEL=debug.
The main process is executing workflow 10 and workflow 15 which are RabbitMQ and Time Interval triggers(works as expected). And none of them was from a Webhook(I still have some others low traffic webhook enabled and are executed in the webhook processors as expected too).

But with high traffic those non Webhook workflows was not showing in n8n main process. Which let me suspect that the n8n main process is still executing those non Webhook workflows but showing an incorrect Execution Id.

If you have some insight about this possible behavior let me know. In the mean time I will try to replicate it in a sandbox to validate my suspicion

Yes you are correct @stwo

The ID displayed on your terminal is not the execution ID inside n8n. It is rather an internal ID used in Redis queue system (Bull).

Unfortunately there is no way to correlate these IDs.

I created a small PR adding clarification information so it’s not so confusing. You can see it here: Add execution ID to queue processes for clarification by krynble · Pull Request #2168 · n8n-io/n8n · GitHub

1 Like

Thanks a lot @krynble . Merged the PR.

The actual internal execution ID is actually available at that point and does not get displayed. Will be released with the next version.

Thank you for your clarification. Everything is working as expected then.

Got released with [email protected]