Webhook Scaling Issues

Questions
1

Hello!

I’ve been testing out setting up scaling on a self hosted instance following the Scaling guide in the docs.
I’ve successfully set up a worker to handle executions but I’m having issues getting the webhook processor to work.

I have a loadbalancer (Traefik) working correctly as it reaches the Webhook processor but I am not seeing anything happening on the redis server and then I recieve an error page.

The Environmental variables for the webhook processor instance are:

N8N_ENCRYPTION_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
DB_TYPE=postgresdb
DB_POSTGRESDB_DATABASE=n8n
DB_POSTGRESDB_HOST=x.x.x.x
DB_POSTGRESDB_PORT=5432
DB_POSTGRESDB_USER=user
DB_POSTGRESDB_PASSWORD=password
QUEUE_BULL_REDIS_HOST=x.x.x.x
QUEUE_BULL_REDIS_PORT=6379
QUEUE_BULL_PREFIX=webhook
WEBHOOK_URL=https://subdomain.domain.com/

Env for main process:
N8N_BASIC_AUTH_ACTIVE:true,
N8N_BASIC_AUTH_USER:“user”,
N8N_BASIC_AUTH_PASSWORD:“password”,
WEBHOOK_URL: “https://subdomain.domain.com/”,
N8N_LISTEN_ADDRESS: “0.0.0.0”,
N8N_ENDPOINT_WEBHOOK: “api”,
N8N_ENDPOINT_WEBHOOK_TEST: “api-test”,
N8N_METRICS:true,
N8N_PAYLOAD_SIZE_MAX:32,
DB_TYPE:“postgresdb”,
DB_POSTGRESDB_DATABASE:“n8n”,
DB_POSTGRESDB_HOST:“x.x.x.x”,
DB_POSTGRESDB_PORT:“5432”,
DB_POSTGRESDB_USER:“user”,
DB_POSTGRESDB_PASSWORD:“password”,
NODE_FUNCTION_ALLOW_EXTERNAL:“uuid”,
QUEUE_BULL_REDIS_HOST:“localhost”,
QUEUE_BULL_REDIS_PORT:“6379”,
EXECUTIONS_MODE:“queue”,
N8N_DISABLE_PRODUCTION_MAIN_PROCESS:“false”

When hitting the production link for a webhook I get this simple error:
“Cannot GET /api/test”
With a 404 error code.
I can’t seem to pinpoint the issue it may be with redis but I’m not getting any errors associated to redis just an absence of activity.
Or its acting like its own instance and not talking to the main instance at all.

If anyone has any ideas what may be going wrong/ I am doing wrong or has any insight into how the Webhook processors work I’d be greatful to hear it.

Thanks in advance!

Information on your n8n setup

  • n8n version:0.157.1
  • Database you’re using (default: SQLite):Postgres
  • Running n8n with the execution process [own(default), main]:
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]:npm
2

Hi @Jaime_B, is the workflow containing your webhook node active? This would be indicated by a green background on the toggle in the upper right corner of your canvas:
image

For deactivated workflows, a 404 response would be expected when hitting a webhook’s production URL.

3

Hi @MutedJam,

Yes its currently active, I also tested it without the webhook processor and it functions normally.

I’m not sure if I’ve missed something in the docs as it seems to all be correctly set up.

4

I found the issue for anyone who may make a similar mistake!

I forgot to set the env variables:
N8N_ENDPOINT_WEBHOOK: “api”,
N8N_ENDPOINT_WEBHOOK_TEST: “api-test”,
on the webhook processor!

2 Likes
5

Hello,

I have a similar issue - I am getting “cannot GET /webhook/” when I try to start the queue mode with wh processor, however Jaime_B 's solution didn’t work for me.

I run n8n in a docker-compose setup in an ec2 linux instance and have been able to make it run with postgres, but when I add redis and workers+processor, I receive cannot GET.

I suspect the issue in my case comes from load balancing, as this is not documented anywhere in the scaling docs apart from:

If you are using the same domain name for your n8n instance and the webhooks, you can set up your load balancer to route requests as follows:
Redirect any request that matches /webhook/* to the webhook servers pool

@Jaime_B you said you have traefik configured as a load balancer, could you please share thy wisdom on how did you achieve that? Sorry I have nigh 0 cloud networking experience.

If appropriate, I will start my own thread for this with full config details.

Thanks!
Michal

6

Hey @michal,

A new thread might be best, One of the things we don’t document is the load balancing side as it all depends on what you are using and it would be almost impossible to document every combination. I would suggest using whatever you are comfortable with, Personally I would recommend looking at Caddy although I believe Amazon have their own cloud load balancer that you could use.

7

@michal I would agree with Jon here, possibly starting a new thread with details on the current load balancing set up if any exists.
I’ve also recently moved some of my instances to Caddy aswell which is super simple to set up but you’d need to know how to configure your private networks and firewalls within AWS to ensure routing is sufficient

1 Like
8

Thanks guys. As suggested, I created a new topic for this, with detailed rundown of my setup.
https://community.n8n.io/t/n8n-scaling-problems-cannot-get/17882