The best practice for scaling n8n involves separate processes for managing webhooks, such that urls with ‘webhook-test’ are routed to the main n8n process, while production webhooks are routed to the webhook processes. Currently, given that there is only one environment variable for setting up the webhook host, both the test and production webhooks are sent to that host.
If there was a TESTING_WEBHOOK_URL environment variable, that defaulted to the WEBHOOK_URL environment variable, that enabled the UI to point the webhook test url to the main process endpoint, that would simplify deployment substantially.
Similar to how there is a N8N_ENDPOINT_WEBHOOK and N8N_ENDPOINT_WEBHOOK_TEST.
My use case:
I’d like to deploy n8n without having to also deploy a loadbalancer/reverse proxy to send webhook tests to the main process and production webhooks to the webhook processes.
I think it would be beneficial to add this because:
simplify scalable deployment by removing the need for a reverse proxy in front of both the main process and webhook processes.
Any resources to support this?
Are you willing to work on this?
Yes! I’m not the best at javascript, but if pointed in the right direction (where to make the changes and where the tests live) I’d be happy to put together a PR.
It doesn’t throw any errors, nor does it break tests, however, I cannot, for the life of me, figure out why it’s not showing. So not really an implementation, more of an attempted implementation.
Someone who understands vue perhaps could help figure out why we aren’t displaying the correct value for the test webhook?
I am a bit curious is this just around having different test urls or around scaling? It would be interesting to know a bit more as I am not sure there is a way to scale nicely without using a load balancer or a reverse proxy as you would likely want to test the endpoints and distribute the load where possible.
Hi Jon!
Thanks for following up and taking a look at this!
This is about scaling and user experience: I’m looking at PaaS options and many, caprover, fly.io, etc don’t make it easy to run different process types in the same app/domain. Moreover, many PaaS ship with their own load balancers in front of apps, distributing requests automatically to application nodes. Also, I would like to keep where my users edit nodes and the public webhook urls separate: my public webhooks need less security. Beyond that, my understanding is that if you run workflows on a schedule (cron) you only want to have one main node.
In my ideal setup, I would have three distinct “apps”: main, worker, webhook. Each on their own domain, all with the same postgres/redis backend.
In order for my users to have a straightforward workflow development process, either all requests must proxy through my “main” app, or my users must edit their testing urls, or I have an additional layer in my application stack that does routing that could be done by the platform if I could specify webhook testing and production URLs.
If I could specify the domains that should receive production and testing webhooks separately, I would not have to maintain a routing/proxy layer in my app, which would reduce complexity and overhead.
My incomplete and non-functional implementation defaults to current behavior if WEBHOOK_TEST_URL isn’t specified, so shouldn’t break anyone’s setup.
In short, there is a load balancer, it’s just provided by the PaaS, and to make things simple for my users, being able to have the webhook tests be a different domain than the production webhooks would simplify the infrastructure.