Hey @bmb very interesting questions you’ve brought.
So, let’s divide the answer in separates parts:
Your current deployment
Using kubernetes to scale the number of n8n instances with a load balancer is fine but can have problems in the future IF you decide to use any trigger that is not based on an HTTP Request (a cron, interval or similar, for instance).
This will happen because you now have multiple n8n instances doing the same thing, which is triggering an execution once some time has passed. You would have duplication of work.
This is one of the reasons we have separated the workers and webhooks to separate processes, so these can be scaled.
TL;DR: right now it won’t cause trouble if all your workflows are initiated from HTTP Requests.
Improvements by switching to queue
This would make your n8n deployment more future-proof, separating webhook and worker processes into separate containers and adding a message broker (Redis, in our case) to handle the traffic.
Using main mode
If you’re not using queue mode, then
own are the 2 possible execution methods. When using
main, it means the executions all happen inside a single process, which is the n8n process. This comes with a major drawback: uses only 1 CPU core at a time, makes this process bigger (by concentrating resources) and makes your executions compete for CPU time in this single threaded environment. To counter this, you have blazing fast speed as everything happens in a single place, and if latency is sensitive, this could be the way to go.
On the other hand, when using
own each execution happens in a totally separate process. This means all the incurred costs related to forking a new process (this is an intense OS operation) but it is more isolated therefore more secure (i.e. less prone to crashes), can use multiple cores from your machine and scales better with the hardware you have.
I hope these points made it clearer for you! If anything is not clear, please feel free to ask!