N8n process eating away all the available memory on VM

I have a workflow which uses some custom node to call some API and send data to airtable via MQTT triggers. I have setup it on both local and an aws instance. The workflow runs fine on local machine. However, the same workflow is eating away the whole memory available on aws instance. Even after doubling the amount of RAM, it’s taking over whatever amount is available. I also tried with some other workflows with active MQTT triggers, but had the same issue on VM. I am using n8n version 0.144.0. It is built on WSL 2 Ubuntu 20.04.3 LTS on local machine. On aws instance, it is built on Ubuntu 20.04.2 LTS.

I am attaching the htop screenshot for both local machine and aws instance. On local machine, I guess, some of the threads are sleeping, while others running. On aws instance (one with 16GB RAM), all of the threads seems to be running. Kindly give some insight, why this might be happening.

1 Like

I assume the problem is that you receive a lot of events via MQTT and n8n starts a new workflow execution for each of them.

So to solve that you have to either:

  • make sure that it sends fewer MQTT events
    or if that is not possible try:
  • give n8n more resources (so that n8n get all the resources it needs to process them)
  • reduce the resources each execution requires by making the workflows faster to process
  • make sure to use main mode so that each execution does not start an own process, that will lighten the load a lot (set environment variable EXECUTIONS_PROCESS=main) ← First thing I would try!
4 Likes

Thanks. The last solution was what I needed.

2 Likes

Glad to hear. Have fun!

1 Like