Excessive RAM usage on Client

FelixL · April 18, 2023, 4:02pm

Sadly, my Laptop only has 16 GB ram right now (soldered)
I am using Firefox right now, and it’s the only program opened on my Laptop with only 1 Tab of N8N with a quite big workflow.
After half an hour working on the workflow, Firefox uses over 8 GB ram.
Reopening the workflow helps.

Is that expected behaviour right now?
If not, is there any info/logs I can provide to help to get to the root cause of this?

Information on your n8n setup

n8n version: 0.222.3
Database (default: SQLite): MySQL
n8n EXECUTIONS_PROCESS setting (default: own, main):
Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
Operating system: Ubuntu (Windows 11 on Client)

Jon · April 19, 2023, 7:10am

Hey @FelixL,

I had noticed a similar issue last night while working on a workflow, I managed to create a workflow that triggers the same issue which is handy.

Would you be able to also share the workflow you were working on so we can check for any patterns just to confirm we are on the right path?

FelixL · April 19, 2023, 7:30am

Yeah sure, but I do not know any more which workflow I worked on.

I’ll work a few hours today with N8N, as soon as I see similar behaviour again I’ll send you a pm (or here, depending on the workflow) with the workflow.

FelixL · April 19, 2023, 9:37am

I couldn’t reproduce the issue right now, I’ll make an update when I have the problem again.

Should I try your workflow on my system?
Which nodes/combination of nodes are you using? (To narrow down which workflow caused this in my case.)

Jon · April 19, 2023, 10:25am

Hey @FelixL,

For me it was my workflow that clears old workflows out, Specifically though it appears to be when using set nodes with lots of data coming in. We have the team looking into it though so I didn’t want to lead you down any specific paths with it

FelixL · April 19, 2023, 10:45am

ok, got a few set nodes with a lot of items ^^

I am using the set nodes to remove not needed fields to help with memory requirements. I hope that’s the right way of thinking here.

I’ve also found the workflow which used 8GB of RAM on the client yesterday:
It has two pipedrive nodes which read about 600 deals at the same time and processes the data for our dashboard. (I have to split it two workflows, but thats at the moment not really easy todo: Bug node selection does not work)

When execution the workflow it uses 800MB on the Client (150MB on the Server), but as soon as the Execution finishes it releases that memory, which was not the case yesterday.

Writing another post rn, where one of my workflows got a memory problem and with it failed multiple other workflows.

FelixL · April 19, 2023, 10:52am

Another workflow which also caused problems yesterday:

Workflow

This workflow has 3 pipedrive input nodes, one with about 7000 items, one with, 600 items and one with 1300 items. all 3 inputs get cleaned with a set node.

I’ve already split that workflow into 4 smaller workflows.

Jon · April 19, 2023, 11:03am

That is perfect, So I suspect it is the same thing. Thanks for sharing those.

FelixL · April 19, 2023, 12:55pm

If there is anything else I can provide, i am happy to assist (:

Here is a memory report of one of the workflows:
made in Firefox with “about:memory”

memory report

webIsolated=https://example.at (pid 25192)
Explicit Allocations

1,934.90 MB (100.0%) -- explicit
├──1,721.66 MB (88.98%) -- window-objects
│  ├────591.36 MB (30.56%) -- top(https://n8n.example.at/workflow/19/executions/63421, id=8589934599)
│  │    ├──404.36 MB (20.90%) -- active
│  │    │  ├──212.51 MB (10.98%) -- window(https://n8n.example.at/workflows/demo)
│  │    │  │  ├──206.72 MB (10.68%) -- js-realm(https://n8n.example.at/workflows/demo, about:blank)
│  │    │  │  │  ├──200.30 MB (10.35%) -- classes
│  │    │  │  │  │  ├───58.32 MB (03.01%) -- class(Object)/objects
│  │    │  │  │  │  │   ├──41.59 MB (02.15%) ── gc-heap
│  │    │  │  │  │  │   └──16.73 MB (00.86%) ++ malloc-heap
│  │    │  │  │  │  ├───57.05 MB (02.95%) -- class(Array)/objects
│  │    │  │  │  │  │   ├──44.07 MB (02.28%) ── gc-heap
│  │    │  │  │  │  │   └──12.98 MB (00.67%) ++ malloc-heap
│  │    │  │  │  │  ├───34.07 MB (01.76%) -- class(Function)/objects
│  │    │  │  │  │  │   ├──33.72 MB (01.74%) ── gc-heap
│  │    │  │  │  │  │   └───0.35 MB (00.02%) ── malloc-heap/slots
│  │    │  │  │  │  ├───31.20 MB (01.61%) -- class(Call)/objects
│  │    │  │  │  │  │   ├──31.15 MB (01.61%) ── gc-heap
│  │    │  │  │  │  │   └───0.05 MB (00.00%) ── malloc-heap/slots
│  │    │  │  │  │  └───19.68 MB (01.02%) ++ (10 tiny)
│  │    │  │  │  └────6.41 MB (00.33%) ++ (5 tiny)
│  │    │  │  └────5.80 MB (00.30%) ++ (3 tiny)
│  │    │  └──191.84 MB (09.91%) -- window(https://n8n.example.at/workflow/19/executions/63421)
│  │    │     ├──185.17 MB (09.57%) -- js-realm(https://n8n.example.at/workflow/19/executions/62662, about:blank)
│  │    │     │  ├──177.45 MB (09.17%) -- classes
│  │    │     │  │  ├───54.14 MB (02.80%) -- class(Object)/objects
│  │    │     │  │  │   ├──38.43 MB (01.99%) ── gc-heap
│  │    │     │  │  │   └──15.71 MB (00.81%) ++ malloc-heap
│  │    │     │  │  ├───50.22 MB (02.60%) -- class(Array)/objects
│  │    │     │  │  │   ├──40.09 MB (02.07%) ── gc-heap
│  │    │     │  │  │   └──10.13 MB (00.52%) ++ malloc-heap
│  │    │     │  │  ├───30.58 MB (01.58%) -- class(Function)/objects
│  │    │     │  │  │   ├──30.25 MB (01.56%) ── gc-heap
│  │    │     │  │  │   └───0.33 MB (00.02%) ── malloc-heap/slots
│  │    │     │  │  ├───27.47 MB (01.42%) -- class(Call)/objects
│  │    │     │  │  │   ├──27.42 MB (01.42%) ── gc-heap
│  │    │     │  │  │   └───0.05 MB (00.00%) ── malloc-heap/slots
│  │    │     │  │  └───15.04 MB (00.78%) ++ (10 tiny)
│  │    │     │  └────7.72 MB (00.40%) ++ (6 tiny)
│  │    │     └────6.67 MB (00.34%) ++ (3 tiny)
│  │    └──187.01 MB (09.66%) -- js-zone(0x1e59e4d8300)
│  │       ├───64.30 MB (03.32%) ++ (15 tiny)
│  │       ├───61.04 MB (03.15%) -- property-maps
│  │       │   ├──33.89 MB (01.75%) -- gc-heap
│  │       │   │  ├──31.41 MB (01.62%) ── dict
│  │       │   │  └───2.47 MB (00.13%) ++ (2 tiny)
│  │       │   └──27.16 MB (01.40%) -- malloc-heap
│  │       │      ├──26.93 MB (01.39%) ── tables
│  │       │      └───0.23 MB (00.01%) ── children
│  │       ├───41.66 MB (02.15%) ── unused-gc-things
│  │       └───20.00 MB (01.03%) ── unique-id-map
│  ├────430.08 MB (22.23%) -- top(none)/detached/window(https://n8n.example.at/workflows/demo)
│  │    ├──420.39 MB (21.73%) -- js-realm(https://n8n.example.at/workflows/demo, about:blank)
│  │    │  ├──408.04 MB (21.09%) -- classes
│  │    │  │  ├──119.29 MB (06.17%) -- class(Object)/objects
│  │    │  │  │  ├───84.22 MB (04.35%) ── gc-heap [2]
│  │    │  │  │  └───35.07 MB (01.81%) -- malloc-heap
│  │    │  │  │      ├──33.96 MB (01.76%) ── slots [2]
│  │    │  │  │      └───1.11 MB (00.06%) ── elements/normal [2]
│  │    │  │  ├──115.00 MB (05.94%) -- class(Array)/objects
│  │    │  │  │  ├───89.31 MB (04.62%) ── gc-heap [2]
│  │    │  │  │  └───25.69 MB (01.33%) -- malloc-heap
│  │    │  │  │      ├──24.70 MB (01.28%) ── elements/normal [2]
│  │    │  │  │      └───0.99 MB (00.05%) ── slots [2]
│  │    │  │  ├───68.99 MB (03.57%) -- class(Function)/objects
│  │    │  │  │   ├──68.22 MB (03.53%) ── gc-heap [2]
│  │    │  │  │   └───0.77 MB (00.04%) ── malloc-heap/slots [2]
│  │    │  │  ├───63.01 MB (03.26%) -- class(Call)/objects
│  │    │  │  │   ├──62.92 MB (03.25%) ── gc-heap [2]
│  │    │  │  │   └───0.10 MB (00.01%) ── malloc-heap/slots [2]
│  │    │  │  ├───38.47 MB (01.99%) -- class(Set)/objects
│  │    │  │  │   ├──37.30 MB (01.93%) ── malloc-heap/misc [2]
│  │    │  │  │   └───1.17 MB (00.06%) ── gc-heap [2]
│  │    │  │  └────3.28 MB (00.17%) ++ (10 tiny)
│  │    │  └───12.35 MB (00.64%) ++ (3 tiny)
│  │    └────9.68 MB (00.50%) ++ (3 tiny)
│  ├────373.20 MB (19.29%) -- top(https://n8n.example.at/executions, id=8589934594)
│  │    ├──287.77 MB (14.87%) -- active/window(https://n8n.example.at/executions)
│  │    │  ├──269.62 MB (13.93%) -- js-realm(https://n8n.example.at/workflow/16/executions/62660, about:blank)
│  │    │  │  ├──260.89 MB (13.48%) -- classes
│  │    │  │  │  ├──104.60 MB (05.41%) -- class(Object)/objects
│  │    │  │  │  │  ├───57.54 MB (02.97%) ── gc-heap
│  │    │  │  │  │  └───47.06 MB (02.43%) -- malloc-heap
│  │    │  │  │  │      ├──46.46 MB (02.40%) ── slots
│  │    │  │  │  │      └───0.60 MB (00.03%) ── elements/normal
│  │    │  │  │  ├───62.24 MB (03.22%) -- class(Array)/objects
│  │    │  │  │  │   ├──51.85 MB (02.68%) ── gc-heap
│  │    │  │  │  │   └──10.40 MB (00.54%) ++ malloc-heap
│  │    │  │  │  ├───38.32 MB (01.98%) -- class(Function)/objects
│  │    │  │  │  │   ├──37.45 MB (01.94%) ── gc-heap
│  │    │  │  │  │   └───0.87 MB (00.05%) ── malloc-heap/slots
│  │    │  │  │  ├───33.91 MB (01.75%) -- class(Call)/objects
│  │    │  │  │  │   ├──33.87 MB (01.75%) ── gc-heap
│  │    │  │  │  │   └───0.05 MB (00.00%) ── malloc-heap/slots
│  │    │  │  │  └───21.81 MB (01.13%) ++ (21 tiny)
│  │    │  │  └────8.73 MB (00.45%) ++ (6 tiny)
│  │    │  └───18.15 MB (00.94%) ++ (3 tiny)
│  │    └───85.43 MB (04.42%) -- js-zone(0x1e599583100)
│  │        ├──58.91 MB (03.04%) ++ (17 tiny)
│  │        └──26.52 MB (01.37%) -- property-maps
│  │           ├──24.49 MB (01.27%) -- gc-heap
│  │           │  ├──21.95 MB (01.13%) ── dict
│  │           │  └───2.53 MB (00.13%) ++ (2 tiny)
│  │           └───2.03 MB (00.11%) ++ malloc-heap
│  └────327.01 MB (16.90%) -- top(https://n8n.example.at/executions, id=53)
│       ├──279.40 MB (14.44%) -- active/window(https://n8n.example.at/executions)
│       │  ├──263.73 MB (13.63%) -- js-realm(https://n8n.example.at/)
│       │  │  ├──255.08 MB (13.18%) -- classes
│       │  │  │  ├───97.53 MB (05.04%) -- class(Object)/objects
│       │  │  │  │   ├──55.60 MB (02.87%) ── gc-heap
│       │  │  │  │   └──41.92 MB (02.17%) -- malloc-heap
│       │  │  │  │      ├──41.31 MB (02.13%) ── slots
│       │  │  │  │      └───0.62 MB (00.03%) ── elements/normal
│       │  │  │  ├───62.53 MB (03.23%) -- class(Array)/objects
│       │  │  │  │   ├──51.61 MB (02.67%) ── gc-heap
│       │  │  │  │   └──10.93 MB (00.56%) ++ malloc-heap
│       │  │  │  ├───38.62 MB (02.00%) -- class(Function)/objects
│       │  │  │  │   ├──37.82 MB (01.95%) ── gc-heap
│       │  │  │  │   └───0.80 MB (00.04%) ── malloc-heap/slots
│       │  │  │  ├───34.47 MB (01.78%) -- class(Call)/objects
│       │  │  │  │   ├──34.42 MB (01.78%) ── gc-heap
│       │  │  │  │   └───0.05 MB (00.00%) ── malloc-heap/slots
│       │  │  │  └───21.94 MB (01.13%) ++ (20 tiny)
│       │  │  └────8.65 MB (00.45%) ++ (6 tiny)
│       │  └───15.67 MB (00.81%) ++ (3 tiny)
│       └───47.61 MB (02.46%) ++ js-zone(0x1e58b750100)
├─────74.07 MB (03.83%) -- js-non-window
│     ├──29.84 MB (01.54%) -- gc-heap
│     │  ├──22.63 MB (01.17%) ── chunk-admin
│     │  └───7.21 MB (00.37%) ++ (2 tiny)
│     ├──25.25 MB (01.30%) ++ runtime
│     └──18.98 MB (00.98%) ++ (2 tiny)
├─────68.87 MB (03.56%) ── heap-unclassified
├─────40.91 MB (02.11%) -- heap-overhead
│     ├──26.68 MB (01.38%) ++ bin-unused
│     └──14.23 MB (00.74%) ++ (2 tiny)
└─────29.39 MB (01.52%) ++ (22 tiny)

netroy · April 20, 2023, 4:03pm

@FelixL A potentially related memory leak was fixed in this PR and released in 0.224.2.
Can you please upgrade, and if that fixes the issue for you, please let is know ?

FelixL · April 22, 2023, 2:36pm

@netroy I’ve just installed it, I’ll give some feedback as soon as I’ve worked a few hours with n8n.

FelixL · April 27, 2023, 10:23am

Hi @netroy,

I have not had any excessive ram usage in the last few days with any of my workflows.

But I just noticed, when I open this forum post, Firefox uses about 1.2 GB more RAM.
As soon as I go to another page on the N8N forum, 1.2 GB get released again.

netroy · April 27, 2023, 1:19pm

Thanks for reporting. That issue should be fixed now as well.