Excessive ram usage by FTP node

Hi everyone,
first of all let me tell you n8n is an amazing product and we are making extensive use of it in my company.
Some days ago I was checking the memory usage and I encountered a weird issue: the ftp node and generally speaking, every other node involving the buffer, use a huge and very unusual amount of ram.

In the example workflow down here I simply download a test file (100 MB) and the ram usage peaks at 1.5 GB.
For bigger projects we “fixed” the issue by setting the filesystem as temporary storage, but I was wondering if you could maybe address the issue.

The credential for the FTP connection are public:

  • Host: ftp.otenet.gr
  • Port: 21
  • User: speedtest
  • Password: speedtest

We are currently running n8n via Docker at the latest version.

Thank you and keep up the great work!

1 Like

Hi @Shockwave, welcome to the community!

Many thanks for sharing these details. By mentioning filesystem as temporary storage, do you mean you’re already setting the N8N_DEFAULT_BINARY_DATA_MODE=filesystem environment variable (to tell n8n to use the filesystem rather than memory for keeping binary data)? This would keep the memory usage below 500MB when I just tested this.

Hi @MutedJam , thank you for your reply!

Yes, you are correct: in order to solve the issue I set the N8N_DEFAULT_BINARY_DATA_MODE=filesystem env. Despite this change, I think ~500MB is probably a still preatty high mem usage, and I’m also wondering if there’s a more permanent fix to the original memory leak that occurs when I keep the default parameter and use the ram to store data.

Thank you

I am not aware of any plans to improve memory consumption when working with binary data (beyond the filesystem-based handling) at the moment unfortunately, mut maybe @sirdavidoff can share more details here?