ERROR: Cannot create a string longer than 0x3fffffe7 characters

Questions
1

I am using the FTP download node and I have a zip file I am trying to download and I get the following error:
“ERROR: Cannot create a string longer than 0x3fffffe7 characters”

Error: Cannot create a string longer than 0x3fffffe7 characters
at Object.slice (buffer.js:653:37)
at Buffer.toString (buffer.js:801:14)
at Object.prepareBinaryData (/usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/src/NodeExecuteFunctions.js:34:26)
at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Ftp.node.js:427:84)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/src/WorkflowExecute.js:395:47

I’ve looked online and the solution is to change the way this node works:

2

How large is the file you are trying to download?

3

I think that ftp node is trying to allocate file to memory before downloading, but memory is lower than file size.

Usually, downloading file by streams make sense with big files. Some sample in nodejs can be found here

4

It’s 1 GB, and I expect it to be a little bit larger than that.
I wouldn’t have thought this is a problem for real FTP (so please don’t say, make the file smaller). :slight_smile:

5

So, you’re saying, I can make the n8n machine have more RAM and it’ll be fine?
(In which case I should mention there’s over 4 GB’s free memory for it)

6

If you have limited your container to 1GB of memory, probably the problem is caused by this limitation.

Can you retry to download file by ftp after increasing container memory to 2GB?

7

Yes, that will currently not be possible with n8n. The reason is the design. It was simply never written to handle huge files. That is maybe something that will change in the future but that would require a bigger refactor.

Technical explanation. All data in n8n gets saved as JSON. All binary data gets converted to base64 and so also saved as a string. The maximum size of a string in Node.js is 1GB. Considering that base64 increases the file size, does it mean that the maximum file size n8n can handle is probably around 750 MB.

3 Likes
8

Got it. @voarsh2 this is your response. There is a string limitation of 750MB.

I think you can try to work with Execute Command node.

You can pass file to download it using an ftp client (internally working with streams when downloading big files).

This will work to by-pass FTP node limitations.

Hope this helps.

9

I’ve no idea how it might work with the Execute Command Node.

I can’t even install PIP on Execute Command Node?
It might be easier to run n8n on a LXC/VM that has a mounted NFS folder, and copy it. But I’d still need to download it first.

10

I have checked and ftp client is not available by default.
You can’t install pip directly.

Probably yes. Accesd¡sing NFS folder would be the best option.

11

I have checked and ftp client is not available by default.

You mean on the Execute Command Node??
Can Execute Command Node do scp file transfer?

12

You can literally run anything you want with the Execute Command Node that is available on the system on which n8n runs.

13

That’s strange, because on docker, on my VM, I have sshpass installed:

image
image772×223 6.77 KB

But the same doesn’t work on execute node?

image
image845×276 7.23 KB

14

That are two totally different systems. n8n runs in the Docker image and that one will not have sshpass installed unless you build a custom n8n image and you add it there manually.

15

Yes. Docker image contains very basic commands.

If you work with big files (>750MB), you won’t be able to download files from n8n unless you create a custom image for n8n with required binaries.

I usually work with external scripts (lftp, rsync or s3cmd) to copy/synchronize big files between machines.

16

Seems I can’t install anything on my container, so I can’t even do this.

17

Strange, because I’ve ran docker stuff before and were able to run system stuff to install Python, Pip packages and much more.
What’s required for customising the image? Just adding a few lines to the docker-compose.yml file?

If that doesn’t work, then I guess I’m stuck with waiting for this refactoring… or reinstalling n8n and not using docker…

18

You can find an example here:

19

Hi all.

I’ve run into a similar issue, i think.

ERROR: Cannot create a string longer than 0x1fffffe8 characters

I’m trying to load (tried via Google Drive/HTTP/Read a Binary File nodes) a zip file around 450MB in size.

Is there any workaround for this?

Env:

n8n 0.164.1
Local (Windows) System
Docker
Docker RAM: 3GB

20

This is memory limit issue. 0x1fffffe8 means 512MB

Found this StackOverflow