Uncompress large zip > 1GB

[Simone_Di_Cola]

Describe the problem/error/question

I am unable to decompress a large zip file (1.6 GB). I am running n8n locally on node and it is stuck for 20 minutes eating up 32GB of RAM and almost all the CPU (got a mac studio max).

What is the error message (if any)?

N/A

Please share your workflow

Share the output returned by the last node

Information on your n8n setup

• n8n version: 1.25.1
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app): node
• Operating system: macOS 14.2.1 (23C71)

[Simone_Di_Cola]

I am more than happy to split that in other zip files and create a sub workflow to process those, but I have no idea how to get the path of the downloaded archive from the previous step

split -b 100M /path/to/your/largefile.zip /path/to/destination/folder/prefix

[Simone_Di_Cola]

To clarify, n8n does uncrompress it, but it takes forever and uses an obscene amount of resource.

[Jon]

Hey @Simone_Di_Cola,

Welcome to the community

How do you have n8n configured? If you have not changed the default binary handling mode to filesystem it may cause a bit of slowness and everything would be handled in memory.

There is also a good change that we could improve the performance of the decompress node as I don’t think we have touched it in a while.

If I had a large file to decompress through and I was running n8n from npm I would probably use the execute command node and pass it off to the OS to decompress using the tools it ships with as it may be quicker.

[Simone_Di_Cola]

Thanks for your answer @Jon . I do use filesystem in my settings. You can try the workflow I am using yourself and you can see that in order to uncrompress 1.7GB takes about 25 minutes…

[Simone_Di_Cola]

@Jon I have saved the zip and I am trying to unzip it using the cmd node.

Now it got stuck there

[Jon]

It was indeed very slow on my machine as well, How long does it take your Mac to decompress the file from CLI normally?

Just tested it on mine and there is a pretty big difference, Sounds like there are some improvements to be made there.

[Simone_Di_Cola]

it takes less than a minute.

[Simone_Di_Cola]

it hangs there

[Simone_Di_Cola]

ok I got it working using unzip.

[Jon]

Hey @Simone_Di_Cola,

That is good to hear, We have created a dev ticket to improve the performance of the compress node in the future as well.