Hello Community!
I’m using the Cloud version of n8n and running into memory issues when trying to process a large CSV file. Here’s what I’m doing:
-
I use an HTTP Request (GET) node to download a CSV file from a URL . The URL directly returns a CSV file (~20MB, ~110,000 rows), and the output of the HTTP Request node is the binary CSV file.
-
I pass that binary data into an Extract node , trying to convert it into JSON so I can standardise the fields in subsequent steps.
The problem: when I try to process the full file, the Extract node produces no output at all. I don’t get any error message — just “Execute this node to view data or set mock data” is shown under the node. However, if I set the Extract node to limit the rows to 50,000, it does work and I get valid JSON output.
But I need to process all 110,000 rows, and I’m not sure what the best practice is here. I’ve read that breaking up data and using sub-workflows helps reduce memory usage, but I’m unclear how to do that when starting with a large binary CSV file from an HTTP Request node.
Any suggestions on how to approach this? Examples would be super helpful!
My Workflow
Detail for HTTP node
Detail for Extract node
- n8n version: 1.91.3
- Database (default: SQLite): SQLite
- n8n EXECUTIONS_PROCESS setting (default: own, main): own
- Running n8n via (Docker, npm, n8n cloud, desktop app): n8n cloud
- Operating system: Windows 10
Any help would be greatly appreciated!