CSV load fail

Questions
1

I’m working on a workflow that goes as follows: take a .csv file of 100 MB or more, read it, send SMS messages, then update the status saying that it has been sent and move it to a new folder for sent files (on the same machine where N8N is running).

I’m having trouble reading the file. I’ve tried converting it to JSON, but it resulted in an error. I’ve also tried extracting the data in other ways, but it didn’t work.

Today, I’ll test it with RabbitMQ. I’ll send the entire file to RabbitMQ and then process it using N8N, following the workflow.

I’m open to suggestions using N8N due to the urgency. Otherwise, I would have gone with coding… Even Python, in which I’m still a junior.

Thanks in advance, and have a great week, everyone.

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

hello @MarcoFinal

Can you share the workflow? And what is the error/issue?

4

sorry

  • n8n version: 1.15.2
  • Database (default: SQLite): postgres
  • n8n EXECUTIONS_PROCESS setting (default: own, main): own
  • Running n8n via (Docker, npm, n8n cloud, desktop app): docker swarm
  • Operating system: Debian
5

image
image2184×946 121 KB

I still don’t know how to share the workflow
but
printscreen

6

to share a workflow, press crtl+a ctrl+c and past the content here after pressing the </> button

image
image1064×56 10.3 KB

1 Like
8
9

And what is the issue?

The code node may consume a lot of memory if the CSV is too big, so I’d suggest to move the Code node into a sub workflow. In that case after the sub workflow will be executed, the memory will be released

10

I ended up making a shell script that reads the CSV, imports it into Postgres and then follows the flow through Postgres
the 90MB CSV doesn’t open and doesn’t extract the data so I can loop through it and send it directly through N8n

1 Like
11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.