Convert Binary file To JSON or CSV

Describe the issue/error/question

Hello everyone, I have to transform a file in Binary (more than 400mb) downloaded from sFTP server and then, use that data in a CSV or JSON format.

Do you know the best way to handle a file like that and convert it in CSV or JSON?

What is the error message (if any)?

Please share the workflow

I’ve used the node “Move binary data” but it return me a string error formatting, so it fail

I’ve tried to use the “CSV” node, read and write new file, but it return empty item

Share the output returned by the last node

Information on your n8n setup

  • **n8n version:**0.217.2
  • Database you’re using (default: SQLite): npm version
  • Running n8n with the execution process [own(default), main]:
  • Running n8n via [Docker, npm,, desktop app]:
1 Like

Hi @Gabriele_Bracciali, I am sorry to hear you’re having trouble.

Tbh I think that file size could be a bit too much to parse at once for most n8n instances. But perhaps we can come up with a workaround. Is the file you are downloading a CSV file or something else that you would need to convert first? And do you have an example file by any chance?

it’s a CSV file the input.


email id_newsletter id_client genre_BI rfm_actuel rfm_precedent rfm_anciennete rfm_date_rafraichissement marge_ht_apres_log_total sport_client marques_clients_N1 marques_clients_N5 date_last_cmd_mag last_cmd_mag closest_shop_name distance_to_closest_shop update_time_bi
*[email protected] 25123859 2224656 monsieur 8. To test 5. New 38 28/03/2023 Route 136746,1,2238425,2 136746,1,2238425,2 BBA 25 28/03/2023

This is the flow:
download action on FTP node, then try to convert it

So this is a bit tricky, but I think a possible approach would be to store that huge file on the file system, then read it in chunks. This will only work if you’re not using n8n cloud though and you would need to adjust the paths used in this workflow to your own folder structure. But the basic idea could be to have a parent workflow like this to download a large file, write it to the file system and then prepare batches:

Then call a sub-workflow reading only a limited number of lines at once from that file, do something with it and then return only a very small result set to the parent rather than the full data:

Memory usage was quite low when testing this with a dummy file:

Now this is assuming your CSV file is properly formatted and the only problem is indeed its size. If not It’d be great if you could share an actual example of one of these files (you can of course redact anything confidential, the only thing that matter is that any issues you might have can be reproduced with it).

Hope this helps!

I am receiving Permission denied error in the “Count lines in file” node

So this could be a problem on your local filesystem. You might need to swap the file paths used in the workflow to a path that exists on your side and in which you have full permissions. Also, to run the sub-workflow individually you might need to use data pinning first to supply some dummy data.

To do so, first copy the data from the parent:

Then, in the sub-workflow click this button to edit the test data:

Finally, paste the test data and hit Save:

Afterwards you should be able to run your sub-workflow on its own for testing purposes:

I’ve tried:
/Users/gabrielebracciali/n8ntemp{{ $ }}
Here maybe I dont’have full permission, but I have changed into desktop and same permission error.

I am not arrived at the second workflow yet.

Hi @Gabriele_Bracciali, is your n8n instance perhaps running as a user other than gabrielebracciali, so it doesn’t have permissions to write into said user directory? What if you create a designated folder on your hard drive and simply allow everyone to write in there (just for testing purposes)?

Idk, I’ve setted the Sharing & Permission option like everyone Read & Write, but same error.
mac user

Are you by any chance not running n8n through npm (as suggested above) but using Docker (which would be the common way of doing so)?

I am running n8n through npm

Hm, I am afraid I don’t know why writing to the file system would fail in your case :frowning:

I don’t have a Mac though, so perhaps this OS adds some additional isolation layers somewhere along the way. Maybe fellow Mac user @Jon knows more about this?

Okay okay, thank you for your help, and sorry If we are going a bit off-topic with this other issue.

That’s alright, very large files can be very tricky in n8n. It’s why I thought handling these outside of n8n itself would make things easier, but I was clearly wrong :see_no_evil:

Hey @Gabriele_Bracciali,

Could you share the full error so I can take a look?

ERROR: Command failed: /Users/gabrielebracciali/n8ntemp/upload/flux_user_bi.csv /bin/sh: /Users/gabrielebracciali/n8ntemp/upload/flux_user_bi.csv: Permission denied

NodeOperationError: Command failed: /Users/gabrielebracciali/n8ntemp/upload/flux_user_bi.csv
/bin/sh: /Users/gabrielebracciali/n8ntemp/upload/flux_user_bi.csv: Permission denied

    at Object.execute (/opt/homebrew/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/ExecuteCommand/ExecuteCommand.node.ts:96:12)
    at Workflow.runNode (/opt/homebrew/lib/node_modules/n8n/node_modules/n8n-workflow/src/Workflow.ts:1270:19)
    at /opt/homebrew/lib/node_modules/n8n/node_modules/n8n-core/src/WorkflowExecute.ts:949:29

Can you share the input of the execute command node? My initial thought is maybe you are missing the command and just have the file path in there.

I’ve searched online about this error: Permission denied

also used the terminal for allow permission, but doesn’t work.
Same error