ERROR: Cannot create a string longer than 0x1fffffe8 characters

miguelraatz · January 17, 2024, 8:35pm

I’m trying to get open data from a website and insert it into my database, the size of the extracted file is 1.19 gb

NodeOperationError: Cannot create a string longer than 0x1fffffe8 characters
    at Object.execute (C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-nodes-base\nodes\SpreadsheetFile\v2\SpreadsheetFileV2.node.ts:216:12)
    at Workflow.runNode (C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-workflow\src\Workflow.ts:1307:31)
    at C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-core\src\WorkflowExecute.ts:1048:44

Information on your n8n setup

n8n version: 1.15.2
Database (default: SQLite): MySql
n8n EXECUTIONS_PROCESS setting (default: own, main):
Running n8n via (Docker, npm, n8n cloud, desktop app): npm
Operating system: Win 10 pro

jan · January 17, 2024, 9:36pm

Welcome to the community @miguelraatz !

Set the binary mode to filesystem. That will fix that issue. More information here: Scaling binary data in n8n | n8n Docs

miguelraatz · January 18, 2024, 11:29am

Hi @jan, thanks for the reply. Where should I change the variable? In the operating system or in the n8n itself? sorry I’m a beginner.

miguelraatz · January 18, 2024, 11:35am

Another thing, if it’s on n8n, is it only available with the enterprise plan?

Jon · January 18, 2024, 12:24pm

Hey @miguelraatz,

It is an environment variable that can be set on all n8n versions. As you are using NPM you would need to set it before you run the npm command unless you are using pm2 inwhich case you can add it to the the ecosystem file.

miguelraatz · January 18, 2024, 12:30pm

Hi @Jon, What should I do in case I have already used the npm command to install?

Jon · January 18, 2024, 1:53pm

Hey @miguelraatz,

Set the environment variable as you normally would, Assuming you have set the webhook_url it would be the same as that. One way you can do this would be to run export N8N_DEFAULT_BINARY_DATA_MODE=filesystem then run n8n start but this can vary depending on your operating system of choice.

I would recommend using docker though as it makes this a alot easier.

miguelraatz · January 18, 2024, 2:24pm

@Jon
set N8N_DEFAULT_BINARY_DATA_MODE=filesystem
n8n start
it works?

Jon · January 18, 2024, 2:35pm

@miguelraatz I think set could be the windows version so that should work.

miguelraatz · January 18, 2024, 6:39pm

@Jon I followed the steps you said, but I’m not making the same mistake.

ERROR: Cannot create a string longer than 0x1fffffe8 characters
 Details
Time
18/01/2024, 15:35:32
Item Index: 0
Stack
NodeOperationError: Cannot create a string longer than 0x1fffffe8 characters
    at Object.execute (C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-nodes-base\nodes\SpreadsheetFile\v2\SpreadsheetFileV2.node.ts:216:12)
    at Workflow.runNode (C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-workflow\src\Workflow.ts:1307:31)
    at C:\Users\ricardo.vicari\AppData\Roaming\npm\node_modules\n8n\node_modules\n8n-core\src\WorkflowExecute.ts:1048:44

It is taking 92 minutes to complete the request.

miguelraatz · January 18, 2024, 6:40pm

My Workflow

miguelraatz · January 18, 2024, 6:44pm

Jon · January 19, 2024, 8:23am

Hey @miguelraatz,

It could be that set isn’t the correct option for Windows then, I thought it was but I am not a Windows user so had to rely on your knowledge of the platform for that.

It could be that the Spreadsheet node doesn’t support the new binary data option as well, Can you provide the file you are using and the workflow so we can test?

miguelraatz · January 19, 2024, 11:25am

Hey @Jon
I make the HTTP Request through this link → https://dados.rfb.gov.br/CNPJ/Empresas0.zip
There are several, but I will send you just one as a reference.

miguelraatz · January 22, 2024, 11:25am

@Jon any answer?

Jon · January 23, 2024, 9:57am

Hey @miguelraatz,

Sorry for the delay I was looking into a couple of bugs, I will check this now.

miguelraatz · January 23, 2024, 12:28pm

Hey @Jon
I think I solved it using docker. But I have another problem with this node, do you have any idea what it could be?

miguelraatz · January 23, 2024, 12:30pm

@Jon maybe you can suggest me another type of node.

miguelraatz · January 23, 2024, 12:43pm

I need to save this data within the database.

Jon · January 23, 2024, 3:30pm

Hey @miguelraatz,

You need to change the delimeter in the Spreadsheet node to ; and change the format to CSV, I have tried running it but as the file is 1GB I don’t have a machine handy that can actually read it.