AWS S3 upload file failing with "Invalid array length"

[German_Bravo]

Describe the issue/error/question

When trying to upload a big file (less than 2GB because of buffer limit) with AWS S3 node, I get error “Invalid array length”. I thinks it is an error similar to Google Drive node that I described in another question:

File size (15344223093) is greater than 2 GB

Hi, i open this case because in my last workflow created, trying to read a binary file greater than 2GB in size to then upload it to a google drive share, i get the error “File size greater than 2GB”, I read that it is because of a limitation in Node.JS buffer, but is there any upgrade on this? In another post i saw the env configuration: N8N_DEFAULT_BINARY_DATA_MODE=filesystem Could it help in this case? I need to upload files larger than 2GB to Google Drive. Share the output returned by t…

What is the error message (if any)?

ERROR: UNKNOWN ERROR - check the detailed error for more information

Invalid array length

Stack
NodeApiError: UNKNOWN ERROR - check the detailed error for more information
    at Object.requestWithAuthentication (/usr/local/lib/node_modules/n8n/node_modules/n8n-core/src/NodeExecuteFunctions.ts:1554:10)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at Object.awsApiRequestSOAP (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Aws/S3/GenericFunctions.ts:85:19)
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Aws/S3/AwsS3.node.ts:837:23)
    at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/src/Workflow.ts:1270:19)
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/src/WorkflowExecute.ts:939:29

Please share the workflow

Share the output returned by the last node

Error from above…

Information on your n8n setup

• n8n version: 0.209.3
• Database you’re using (default: SQLite): Postgres
• Running n8n with the execution process [own(default), main]: main
• Running n8n via [Docker, npm, n8n.cloud, desktop app]: K8S queue mode

[MutedJam]

Hi @German_Bravo, thanks for reporting this! Perhaps @netroy can have a look at this one when he has a bit of time?

[German_Bravo]

Hi any update on this case? I achieved a workaround using the Command node with the aws-cli for s3, but it is only a workaround and I would like to do it from n8n directly.

Thanks a lot!

[netroy]

@German_Bravo I haven’t started working on this yet. as I was trying to get the S3 node to use the correct set of APIs before I can start implementing multipart upload.
I’m on vacation next week, but this task is on my radar, and will send you an update as soon as I have a PR for testing.

[German_Bravo]

Great news @netroy, i will stay on tune.

Happy vacations!