AWS S3 region call step prevening upload

[Nurgak]

Describe the issue/error/question

I am attempting to use the AWS S3 node to upload some objects. The request fails.

I tried to roll my own node to upload files to S3 and I was successful. Then I went back to investigate the n8n AWS S3 node to understand why it failed.

What is the error message (if any)?

The error message is the following:

NodeApiError: Forbidden - perhaps check your credentials?
    at Object.requestWithAuthentication (/usr/local/lib/node_modules/n8n/node_modules/n8n-core/src/NodeExecuteFunctions.ts:1571:10)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.awsApiRequestSOAP (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Aws/S3/GenericFunctions.ts:85:19)
    at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/nodes/Aws/S3/AwsS3.node.ts:799:22)
    at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/src/Workflow.ts:1270:19)
    at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/src/WorkflowExecute.ts:939:29

Line 799 in the AWS S3 node is a call to define the region of the upload. It seems the way my AWS account is set up (over which I do not have control) is not allowed to make this particular GET request.

However, this call does not seem to be needed as I could upload my file with my custom node that did not have this call.

The region can be taken from the AWS credentials (that’s what I did in my node), so why exactly is there an additional call to define the upload region? Perhaps there is something on AWS side where you restrict the region based on the bucket? I have no use for this function personally, but this check prevents me from doing something that would otherwise work fine.

Although the issue is not in n8n (rather AWS account permissions configuration), might there be a solution that would allow me to use the official n8n AWS S3 node? For example: like many overriding additional fields, make the region call optional upon request, rather than mandatory?

Please share the workflow

Share the output returned by the last node

The output of the last node is the binary of a sample file.

Information on your n8n setup

• n8n version: 0.212.1
• Database you’re using: SQLite
• Running n8n with the execution process: own
• Running n8n via: Docker

[Nurgak]

@Nurgak

However, this call does not seem to be needed as I could upload my file with my custom node that did not have this call.

While looking into the code, it’s quite difficult to see where the final request URI is assembled. However, while looking at the S3 helper functions, it seems the region parameter is not even used… Did I overlook something?

If the region call is necessary, it seems there is a preferred updated way to do it via a HEAD call instead of GetBucketLocation. However, HeadBucket needs a different action: s3:ListBucket on the AWS side, GetBucketLocation needs s3:GetBucketLocation.

All that said, I could actually modify the S3 permissions for myself and have access to the data, solving this issue for me immediately. That might still be an issue for others and my question about the need for the region remains

[Jon]

Hey @Nurgak,

With the S3 node a lot of things are set in the credential file as well like using the region and the signing of the request. What is odd is that call to get the location of the bucket to set the region which does seem a bit odd.

I know we are planning to redo the S3 node shortly which may help with this but for now changing the permissions as you have done is probably the quickest way to get up and running.

[Nurgak]

Hi @Jon

Thank you for your reply.

From what I’ve understood, you have to define in what region your bucket resides in the URI, which can be different from your AWS account’s region. Therefore, this call is justified and it does not need the region in the URI. However, it might be restrictive as you may not have the permissions to call it, as I did, so it would make sense to allow disabling it from the additional fields and use the user account region instead.

When uploading, you’re supposed to use the region in the upload URI, which does not seem to be the case in n8n (the _region parameter does not appear to be used in the call)… but the call still succeeds as this is considered a legacy way of doing it (see Legacy global endpoint).

I guess this would needs addressing in the redo of the S3 node.