AWS S3 node: REST vs. SOAP

Describe the issue/error/question

I ran into a problem with the AWS S3 node: The bucket is accessible (list file, upload file…) using AWS CLI, and Python code etc. properly, but N8N AWS S3 node failed.

Looking at the error message, and looking at the AWS S3 documentation, I think the problem could be SOAP.

Is there a way to ask this AWS S3 node to use REST rather than SOAP? -

What is the error message (if any)?

ERROR: UNKNOWN ERROR - check the detailed error for more information

AccessDeniedAccess DeniedP0TK6PCYE27XW07PecUT5STV6B6qQZUktkF9eG8/EY0A5tCti//Kibz2gddSUGsCVsg8bSVPuY+gIq1p5g8TpsoJzkc=

Details

Time

22/11/2022, 21:20:01

Stack

NodeApiError: UNKNOWN ERROR - check the detailed error for more information
    at Object.awsApiRequest (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/GenericFunctions.js:27:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.awsApiRequestSOAP (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/GenericFunctions.js:42:22)
    at async Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/AwsS3.node.js:249:40)
    at async Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:658:28)
    at async /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:585:53

Please share the workflow

(Select the nodes and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow respectively)

Share the output returned by the last node

Information on your n8n setup

  • n8n version: 0.202.1
  • Database you’re using (default: SQLite):
  • Running n8n with the execution process [own(default), main]: own
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker

Hi @steven.n.t, the only way to change the requests made a node would be to change the respective node itself.

However, if you want to make a REST request you could consider using the generic HTTP Request node.

That said, I just gave the AWS S3 node a go on my end and didn’t run into trouble when listing files for example:

This was my workflow (running on the latest n8n version 0.203.1):

Perhaps you can share some more details on what exactly you were doing when running into the problem? How does your workflow look like and do your AWS credentials work in other n8n nodes?

Hi @MutedJam,

My experiment was simple: just one AWS S3 node calling an S3 bucket to list the files.

My situation is: there is an existing client S3 bucket that my app need to upload data to. That S3 bucket is set up in a way that doesn’t allow SOAP. Reason: SOAP is deprecated. - Amazon S3 REST API Introduction - Amazon Simple Storage Service

The default N8N AWS S3 node failed to upload to that bucket. Then I looked into the implementation of that node on Github and it seems that SOAP is the default. Therefore, it fails. It’s the mix bag of this client S3 bucket not allowing SOAP, and the N8N S3 node only uses SOAP by default.

That’s why it will be nice for that N8N S3 node to give a choice whether it calls S3 on SOAP or REST.

Hi @steven.n.t, thanks for clarifying! Tbh, I am not sure about our future plans with regards to this node (maybe @sirdavidoff can share more information here), but I shall add this to our engineering backlog for a closer look.