AWS S3 node: REST vs. SOAP

steven.n.t · November 22, 2022, 2:24pm

Describe the issue/error/question

I ran into a problem with the AWS S3 node: The bucket is accessible (list file, upload file…) using AWS CLI, and Python code etc. properly, but N8N AWS S3 node failed.

Looking at the error message, and looking at the AWS S3 documentation, I think the problem could be SOAP.

Is there a way to ask this AWS S3 node to use REST rather than SOAP? -

github.com

n8n-io/n8n/blob/master/packages/nodes-base/nodes/Aws/S3/AwsS3.node.ts

import { paramCase, snakeCase } from 'change-case';

import { createHash } from 'crypto';

import { Builder } from 'xml2js';

import { IExecuteFunctions } from 'n8n-core';

import {
	IBinaryKeyData,
	IDataObject,
	INodeExecutionData,
	INodeType,
	INodeTypeDescription,
	JsonObject,
	NodeOperationError,
} from 'n8n-workflow';

import { bucketFields, bucketOperations } from './BucketDescription';

This file has been truncated. show original

What is the error message (if any)?

ERROR: UNKNOWN ERROR - check the detailed error for more information

AccessDeniedAccess DeniedP0TK6PCYE27XW07PecUT5STV6B6qQZUktkF9eG8/EY0A5tCti//Kibz2gddSUGsCVsg8bSVPuY+gIq1p5g8TpsoJzkc=

Details

Time

22/11/2022, 21:20:01

Stack

NodeApiError: UNKNOWN ERROR - check the detailed error for more information
    at Object.awsApiRequest (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/GenericFunctions.js:27:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.awsApiRequestSOAP (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/GenericFunctions.js:42:22)
    at async Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/Aws/S3/AwsS3.node.js:249:40)
    at async Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:658:28)
    at async /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:585:53

Please share the workflow

(Select the nodes and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow respectively)

Share the output returned by the last node

Information on your n8n setup

n8n version: 0.202.1
Database you’re using (default: SQLite):
Running n8n with the execution process [own(default), main]: own
Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker

MutedJam · November 24, 2022, 8:41am

Hi @steven.n.t, the only way to change the requests made a node would be to change the respective node itself.

However, if you want to make a REST request you could consider using the generic HTTP Request node.

That said, I just gave the AWS S3 node a go on my end and didn’t run into trouble when listing files for example:

This was my workflow (running on the latest n8n version 0.203.1):

Perhaps you can share some more details on what exactly you were doing when running into the problem? How does your workflow look like and do your AWS credentials work in other n8n nodes?

steven.n.t · November 24, 2022, 2:20pm

Hi @MutedJam,

My experiment was simple: just one AWS S3 node calling an S3 bucket to list the files.

My situation is: there is an existing client S3 bucket that my app need to upload data to. That S3 bucket is set up in a way that doesn’t allow SOAP. Reason: SOAP is deprecated. - Amazon S3 REST API Introduction - Amazon Simple Storage Service

The default N8N AWS S3 node failed to upload to that bucket. Then I looked into the implementation of that node on Github and it seems that SOAP is the default. Therefore, it fails. It’s the mix bag of this client S3 bucket not allowing SOAP, and the N8N S3 node only uses SOAP by default.

That’s why it will be nice for that N8N S3 node to give a choice whether it calls S3 on SOAP or REST.

MutedJam · November 28, 2022, 1:41pm

Hi @steven.n.t, thanks for clarifying! Tbh, I am not sure about our future plans with regards to this node (maybe @sirdavidoff can share more information here), but I shall add this to our engineering backlog for a closer look.

Jon · December 6, 2022, 4:01pm

Hey @steven.n.t,

I was just looking into this and seeing if we can provide a more useful error message but when checking the S3 documentation and the settings in the UI I can’t actually see an option to disable SOAP API access to a bucket, Do you happen to know where this setting is or what the ACL name is so I can reproduce the problem?

From what I can see on the Amazon side though at the moment SOAP is still useable over HTTPS it just won’t be getting new features so we should really look at updating it but for now I want to work on displaying a notice to let users know that it could be because SOAP is not enabled for the bucket.

system · March 6, 2023, 4:02pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

jan · June 22, 2023, 2:56pm

New version [email protected] got released which includes the GitHub PR 6017.