S3 nodes return incorrect data and HTTP tool not working

[Gleeb]

Describe the problem/error/question

I am trying to build a simple solution that will be able to list/read/download files from S3.

Firstly I tried:
AWS S3 Tool - The tool refuses to see folder that were not created by the console.
the only way to see folders is to go to the console and hit the “create folder” button.
Any that you upload by dragging in a folder or uploading via aws cli or any other method will produce blank. no idea why no idea how to solve this.

So I move to the HTTP node - well, that worked… but when i tried to create a tool with the exact same parameters. I get an error:

{
  "response": "HTTP ERR_INVALID_URL There was an error: \"Invalid URL\""
}

Ok sure, After I tried everything i could i just decided to give https://www.google.com , also getting the same error… this is broken!

So the last option that I have is to start coding it with boto3 in python or AWS CLI or something which is just ridiculous

what is actually happening here. are the nodes broken, is there some trick that i am not aware of? I am getting to the point where N8N is becoming unusable and I will need to explore other solutions…

Also note that the credentials work, they work in some cases with the nodes (but the data is returned missing) and it works when I try the credentials directly with aws cli (which returns the correct data).

Thanks

Information on your n8n setup

• n8n version: 1.80.3
• Database (default: SQLite): SQlite
• n8n EXECUTIONS_PROCESS setting (default: own, main):?
• Running n8n via (Docker, npm, n8n cloud, desktop app): kubernetes in AWS
• Operating system:Linux

[Knight]

Hi @Gleeb,

I totally get the frustration. Even though your credentials work with AWS CLI, it’s worth double-checking that the IAM role you’re using has the correct permissions to access all parts of your S3 bucket (e.g., s3:ListBucket, s3:GetObject). Sometimes permissions can vary depending on how you’re accessing it (console vs. CLI vs. n8n).

Let me know how it goes!

[Gleeb]

Hi knight, thanks for the input.
I did check the credentials and they cant get any better than this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

I am still having this issue, i have exhusted all options, both with the custom http node and the aws s3 node. i am trying to go with just coding it with the code node

I dont understand what it means that “permissions can vary”, n8n cant assume pod roles from aws so the only option is a key/secret, if it works it works, and it works in cli, boto, and any other method i try outside n8n

For the same credentials:
S3 node returns partial response
http node returns full response
http tool node returns error on any url