Issue with Dynamic URLs in HTTP Request Node

Hello n8n Community,

I’ve been encountering an issue with the HTTP Request node where it seems to treat my dynamic URLs differently than when I input the URL directly. The dynamic URLs result in an error, while the exact same URLs inputted manually lead to valid results.

I can reproduce this behavior unlimited times. So it doesn’t seem to be caused by limitations or rules on the responding server.

A user was reporting similar behavior in this threat: Issue with Dynamic HTTP Request - #4 by Lellis

He mentioned that using a set node one step before, worked in his case. In my case, it didn’t change the outcome.

Here’s what I’m trying to do: I have a workflow that extracts several URLs using JavaScript code and stores them in an array called URLS. I’m splitting the array out into single strings. I then want to use these URLs in an HTTP Request node to fetch data from the URLS.

When I use a static URL (in this specific example, "https://www.otto.de/partner-details/partner/1075392?asLayer=true") directly in the HTTP Request node, it works fine and I get the expected response. However, when I try to use a dynamic URL (in this case, {{ $json.URL}}) that contains the exact same URL, I get an error response from the API.

The error message is:

Item Index: 8
HTTP Code: 404
Stack:
NodeApiError: The resource you are requesting could not be found
at Object.execute (/usr/local/lib/node_modules/n8n/node_modules/n8n-nodes-base/dist/nodes/HttpRequest/V3/HttpRequestV3.node.js:1540:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at Workflow.runNode (/usr/local/lib/node_modules/n8n/node_modules/n8n-workflow/dist/Workflow.js:698:19)
at /usr/local/lib/node_modules/n8n/node_modules/n8n-core/dist/WorkflowExecute.js:656:53

Here are the steps I’ve taken:

  1. Extracted the URLs and confirmed they are correctly stored in the matches array.
  2. Used the HTTP Request node with a static URL – which works without any issues.
  3. Replaced the static URL with the dynamic variable from my matches array, which results in an error.
  4. Used a Set-Node before the HTTP Request, as suggested in this post

I’m including screenshots of my workflow configuration, especially the HTTP Request node setup and the error I receive.

Thank you in advance for any guidance or assistance you can provide!

Best regards, JayJay


It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
  • n8n version: 1.25.1
  • Database: SQLite
  • n8n EXECUTIONS_PROCESS default
  • Running n8n via 8n cloud

I have been running into similar problems for using:
{{ $json.VAR }}
It seems that sometimes the data isn’t there even if it should be.
So I always use this instead:
{{ $('my node name').all()[0].json.VAR }}
is uglier, is longer and it makes it harder to read, but it will always work even if you change node position, or add a node between them.

Even so when you hit the TEST STEP button, the data sometimes is not there, even if you executed previous steps. You have to run the entire workflow and it should work fine.

Hope it helps.

Thank you, Cristian! It works in combination with the Set-Node one step before!
I really appreciate your help!

Best regards,
JayJay

Hey Cristian,

what would be your approach, when you are dealing with multiple URLs?:

{{ $(‘my node name’).all()[0].json.VAR }}

always returns the first URL. I tried to adjust it to:

{{$(‘my node name’).all()[$itemIndex].json.VAR}}

But then the same error pops up again (see initial message).

Best regards,
JayJay

The way I handle it is to use {{ $json.URL }} which should work, I suspect you are seeing that issue because one of the 9 input items has an invalid URL, If you enable the option to output the full response and to never fail you might be able to find out which one is causing the issue.

You are absolutely right, my friend.
Thank you so much for pointing that out!!

1 Like

@jayjay2 also keep in mind the following:

all()[0]

must always be with index 0.

And imagine you have an Array in the previous nodes that you need to extract data from.
This part is just to reach the data inside the previous node:
$(‘my node name’).all()[0].json.
this means that the json Object is holding all the data that you see in the Output by that previous node.
This means that:
$(‘my node name’).all()[0].json.
will always be unchanged.

Depending on the type of node you are getting the data from, there will be another Object named body or data or if it is a Code node it will have the Keys that you set. Just see the Output of the Node that you need to get the data from and you will see the Object name that you need to access.

For example if you have a Node of type Code and you return something similar too:

myObj = {};
myObj['key1'] = "my data";
myObj['key2'] = "my other data";
myObj['key3'] = "my third data";

return myObj;

You would reach that data simply by doing:

$(‘my node name’).all()[0].json.key1
$(‘my node name’).all()[0].json.key2
$(‘my node name’).all()[0].json.key3

Took me some time to figure out how to do this and for a beginner it can be tricky.
Hope it helps.

1 Like

Hi Cristian,

Thank you for sharing this method. I appreciate it! I recently started using N8N and welcome any tips to improve my code efficiency. :slightly_smiling_face: :+1:

Best regards,
jayjay

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.