How to fetch a file from a URL

[ThatPrivacyGuy]

Describe the problem/error/question

I have a node which reads a csv file and builds an array of json objects each containing a url and a tType.

I need to use this array of objects to check the tType value and then I need to grab the file from the URL if the tType value equals “JavaScript”.

I have successfully extracted the array of objects and passed them as output and I can act on those objects using a for loop node - but I am not sure how to then grab the file from the url in the for loop.

What is the error message (if any)?

N/A

Please share your workflow

for (const item of $input.all()) {
  if (item.json.tType === 'JavaScript') {
    << Here I need to grab the file in the URL (they are all javascript files) and save them to a specific folder as they will be fed to an AI agent in the next part of the workflow >>
  }
}

I also need to return something (the something will be the paths for the saved javascript files as an array which can be used for input for the AI Agent)

Share the output returned by the last node

[
{
"scriptSrc": 
"https://example.com/",
"tType": 
"First Party Cookie"
},
{
"scriptSrc": 
"https://example.com/",
"tType": 
"First Party Cookie"
},
{
"scriptSrc": 
"https://www.example.com/wp-content/cache/autoptimize/js/autoptimize_single_02af490eacf772c17836ddf05c95bee7.js",
"tType": 
"JavaScript"
}
]

Information on your n8n setup

• n8n version: 1.61.0
• Database (default: SQLite): SQLite
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app): npm
• Operating system: MacOS

[ThatPrivacyGuy]

@ThatPrivacyGuy

for (const item of $input.all()) {
  if (item.json.tType === 'JavaScript') {
    << Here I need to grab the file in the URL (they are all javascript files) and save them to a specific folder as they will be fed to an AI agent in the next part of the workflow >>
  }
}

So I have managed to build the array now and return it as output:

const urls = [];
for (const item of $input.all()) {
  if (item.json.tType === 'JavaScript') {
    urls.push(item.json.scriptSrc);
  }
}
const jsonObjects = urls.map(url => ({ scriptUrl: url }));

return jsonObjects;

Now I need to figure out how to get the javascript files saved locally…

[ihortom]

Hey @ThatPrivacyGuy , do you already have the files saved locally or you still need to download them? Looking at your items I can see some external URL to JS script files.

Do you mean to use those URL to download the files? If so, it is done with HTTP Request node in GET mode asuming the URL is the locator of the file and not just the web page address.

[ThatPrivacyGuy]

It’s ok, I got it figured out

Will post solution after weekend.

[Bojan]

I see you already solved it, was a bit delayed posting the following proposal. Nice work battling through and figuring it out