Collect data from workflow (loop)

Describe the issue/error/question

my work flow in this example is fetching pages to scrape, im trying to think on a way to collect all the data to one list, for now i need to add the data to a database on every page, instead of running the scrape process and when finish to send all

Please share the workflow

try to run it you will see loop with 3 entries for 3 pages, how can i collect the data and send once in the end?

just an overview on this project, im creating npmjs automation and a dashboard using appsmith

Information on your n8n setup

  • n8n version:latest
  • Database: postgres
  • Running n8n with the execution process [own(default), main]:
  • Running n8n via [Docker]:

Hi @Nir_Adler ,

I hope this is a solution for you

The needed function:

// Get allways the last runIndex ...
// Needed because $items('HTML Extract', 0, i) throw an error if 'i' is not a runIndex
const maxRunIndex = $node["HTML Extract"].runIndex;

let packages = [];

let i = 0;
while(i <= maxRunIndex) {
  // Get the special attribute ...
  const _packages = $items('HTML Extract', 0, i)[0].json.packages
  packages = packages.concat(_packages)

// Map it for n8n B-)
return => { return {json: {package: e}}});

amazing! i tried to do similar solution but i missed this option to go back to previous run

$items('HTML Extract', 0, i)