Workflow stops – how to fix loops in n8n?

Questions
, ,
1

Hi,
I built a workflow in n8n that:

  • takes a list of category URLs,

  • checks pagination and builds all pageUrls,

  • extracts all products with their URLs and names,

  • opens each product page to grab EAN + final product name,

  • writes everything row by row into Google Sheets (categoryUrl | pageUrl | productUrl | productName | ean).

I used several Loop nodes so everything runs one by one (categories → pages → products).

The problem: after processing 2 product links the workflow just stops (as shown on the screenshot) and I can’t figure out why.

Does anyone know how to fix this so the loop continues through all categories/pages/products?
Is it even possible to build such workflow?

Capture
Capture3136×518 178 KB

2

The loops need to know when to start the next cycle, for that we connect the end of the loop cycle back to the loop over items node, like so:

without this link, the loop over items node doesn’t loop.

3

Should it be like that? I’ve tried multiple versions and it seems that nothing works :neutral_face:
In this case wokrflow runs through all items in the first and second page from first link but after receiving another category link it stops.

Capture2
Capture22514×426 125 KB

4

No, not like that, the loopback needs to happen from the end of the loop. Think of it this way: every time the loop over items node takes one item from the input and passes it to the next node. The loopback needs to happen when you want the loop over items node to pick up the next value.

5

Chances are you don’t even need that many loops to begin with. If you share what it is exactly you are scraping, I can take a look.

6

For example these are links, I am trying to scrape:

I want to go through every link one by one, scrape Items from all pages (if they exist) also one by one and when there are no more items to scrape, I want to go to the next link and repeat the process.

7

Ok, take a look at this and tell me if I got this pretty close:

Sheet:

image
image1808×1160 433 KB

8

Yes you did, but was it done one by one? I used split into batches not to send so many request to HTTP.

9

If you have to, you can batch your requests in the HTTP Request node too:

image
image870×522 22.8 KB

10

That’s very helpful. Thanks to that I don’t have to use so many loops. Thank you very much. I think now I will be able to do it :slight_smile:

1 Like
11

Great! Kindly mark my answer as solution if it helped you solve this. Thanks!

Cheers!

2 Likes
12

This is great my friend.

1 Like
13

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.