Issue - Loop - HTTP Request - ID #

Questions
, ,
1

Greetings,

I am trying to create a flow that is basically will do a page scrape of a website.

For example:
I can do a single page without issues using the following:

It goes simply to the web page via HTTP Request (Directory) and then change the last number to get different page

Trying to do it in a loop.

So far I did the setup using for sure SplitInBatch + Excel sheet for the numbering

It does the loop, however it outputs only the same page, not sure what is missing to get the counter of the page working in the loop of the HTTP Request.

Any help would be appreciated.

2

Hey ansread,
in your Http Request node, instead of using this expression

image
image1036Ă—118 11.4 KB

could you try using this one

image
image754Ă—64 6.75 KB

Also, does it work without using SplitInBatches? The HttpRequest node should go through all input input items by itself.

1 Like
3

Thanks @marcus for the response.

I tried the {{json["id"]}} instead, still didn’t work.

I tried without the SplitInBatches, and it kept looping!

What I noticed that the SplitInBatches is working fine, and the counter is working as you can see from the left side in the screenshot, result 3 shows Id 3 and so on.

Screen Shot 2022-10-05 at 9.51.55 AM
Screen Shot 2022-10-05 at 9.51.55 AM1705Ă—685 90.5 KB

However, the http request node doesn’t get updated with that input for some reason.

4

Update:

I had to click on the current node and not to type manually the {{json[“id”]}}

With that it worked! Strange, but it worked!

Any idea why it doesn’t work when you type it manually even if it’s in the right syntax?

Thanks @marcus

5

Hey ansred,
glad to hear it worked. When you are in the expression editor both methods, clicking incoming data on the left menu or typing expressions manually, should work. I can’t really tell why it did not work for you.

Now that you aren’t using SplitInBatches anymore I think you can remove your loop. The Http Request node should go through all incoming Google Sheets items.

1 Like
6

Hey @marcus you are right!

The loop was removed and removed the SplitInBatches and it worked!

This time it worked and dumped all the data at once in the result instead of going one by one (interesting)

Thanks for the great help!!

Here is the final config

1 Like