Help with pagination / looping

Hello everyone,

Just discovered n8n (just last week!!) and I am trying to see if I can create few workflows to help automate and make my job easier. I have gone thru two other similar questions that came up in search results, but didnt seem to fit what I want. I wanted to check with you all on what the best approach would be. (Obviously a total noob to n8n, sorry if this is covered elsewhere).

I have a simple internal api that takes a “page” param as part of the url, and the json response has a “currentPage”, “totalPages” in the json response.

I would like to start with 0 as the initial page and based on the “currentPage”, “totalPages” , keep fetching data until “currentPage” == “totalPages”.

So far I am thinking if using a Set to keep track of pages is the best approach ? But I cant yet figure out how to keep incrementing page ?
Also would need to combine all the responses from all those pages.

Would appreciate a sample workflow so I can understand how Set / Loops / works. Also clueless about how to combine all the results.

url - http://testurl.com/?page=0

response -
{
currentPage:1,
totalPages:10,

}

Cheers !

I just implemented a work flow yesterday that has something similar. The incrementing was not working for me. I used another variable and just checked if there were items in the pagination call.

Thanks @cgsmith ! I can probably follow the same approach.
Does it mean the values in a Set cannot be updated as part of the flow ?

Do you have a save to db (or file) node between the http node and the IF node, to save the results from http response ?
Is there a efficient way of combining the results from all the http responses ?

I just solved this last week for a client & thought it would make a nice quick tip for my youtube channel. Nodemation Deep Dive: Looping & Pagination of HTTP Requests - YouTube
Let me know if that is helpful.

Mock API Workflow:

Actual Pagination:

2 Likes

@johnnyk - that is incredible ! Looks like exactly what I am looking for.
Just looked at your video too, and seems like you have solved it for me !

Really appreciate the help, Cheers !

1 Like

Your welcome!