Pagination Using Offset Parameter with HTTP Node

Questions
#1

I have an HTTP node for a service that only returns 100 records at a time but can take an offset parameter to paginate through results. I’ve read through several posts that talk about using IF loops to retrieve all results using pagination, but am unsure how to make it work for my workflow. Anyone have some insight on how to do this? I know most nodes for specific services do this automatically, but the HTTP node doesn’t seem to have an option for Return All results like the others. Thanks in advance for any assistance you can give!

Here’s my workflow:

Screen Shot 2021-02-09 at 9.45.45 PM
Screen Shot 2021-02-09 at 9.45.45 PM2442×572 80.8 KB

#2

Check in the if node if there is an offset and connect the false output to Get All Adastack Users. Once all users are retrieved, you have to use a function node to merge all the iterations.

I did something similar for Hubspot here Hubspot node 'get all contacts': only subset of max 300 entries

#3

Awesome, I’m looking into this now. If you don’t mind me asking, what is the purpose in the example workflow of the No Op and Wait nodes in this case? To prevent timeout?

#4

To prevent to hit the API’s rate limit.