HTTP Pagination Mode in n8n

Hello everyone, this is my old pagination process. In the new pagination process in n8n’s http, I have seen the Pagination Mode feature. How can I switch from the old method to the new method using the Pagination Mode feature?

“pageSize”: int?, // number of items per page, default is 20 items, maximum is 100 items
“currentItem”: int, // retrieve data from the current record, if not entered, default is 0

Hey @tienhuy,

Without more details on the API you are using the below is the best I can come up with, You will just need to work out what to use for the “Pagination Complete When” option.

I also had some issues getting to understand it in the first place, but checking my API response header and footer data it was clear and mostly I get through with the following:

Sometimes you need to activate “Include Response Headers and Status” on your node to get the next page URL if using “Response contains next url” instead of the above example.

Thank you very much, it works perfectly.

I use the paid plan, and I don’t see the pagination option in the HTTP request node. I see Batching and it works great. Thanks

@davidl in that case you have to upgrade to a recent version.

I am working through search results of a HubSpot custom object and am given paging data in the Output area as shown in the image. How would I set that up in the paging option of the HTTP Request node?

image

Here is what HubSpot provides us about how to use the paging. CRM API | Search

Hey @davidl,

The example at the bottom of this post has worked for me. For the pagination we set a body parameter called after and set the value to use an expression of {{ $response.body.paging.next.after }} we then tell the pagination option that there are no more items when the after option is not there by setting Complete When > Other and the expression to {{ !$response.body.paging.next.after }} this will give something like…

2 Likes

@Jon

This works great. HubSpot has a rate limit of four search requests per second, so I need to inject a delay after each page or 4 pages. Any thoughts on how to add that to the https request?

Batching option in the http node :wink:

image

Already tried that. The batching option does not work with this type of pagination.

That sounds like we should probably allow for a delay in the pagination.

2 Likes

That would be super cool. :sunglasses:

Let me know when there is an update to the node.

1 Like

@Jon Thanks for all your help.

Since there is not a wait option in the http request pagination, how do I create a loop that includes a wait and feeds the Next pagination value into the http request? I know how to loop with a delay when a url is in the response value but don’t know how to do it using a pagination number value.

image