Troubles with pagination

Questions
1

Hello,

I’m having hard time with the pagination of HTTP request component.
There are some options in this node which is great, but I’m unable to make it work.

I’m facing the message “The returned response was identical 5x, so requests got stopped” and I don’t know what to do.

I want to use the feature " Pagination Complete When" with “Other” and “Complete Expression”.

Let’s get this API, which gives a similar return of the one I’m working with ( this one needs no authentication so it’s more convenient to share it ) :
https://pokeapi.co/api/v2/pokemon :

  • Status code 200, so I can’t use the option checking the status code
  • I would use “the response is empty” because “reults” is empty but n8n doesn’t know this path contains the data, so it doesn’t work, that’s fine.
  • The array that contains items is just empty. This is what I want to check to define if all pages have been paginated.

Here is my “workflow” :

The $response autocompletion is showing the right fields :

image
image360×175 3.76 KB

I’ve tried

{{ $response.body.results.length === 0  }}
// ==> The returned response was identical 5x, so requests got stopped
{{ $response.body.results.length > 0 }}
// => No error, but just one page if fetched. 
{{ $response.body.results[0]?.name == "undefined"  }}
// ==> The returned response was identical 5x, so requests got stopped

No matter what, I see just one call in the console, and the best I have is finishing with just one page of 60 records.

In the console, we can’t see the response, pageCount, body. So it’s pretty hard to know what’s happening. We can’t use console.log in this context.
I was looking for examples of how am I supposed to use this feature, I’ve found some tests here :

but the only ocurrence is

"paginationCompleteWhen": "other",
"completeExpression": "={{ $response.statusCode === 404 }}"

I’ve also tried to =make it “the old way”, with loops in old posts and all, but lot of things have changed and it didn’t work for me.

I’m probably missing something big, because it’s a very common problem, and I’m surprised it’s that hard to solve it. Unless there’s a bug.

Thanks for your help

Information on your n8n setup

  • n8n version: [email protected] Beta
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Cloud
  • Operating system: Windows 11
2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Requested info :

  • n8n version: 1.76.1 beta
  • Database (default: SQLite): default
  • n8n EXECUTIONS_PROCESS setting (default: own, main): : I don’t know what it is / how to find it, probably default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): : n8n cloud
  • Operating system: : Windows 11
4

Have you tried this with any other websites?

5

Hello,

Yes, I had the same issue with 2 APIs : The one I was initially working with, and Pokémon API described in my first post.

I haven’t tried again, I gave up.
I’ve built my workflow differently, in an under-optimized but working way.

Instead of getting all the results of “Entity B” once, and picking the right value when needed to enrich dataset of “Entity A”, I’m using a loop to iterate on Entity A, item by item, and I call the API of Entity B as many times as I have records of Entity A. I’ve also added a wait node to respect the API quota limitations

6

The response contains a ‘next’ url:

image

Now just use that, and end the pagination when there’s no more ‘next’ value:

image

Here’s a workflow that’ll do it:

7

Hi,

Thanks for your time,
I’m sorry, the API doesn’t include a “next” parameter. I should have been more careful when searching for a suitable API.

Here is the official documentation about pagination :

Paging is handled through 2 query parameters: ?limit={page_size} and ?page={page_number} . For example, to retrieve the first page of 100 items: ?limit=100&page=1 .

Retrieve the total number of items across all pages through ?fields.root=count .

You can’t work with this API without an account but here is the link through their doc :
v3 VS v4 - Lucca API ( v4 )

I tried this without success :

image
image364×665 8.42 KB

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.