HTTP Node pagination with problem

Questions
1

Describe the problem/error/question

I am trying to get data from an HTTP node using the pagination instruction of the documentation. However, when I try, it get error because it is returning response identical 5x, so requests got stopped.

Documentation on Swagger

What is the error message (if any)?

The returned response was identical 5x, so requests got stopped

Please share your workflow

Share the output returned by the last node

[
  {
    "page": 0,
    "pageSize": 101,
    "product": [
      "CONNECTONE",
      "CONNECTTWO",
      "ANOTHERCONNECTION",
          ],
    "dataCollected": []
  }
]

Information on your n8n setup

  • n8n version: 1.71.3
  • Node.js Version: -
    -Database: SQLite (default)
    -Execution mode: main (default)
2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
3

Welcome to the community @edilbertounbfga !

Tip for sharing information

Pasting your n8n workflow

Ensure to copy your n8n workflow and paste it in the code block, that is in between the pairs of triple backticks, which also could be achieved by clicking </> (preformatted text) in the editor and pasting in your workflow.

```
<your workflow>
```

That implies to any JSON output you would like to share with us.

Make sure that you have removed any sensitive information from your workflow and include dummy or pinned data with it!

The output suggests that the response is not empty. You would need to adjust the Completion condition. Does dataCollected represent the expected data you are after? If so the Completion logic could be modified to become {{ $response.body.dataCollected.isEmpty() }}

image
image810×272 17.1 KB

4

For example, in this endpoint, I know that 350 objects are returned, but I won’t always know how many objects will be returned.

I am trying to obtain them through pagination, but I am not able to:

  • Insert the obtained data into dataCollected
  • Change the pagination parameter to +1.

Thus, as pageSize=101 and the page parameter starts from 0, it would be necessary:

  • page =0
  • page =1
  • page =2
  • page =3

In each of these pages, the returned data should be saved in dataCollected.
When page=4 occurs, the request presents an error and I need to change

  • {{ $('Header Variables').item.json.product[0] }} to {{ $('Header Variables').item.json.product[1] }}
  • reset page again to 0 and repeat the process, saving data responses in dataCollected until page=n, where n is the pagination number that returns an error.

The process should end when {{ $('Header Variables').item.json.product[2] }} and for page = n, where n is a value that returns an error."

Here are a new piece of my workflow

5

You would need to split your products with Split Out node and Loop the pagination through those products. Visually it will look something like this

image
image1830×872 58.4 KB

If that happens then you’re likely doing something wrong. Error should not occur, you shouldn’t let it by utilizing the pagination technique properly.

6

Wow! It is almost done! However my pagination keeps not working

7

You need to check the error when HTTP Request node fails, it should give you a hint what is going wrong. Better yet, locate the API docs for the service you connect to and see what the pagination technique is used to figure out how to configure it correctly. Different APIs require different approach to get pagination working.

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.