HTTP batch get.list - loop and merge through runs. (newbie question)

Describe the problem/error/question

What is the error message (if any)?

Please share your workflow

I am coming from and am amazed by n8n. Automating is becoming my life and with n8n I hope I will learn lots of new stuff. Unfortunately make had some features out of the box, where I need to learn getting same effects on n8n. I am just at the beginning so please be understanding. I need to change the way of thinking into more professional.

Could anyone help me merging batches of data. I am able to list only 50 results at one http request with the information which was the last item number. To get the rest I need to include “start”:{value of “next”} from the last batch of data in the request. This is how one batch looks like

    "result": [
        "ID": "97833",
        "TITLE": "deal title",
        "TYPE_ID": "SALE",
        "STAGE_ID": "NEW",
        "PROBABILITY": "48",
        "CURRENCY_ID": "PLN",
        "OPPORTUNITY": "0.00",
        "TAX_VALUE": null,
        "LEAD_ID": "71603",
        "COMPANY_ID": "0",
        "CONTACT_ID": "66603",
        "QUOTE_ID": null,
        "BEGINDATE": "2024-02-03T03:00:00+03:00",
        "CLOSEDATE": "2024-02-10T03:00:00+03:00",
        "ASSIGNED_BY_ID": "19",
        "CREATED_BY_ID": "1",
        "MODIFY_BY_ID": "1",
        "DATE_CREATE": "2024-02-03T12:31:52+03:00",
        "DATE_MODIFY": "2024-02-03T12:35:18+03:00",
        "OPENED": "N",
        "CLOSED": "N",
        "COMMENTS": null,
        "ADDITIONAL_INFO": null,
        "LOCATION_ID": null,
        "CATEGORY_ID": "0",
        "STAGE_SEMANTIC_ID": "P",
        "IS_NEW": "Y",
        "IS_RECURRING": "N",
        "IS_RETURN_CUSTOMER": "N",
        "SOURCE_ID": "3",
        "SOURCE_DESCRIPTION": "x",
        "ORIGINATOR_ID": null,
        "ORIGIN_ID": null,
        "MOVED_BY_ID": "1",
        "MOVED_TIME": "2024-02-03T12:31:52+03:00",
        "LAST_ACTIVITY_TIME": "2024-02-03T12:31:52+03:00",
        "UTM_SOURCE": null,
        "UTM_MEDIUM": null,
        "UTM_CAMPAIGN": null,
        "UTM_CONTENT": null,
        "UTM_TERM": null,
        "LAST_ACTIVITY_BY": "1"
   "next": 50,
   "total": 1402,
    "time": {
      "start": 1706954368.239976,
      "finish": 1706954368.284911,
      "duration": 0.044934988021850586,
      "processing": 0.023025989532470703,
      "date_start": "2024-02-03T12:59:28+03:00",
      "date_finish": "2024-02-03T12:59:28+03:00",
      "operating_reset_at": 1706954968,
      "operating": 0.778087854385376

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Welcome to the community @Kamil_Muras ! Great to have you with us!

Did you check out the pagination functionally? The docs can be found here:

1 Like

Thank you, hopefully I will ask less stupid questions in the future ^^

Glad to hear that it was helpful. There are no stupid questions. It is normally a sign that we have to improve things and make it clearer and easier discoverable for starters.

Have fun!

This means that my newbie attitude is valuable here ^^. What I have noticed - the easiest way of learning new things is looking at working examples. I have read the pagination documentation but I lack some knowledge that would let me jump straight into it, so I am looking at YouTube videos which , what I assume on the basis of n8n development pace , might be little outdated. All of this makes learning little too long because I am still learning pagination ^^. Hopefully it will help.


Unfortunatelly I am not able to get through this, anyone could help me and show example how to setup this scenario ?
My main issue is how to get the data from response because I see no input that I could process or put into pagination setting. This is the point where I am right now:

Hay @Kamil_Muras What the response you are getting or error in this case?

is response the same as output? I dont know where should I look for response

When you start this node what the output or response do you have right now with all your last settings?

I meant, when you implemented pagination in node you’ve show in a post above?

Does {{$response.body["0"].next}} shows the Value of 50?

According to the specifications provided by your API provider, you should include it in your subsequent request as a ‘start’ parameter. For more details, please refer to:

1 Like

Now the result was I had 3 identical runs with same data because I have limited pagination runs to 3
I think I start to understand what is going on but not sure.
I think I have issue with putting start value into the query or getting response value.
I dont know how to check what value does " `{{$response.body[“0”].next}} " return.
RIght now it looks like this because query as JSON is the only working one:
And I dont know how to change query from json to fields so instead of this: { “order”:{ “STAGE_ID”:“ASC” }, “filter”:{ “ASSIGNED_BY_ID”:17 }, “select”:“*”} I could use fields as query parameters. I started a new topic here: Converting JSON query to fields in HTTP request

I think It has strong connection in the way of using pagination and putting the “next” value in “start” query parameter.

This is a screenshot from browser developer, I think there is no “start” qery value

Hey Ruslan, thank you for your answer about fields, but it’s not very connected with the topic. Would you be so kind and give an answer on another topic that I have created so I can credit you and close the topic. Link: Converting JSON query to fields in HTTP request

Finally I managed to work it out. Thank you everyone, and am very sorry for being such a noob.

@jan : Mistakes that I have made:

  1. Was using query parameters as Fixed instead of Expression ( bad habit from, which was simplifying many things)
  2. Did not understand that I can fetch some data from response body by this: {{$}} - additionally I didn’t know I need to use “Expression” also. Similary I was looking for “” click and drag - need to learn some coding.

Once again thank you very much, I promise I will add some value in the community once I get used to everything.


Amazing, and thanks a lot @Kamil_Muras for sharing your experience. It is not just helpful for other users, also for us to know where we should improve product and/or documentation.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.