HTTP Pagination with Parameter Update in Array?

Questions
,
1

Describe the problem/error/question

I’m trying to implement the HTTP pagination for a request.
I successfully used the pagination in other APIs, but in this one I have to give filtering parameters in a JSON array. One element in this array is the “page counter” I have to increment.

Here is an example of the JSON body:

{
	"ServiceRequest": {
		"preferences": {
			"limitResults": 10
		},
		"filters": {
			"Criteria": [
				{
					"field": "id",
					"operator": "GREATER",
					"value": {{ $json["lastId"] }}
				},
				{
					"field": "trackingMethod",
					"operator": "EQUALS",
					"value": "QAGENT"
				},
				{
					"field": "tagName",
					"operator": "EQUALS",
					"value": "{{ $json.customer_tag }}"
				}
			]
		}
	}
}

The value that needs to be updated is in the first Criteria element field value.

I tried to use the pagination mode “Update a Parameter in Each Request” with type “Body” and name ServiceRequest.filters.Criteria[0].value, but this does not seem to work.
I also tried ServiceRequest.filters.Criteria.0.value and other variations but nothing works.

Does anyone have an idea how to solve this?

2

Hello @mnebel

You can omit built-in pagination and use HTTP node with custom logic for pagination, e.g.

The data table part is optional

1 Like
3

Hi @barn4k,

thank you for your reply!

I had it working with some custom logic, but I wanted to have it as simple and stable as possible, and thus want to use the built-in functionality.

I’m also pretty sure that I’m near to a solution. The question is, if this is a known issue or if I’m using the wrong syntaxt for the update field?

Best regards

4

Hi @mnebel!
From what I can see, this isn’t a bug and it’s not a syntax issue either. The Update a Parameter in Each Request pagination mode in the HTTP Request node is mainly meant for simple parameters like query params or top-level body fields, as described in the n8n pagination docs. It’s not really built to safely update deeply nested values inside indexed JSON arrays like Criteria[0].value. In cases like yours, I’ve found it more reliable to control the pagination explicitly using expressions in the body together with a small loop pattern (for example with Set + IF or Loop Over Items). The HTTP node is behaving as designed, it’s just that this specific helper isn’t the right fit for complex nested array updates.

1 Like
5

Hi @tamy.santos,

thanks for your reply!

What bothers me about this solution is that once pagination is complete, you have to merge all the results from the executed HTTP node iterations back together. While it works, it still feels like a rather hacky workaround to me, especially considering that the HTTP node already provides this functionality in the first place.

Would it be very complicated to incorporate support for setting nested JSON fields directly into the HTTP pagination functionality?

6

It’s not hacky, it’s less obvious. However, explicit pagination allows you to work with batches independently without overloading any dependent services. By using the sub workflows, you can even process hundreds of thousands of items, which is not possible with the built-in pagination.

You can submit a feature request here:
Latest Feature Requests topics - n8n Community

1 Like
7

I don’t know the answer to that, but I’ll look for more information. If you find out anything, please share it as well @mnebel :slight_smile:

1 Like