I am having a workflow where we do a post request and we get id which in return need to call a one more api to get status of the request and this might take some time hence we add some delay and keep checking the status of the request and then we get data from the request once succeded here i want to add built in pagination. Since these start-query and get-query nodes are interconnected how can i add pagination here the max limit allowed to get is 1000 ? If it was a single api call then adding pagination is staright forward anyone can help me here?
Yeah I’d treat that as two separate parts, not normal pagination on the first request.
First request starts the job, then you keep checking the status until it says succeeded. Once it’s done, then paginate the final results request.
If the API gives you a next cursor/page, loop on that. If not, use offset like 0, 1000, 2000 until the response comes back with less than 1000 items.
1 Like
I would split this into two phases instead of treating the first response as normal pagination:
- Start the job with the POST request and store the returned job/id value.
- Poll the status endpoint on a short wait loop until the response says complete/succeeded.
- Add a max-attempt or timeout branch so the workflow fails clearly instead of looping forever.
- Once the job is complete, paginate only the final result endpoint, logging page token/page number and item count each pass.
That separation usually makes the expressions much simpler: one loop answers “is the job done yet?”, the next loop answers “are there more result pages?”.
1 Like
There are actually two separate issues and concerns here, this is more manageable than it looks.
You can keep your existing retry loop as-is for checking the status. Only once the IF node confirms success, you add a separate pagination loop before hitting the “Parse call list”:
After “Get Query Results”, check if the response includes a NextToken or offset this is usually returned with most paginated APIs.
Then use an IF node to check if NextToken exists, loop back to another “Get Results” HTTP call which passes the token; if not, you are all set.
Finally use a Merge or Code node to accumulate results across the pages before passing everything downstream to parse.
The key is to use a Code node or Set node to carry accumulated results and the current token forward on each loop iteration.
If you want to share the API results I can advise more on the accumulator logic.
1 Like
@mylearnings_abu the 2-phase pattern described — heres what it looks like as JSON u can drop in after ur “Get Query Results” node:
the loop is: IF has token → fetch next page → accumulate → back to IF. when nextToken is empty/missing, the IF’s false branch falls through to ur Parse step. if ur API uses offset/page instead of cursor, swap nextToken for a counter that increments by limit each pass.
1 Like