When I am working on this workflow, if I edit much of anything in the loop, the tab I’m working in hangs. I get a notification that n8n has stopped responding and my options are to wait or exit.
If I choose wait, the options come back after a timeout.
If I choose exit, any tabs I have open with n8n die, losing any edits I have not yet saved. Sometimes, it does not kill one or more other tabs.
I think I may have figured out what’s going on. I’d like confirmation and how to avoid this.
In the Next Page node, I increment the page number with
{{ $('Set Body').item.json["pageNum"]++ }}
I found this by attempting to manually rebuild the node. When I got to the Next Page node, after I inserted the variable, I added “++” to it so it would increment. I immediately experienced the hang I described earlier.
Yup, that will be a nice feature when I have time to work through it and figure out how it works. I have several legacy workflows that I will have to rewrite when that happens. Additionally, I have at least three different methods of pagination occuring.
I did take a look at it during some copious free time last week. It’s not as straight forward as I thought, especially when I have to calculate how many objects have been retrieved (page# * #ofObjectsPerRequest >= totalObjectsToPull), or “go until an empty page is returned”.
If your workflow processes a LOT of data, you should also consider reducing memory consumption by implementing part of your loop as a subworkflow, as explained here.
The workflow I’m working with here is actually 9 workflows. The first calculates and retreives the API credentials and gets a list of organizations, then sends that data to a flow that processes each organization, one at a time.
Now we get to the heavy lifting.
The flow calls three subflows to pull all of the objects for that organization on three platforms. After getting these lists, it combines the lists into:
