How to Efficiently Fetch Exactly 600 Records from Airtable with Pagination in n8n?

[Rohit_Gurav]

Hi n8n community,

I am building a workflow where I need to fetch exactly 600 records from an Airtable table before proceeding with the next steps. I understand that Airtable’s API allows fetching records in batches of 100 using the offset parameter. My current approach involves:

1. Initializing variables like offset, recordsFetched, and allRecords.
2. Using an HTTP Request node to fetch records from Airtable with the offset parameter.
3. A Code node to accumulate records across iterations and update the offset for the next request.
4. An If node to stop the loop once recordsFetched reaches 600.

Challenges:

1. Accumulating records dynamically across iterations.
2. Exiting the loop with exactly 600 records and ensuring no records are lost in the process.
3. Handling the issue where 5 items end up in the false branch and only 1 item in the true branch in the If node. I want all 600 records to move together to the next step after exiting the loop.

Key Requirements:

1. Fetch records in batches of 100.
2. Accumulate the fetched records until the total reaches 600.
3. Dynamically update the offset for each request.
4. Stop the loop once the total records fetched equals or exceeds 600.
5. Ensure all 600 records are available in the true branch after the loop ends, without splitting between true/false branches.

Question:

What is the most efficient way to implement this in n8n? Are there any best practices or alternative solutions to handle this scenario?

Any advice, examples, or pointers would be greatly appreciated!

[n8n]

It looks like your topic is missing some important information. Could you provide the following if applicable.

• n8n version:
• Database (default: SQLite):
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app):
• Operating system:

[ihortom]

Hey @Rohit_Gurav , I cannot see you leveraging the built-in pagination feature of the HTTP Request node. You are better off with that implementation, HTTP Request node documentation | n8n Docs.