Question
How to handle large record bases while triggering them every minute?
Workflow
Airtable is our CRM. With 30K customer records in different stages like call, followup, customer, retarget, churned, ect…
what we are building in n8n:
We request all the records that are set to “Run automation” to true.
Based on the status and other criteria, we perform automations like sending reminder emails, adding information to records, changing dates ect.
Example:
We called a potential new customer and they are interested. We tick the “Run automation” field. n8n gets all the records and sees the new customer and adds the call date. At each run (every minute) it checks if 30 minutes passed to send the automated welcome email through the send email node.
Similar logic is applied to perform sending followups and other customer email. n8n reruns every minute and checks if X time is passed.
I have a feeling maybe requesting and handling thousands of records every minute is not a good idea in this setup.
Hey @blueberry , instead of having an n8n workflow poll your Airtable base every minute, you may want to run this in “batch”.
The gist is:
Option 1:
Run once every 30 minutes
Check all the records for a timestamp field (or other logic fields)
Gather the subset of records that match
Process those in batch
Option 2:
You could also craft an Airtable-native automation to check your records more frequently and then send the RECORD_ID() values to an n8n workflow webhook for faster processing.
Airtable has a hard limit of 5 requests per second per base. If you exceed that, you’ll start getting 429 errors.
I decided to implement some of your suggestions and also create a “Ready to process” field in Airtable that does some of the checking I do with n8n like “is it time to send the email”. This way I can run the workflow more frequently, and n8n will request just the necessary records.