I have created an automation where I read data from a google sheet - there are about 10,000 rows. Each row contains information that I use for a webhook to post to an API endpoint. I use the split in batches function so that I don’t overwhlem the API which has rate limiting.
I want the automation to post 250 webhooks a day then post another 250 the next day and so on until all rows are finished. I wasn’t sure how to achieve this. I tried using two split in batch functions but that doesn’t work as expected.
That is going to take a while to get through them all, Is there more to the limit than 250 a day or can you send all 250 at once?
You could read in the data then split in batches of 250 and add a wait node that waits 24 at the end and in the HTTP Request that I assume is being used to call the API you could use the Batching option and set it to 1 item per batch and set the interval to whatever works for the API.
As an example the workflow below gets 100 fake users, splits them into batches of 10 and the http request node sends the email to webhook.site and waits 2 seconds then sends the next one. Once all 10 are sent the wait node will wait 24 minutes before the workflow processes the next 10 items.
Hi @Jon
Thanks for this. I didn’t see that the HTTP request has an option for batching built in. I have added this now and I expect it will work as intended.
The reason for the 250 per day is simply due to a limitation by a third party provider for the tiered plan I am on.
