Hey there,
I have a HTTP request node that’s been running for more than 30 minutes for 500 items.
But the API has a rate limit of 120 requests per minute which means it should be taking about 4-5 mins for 500 items. Even if it retries between requests, I don’t think it make sense to take so long. May I know the best way to fix this?
I’m using this API: https://rapidapi.com/rockapis-rockapis-default/api/linkedin-data-api (Pro Plan)
Here’s my full node setup:
Thanks in advance!
Leonard
In order to “fix this” you’ve first need to figure out the “why”. Is there a way to see any logs or statistic from the rapidapi side?
yes, there are logs. But even after the last API call, the node in n8n kept running i.e. the last API call in RapidAPI stated 10:22 and the current time is say 10:55 but the node is still running @jabbson
Well, I guess at this point, I’d start worrying too haha. Have you tried doing the same request on 1 or 10 records to… have some sort of a baseline?
yes, I have tried it for 10+ items that’s why i went on to run it to 500+ items. @jabbson
@n8n_Team need your help please!
Try batching your requests like in the example below. I mock out 1000 items in a list, then I use the loop node and set the batch at 100 items, wait 1 minute before I continue with the next set of 100 items/http calls. This should limit your rate limits to 100 items every minute. Off course you’ll need to adjust the waiting time according to the rate limit rules of the api youre calling.
In this case the Loop node will run 10 times. 10x100 = 1000 items
