Rate Limit for BigQuery Nodes

Questions
1

Hey there,

We are using N8N and BigQuery to build an in-house database capable of having all of our TAM enriched at real time.
The problem is that we have a lot of workflows linked to the same table and when we make multiple requests at the same time to it it will crash the query and stop the execution.

We needed a way of either re-run these executions until they get approved, or some kind of rate limit that would say to n8n something like: We are already running a query to this table, wait until it gets done.
We already tried the “retry on fail” option, but we can only do that for 5 times with 5 seconds interval and that is not enough.

Do you have some solution for this?

We can’t use the batches option because these are multiple workflows running, and not only once.

Thanks

3

Hi @Pedro_Azevedo :wave: Welcome to the community :tada:

You wouldn’t be able to do this directly in n8n, though you may be able to do this with a third-party service like RabbitMQ. Admittedly this might be a little bit tricky to configure - I believe you’d need to have one main workflow fetching the messages from RabbitMQ, and then calling the other workflows you’ve mentioned here as sub-workflows :bowing_man:

1 Like