Can I run Google Sheets Trigger workflows sequentially instead of all at once?

Questions
1

Describe the problem/error/question

I have a need to create a bunch of google sheets trigger workflows to check and get new rows from a bunch of different spreadsheets that update randomly about every 5 minutes.

I already have a workflow-creation workflow set up that creates the google sheets trigger workflows with the appropriate sheet ID.

I’d like to figure out a way to have n8n still check for new rows, but iterate over the active google sheets trigger workflows and execute them one by one so that I don’t go over quota limits.

I know that I can create new google projects with new users and up the amount of calls I can make per minute, but I’m running out of that option as well.

Looking through the code here: n8n/packages/nodes-base/nodes/Google/Sheet/GoogleSheetsTrigger.node.ts at master · n8n-io/n8n · GitHub

I see that the trigger node is polling every X amount of time using cron and then checking to see if any revisions have happened, what the old data was, and what the new data is. That’s 3 API calls per check. The trigger node also seems to be keeping some internal state between executions and is using that to determine if there’s data to update.

Given that google’s default for reads per minute per user is 60 (essentially once per second): Limites d'utilisation  |  Google Sheets  |  Google for Developers

I can theoretically check 60 / 3 = a max of 20 sheets, but in practice it’s usually much lower.

The real question:

Is there way to make those triggers check one after the other instead of all at once?

Barring that, is there a way that I can easily check for rows added in a spreadsheet between executions so that I can control things a bit more manually?

Perhaps there’s a way to configure n8n to just execute one thing at a time in n8n itself? Does this do it? Concurrency control | n8n Docs

I have the workflows polling (through the google sheets trigger node) every 5 minutes and the usage graph on the API calls shows a big spike every 5 minutes and then nothing in-between because n8n is using cron to schedule those checks.

I’d love it so that the usage is spread out and I’m not going over my API limits.

Thank you in advance for any advice.

What is the error message (if any)?

I’m getting a google 429 error because I have too many google sheets trigger nodes executing at exactly the same time.

Please share your workflow

Imagine you have a bunch of workflows with these google sheets trigger nodes that are checking for new rows and then posting to an HTTP Request

Information on your n8n setup

  • **n8n version: 1.51.2
  • **Database (default: SQLite): postgres
  • **n8n EXECUTIONS_PROCESS setting (default: own, main): main
  • **Running n8n via (Docker, npm, n8n cloud, desktop app): docker
  • **Operating system: linux
2

hello @aiandwifi

you can set the custom Cron time for each trigger to not overload the API limits
image

Actually, it behaves the same way as the schedule trigger node + a usual gSheet node

3

I’m aware that I can use cron, but the problem still comes up that all of the workflows trigger at that minute and I also need every workflow to check its sheet every 5ish minutes.

The ideal would be something like

Workflow 1 check at 0:00:00
Workflow 2 check at 0:03:00
Workflow 3 check at 0:06:00
Etc

With cron, it’s
Workflow 1 check at 0:00:00
Workflow 2 check at 0:00:01
Workflow 3 check at 0:00:02

The desired effect is that there’s maybe a 2-3 second delay between all workflow checks, aka n8n processes them sequentially, not all at once.

If I can simulate that somehow, I’m ok with it.

Even if I have to put in a random delay onto each workflow that’s fine too.

I just don’t want n8n spamming the API and getting rate limited.

Are any of my other solutions viable?

Is there a better alternative?

Is there code I can change?

Perhaps a community plugin that exists or I could make?

4

Just set the cron expression to trigger them it that way. E.g.:

  • workflow 1 - trigger every 3 minutes starting from 2 - 00:02:00, 00:05:00, 00:08:00, 00:11:00 - 2-59/3 * * * *
  • workflow 2 - trigger every 3 minutes starting from 3 - 00:03:00, 00:06:00, 00:09:00, 00:12:00 - 3-59/3 * * * *
  • workflow 2 - trigger every 3 minutes starting from 4 - 00:04:00, 00:07:00, 00:10:00, 00:13:00 - 4-59/3 * * * *

Crontab.guru - The cron schedule expression generator

5

Thanks, this may be a decent stopgap.

Does anyone else know if there’s an answer to my original question though?

Can I make those triggers check one after the other instead of all at once?

I just want to run one workflow, then the next, then the next, etc. not all at once.

6

Try this

1 workflow runs on your shedule eg 3, 6, 10 etc… Now what this will be doing is this, actiate and deactiate workflows on schedule

  • Set an activatior workflow
  • Set if/ swithc with your cron shcedule
  • IF satisfied,activate workflow X
  • Wait X seconds/minutes
  • Switch workflow X off

This will mean only the desired workflows will run on demand

1 Like
7

I ended up trying a similar solution. I tagged the workflows I wanted to put into a virtual queue so that would be persisted across runs. I then looked up all of the workflows in n8n and sequentially ran them by turning them on and off in batch mode of 1 with 1000ms in between runs.

That worked for a while, but I just have too many google sheets checking and it was a pain to manage balancing them between accounts, etc. to get around the small rate limits.

My final solution was to bypass using Google Sheets Trigger altogether in favor of some web hooks. It made my code much simpler and n8n deals with web hooks in a queue with concurrency controls.

8

you can also just download the spreadsheet document every N minute, use the Extract from file node, get every sheet from it and parse the content manually to check the changes and do whatever you’ll need to do

9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.