Could you consider "Delay after queue" node?

In my case: I have 3 requests for 3 tasks. But if these requests take at the same time, node will take the same task for 3 requests, instead of every.

This solve issues when other database through API have not updated yet, but need to get different IDs for tasks.

This look like:

Workflow without queue
[request 1] - [workflow]<at 9:00 am> - Got task id 01 <done at 9:00>
[request 2] - [workflow]<at 9:00 am> - Got task id 01 <Error! Duplicated ID>
[request 3] - [workflow]<at 9:05 am> - Got task id 02 <done at 9:05>

Workflow with queue
[request 1] - [workflow have queue node]<at 9:00 am> - [queue 1] - Got task id 01 <done at 9:00>
[request 2] - [workflow have queue node]<at 9:00 am> - [queue 2] - Got task id 02 <done at 9:01>
[request 3] - [workflow have queue node]<at 9:05 am> - [queue 1] - Got task id 03 <done at 9:05>
Could you consider "Delay after queue" node? - #5 by cmdntd987

"Delay after queue" node will solve this, for example by this way:

  1. Save data of every request first
  2. Delay and numbered them, ready for queue
  3. Send one by one request in queue

Although we could create my own database for queue, but it is rather complex in some case
I think a node for this function is better.

This is very helpful function in zapier, you could see this here:

The Delay After Queue option allows you to create a queue of actions to run. When a Zap is triggered, it adds actions to the queue. The queue then plays the actions one at a time in the order that they were added to the queue. There is a delay between each set of actions for a Zap.

This can be useful if you are running into limits with too many requests to an app, or if you have multiple Zaps that may try to update a record at the same time.

The maximum time a task can be held for is for one month (31 days). Since tasks will be queued, the maximum number of tasks in the queue depends on the Time Delayed For (value) value. For example, if you set the delay to one day, the maximum number of tasks in the queue will be 31. If a 32nd task is created within that period, it will error on the delay step because the scheduled time it will be released is too far away.
https://zapier.com/help/create/customize/add-delays-to-zaps

I may be missing something but does the new wait node not solve this for you or do you need more than just a wait in your workflow?

I do not think wait node could solve this.
If you use wait node for 3 requests, it will also do nodes in workflow at the same time after.

A “queue” create a order which force requests have different delay time automatically.

Hey!

Would a combination of Split In Batches and the Wait node do the trick? The delay won’t be different for each request, though.

Yes. But main issue is “in queue”. As I talk, I could build a seperated data to solve this.
If you try to build a function to solve problem in queue, you must try to solve in a random number of request, like 30 or 300 requests…

A node for “queue” will try to build a automatic function to make a queue. This could find way to make order for requests in queue. I do not need to pay attention to data of requests, or how many requests I have.

You could know about this logic in explaination zapier link above. They know “in queue” is another kind of delay which is helpful in this case. You must do every request, but force them to do one-by-one, and in order first-to-last, not the same time.

This solve issues when other database through API have not updated yet, but need to get different IDs for tasks.

This look like:

Workflow without queue
[request 1] - [workflow]<at 9:00 am> - Got task id 01 <done at 9:00>
[request 2] - [workflow]<at 9:00 am> - Got task id 01 <Error! Duplicated ID>
[request 3] - [workflow]<at 9:05 am> - Got task id 02 <done at 9:05>

Workflow with queue
[request 1] - [workflow have queue node]<at 9:00 am> - [queue 1] - Got task id 01 <done at 9:00>
[request 2] - [workflow have queue node]<at 9:00 am> - [queue 2] - Got task id 02 <done at 9:01>
[request 3] - [workflow have queue node]<at 9:05 am> - [queue 1] - Got task id 03 <done at 9:05>

Alright so rather than a delay what you want is a process order / job queue. I am not sure how n8n processes jobs and if it waits or if it does them one by one this might be something fixed with the run in process option.

If you have a database though you could create a little system where your workflow gets triggered but writes the data to a database then you can have a second workflow that runs on a cron schedule or gets called by the main workflow once it has checked the job is good to start.

1 Like

Another alternative that I can think of using Redis or services like Kafka/Rabbit MQ. These are still not native solutions that @cmdntd987 is looking for, but is a possible solution :slight_smile:

n8n has nodes for these services, which can reduce the complexity of setting it up a bit.

@harshil1712 what about setting EXECUTIONS_MODE to queue instead of using the default directly option? There also seem to be some Queue variables that can be set as well.

Yes, but that is helpful when you have workers setup.