Add concurrency in workflow settings

Feature Requests
1

The idea is:

I need in workflow settings a way to set the number of concurrent jobs for each workflow.

My use case:

I have a given workflow which must be capable of handling concurrent executions.
I also have a workflow which MUST NOT run concurrent executions because sharing a common resource.

The --concurrency option is not a solution because it applies to all workflows.

I think it would be beneficial to add this because:

I am not the only one in this case Worker concurrency!

Any resources to support this?

Check Pipedream workflow settings.

Are you willing to work on this?

No

2

Hi @LucBerge

There is an easy work around, you can use the rabbitMQ nodes. The trigger node allows you to set the parallel executions. Of course these will be seperate workflow executions and so the workers of course also need to have enough concurrency set for it to actually do it all in parallel.

Hope this helps. :slight_smile:

2 Likes
3

It would be better to have a native solution

1 Like
4

Agreed. Using external tools (e.g., RabbitMQ / Posthook) are really a band-aid to this problem. n8n really needs a native queuing mechanism that allows you to see/manage in-flight requests before they are processed by a given workflow.

2 Likes
5

If there will be a native queue mechanism, it will be Redis :face_with_hand_over_mouth::face_with_hand_over_mouth:

2 Likes
6

That’s totally fine. I really don’t care what the underlying mechanism is – so long as n8n UI provides a way to manage the queue and control how many jobs run at the same time.

Keep in mind, this concurrency control should apply equally to:

  • Incoming Webhook Calls
  • As well as internal calls made from other parent workflows directly calling this workflow
1 Like
7

I agree with @dkindlund

1 Like
8

This is very much needed and long overdue.

Native concurrency control is needed natively to ensure that a workflow is running only once at a given time.

9

One more vote for. We really need a simple solution to define should every single workflow runs in parallel or not.

1 Like
10

This is very needed since there are lots of tools that have rate limiting on their APIs, causing the workflows to fail all the time.

1 Like
11

There are like a million critical reasons for the need to limit runs of a workflow to one at a time. Hands down, this is the biggest flow of n8n. Boggles my mind that it’s still not addressed.

12

You can use a queue brokers like rabbitmq to limit the workflow executions

1 Like