Workflow worker selector/affinity

The idea is:

If multiple workers are available, allow specifying constraints for which worker a workflow execution gets assigned to. A direct (but overly complex for n8n’s usecases) analog is, for example, Kubernetes node affinity.

In very brief terms, this could be a workflow setting with a few sub-properties, all of them optional:

  • IF running in queue mode
  • Toggle: “Specify worker selector”, which makes additional fields visible (again, all of these are optional)
  • Options:
    • Worker machine hostname
    • Worker machine architecture (macos, linux, etc)
    • Worker name (don’t think this exists at the moment, but a way to easily specify a single worker would be useful)
    • Toggle: Strict?

If “Strict” is enabled, a worker matching the constraints MUST be found for execution to be possible; if it’s not enabled, a worker matching the constraints will be preferred, but if none is found, the execution is passed to any other worker.

My use case:

A simple use-case that would be useful for me is being able to run multiple n8n instances on different devices, setup workflows using specific aspects of those devices from a single n8n UI/control plane, and have them run in the correct device.

For example:

  • Run certain filesystem related tasks on the correct device with the expected filesystem
  • Run certain operations requiring a GPU where one is available

I think it would be beneficial to add this because:

See use-case for specific examples

Any resources to support this?

Kubernetes: pod affinity documentation - Assigning Pods to Nodes | Kubernetes

Are you willing to work on this?

Yes! Got some work done on this as a proof of concept, though haven’t had a chance to progress much yet.

Adding as a comment to not muddy-up the original post too far: a cool follow-up on this is having the option of specifying a worker selector workflow - a workflow that can be used to pick a worker for workflow execution based on arbitrary conditions.