N8N Main Instance/Queue Instance Setup?

looks the same as digitalOcean really, thanks. I will check it out.

just an A name to a IP addie

@Anshul_Namdev @achamm

do either of you happen to know if you can send workflows to specific workers or stop it going to particular workers?

ie: Workflow XYZ runs only on w3.domain.com or w4.domain.com

Also, can you connect an n8n-worker to more than one n8n main instance?

n8n doesn’t let you pin workflows to specific workers natively. all workers just stare at the exact same redis queue and grab whatever execution pops up first. they’re built to be totally interchangeable. if you’re spinning up optimized droplets on digitalocean just for heavy workflows, you’d actually need to set up a completely separate n8n main instance and point those specific workers only to that new queue.

same goes for sharing workers across different main instances – doesn’t work. a worker has to use the exact same database and N8N_ENCRYPTION_KEY as its main instance so it can decrypt your credentials when a workflow runs. you can’t hook one worker up to two different environments. basically it’s strictly one worker pool per main instance.

Thanks @A_A4

I figured as much from reading the documents, thought i would ask.

I think it would be a great feature to be able to tag a workflow to worker, would allow you to have one worker (on a server) to handle AI and heavy tasks, and another smaller server to maintain general compute. it makes sense that the workers have no particular agency, but at the same time it would help deployment if there was the ability to set paths to workers based on workers capacity.

I kind of think it wouldn’t be to hard, so have created a feature request. really the worker just need to be able to exclude a tag that isnt its own in the redis queue, and the main n8n needs the ability to share a workflow tag.

thought that was the issue on queue as well, was just wondering if it would be possible to have a pool of workers that you could pull from, something like this.

        ┌───────────────────────────────┐      ┌───────────────────────────────┐
        │        io.domain1.com         │      │        io.domain2.com         │
        │        (Main Server 1)        │      │        (Main Server 2)        │
        │                               │      │                               │
        │          ┌─────────┐          │      │          ┌─────────┐          │
        │          │  Main   │          │      │          │  Main   │          │
        │          └────┬────┘          │      │          └────┬────┘          │
        │               │ pushes jobs   │      │               │ pushes jobs   │
        └───────────────┼───────────────┘      └───────────────┼───────────────┘
                        │                                      │
                        └───────────────┬──────────────────────┘
                                        ▼
                              ┌────────────────────┐
                              │   redis.domain.com │
                              │    (Redis Server)  │
                              └─────────┬──────────┘
                                        │
                                        ▼


┌────────────────────┐   ┌────────────────────┐   ┌────────────────────┐   ┌────────────────────┐
│   w1.domain.com    │   │   w2.domain.com    │   │   w3.domain.com    │   │   w4.domain.com    │
│     (Server 3)     │   │     (Server 4)     │   │     (Server 5)     │   │     (Server 6)     │
│                    │   │                    │   │                    │   │                    │
│    n8n-worker-1    │   │    n8n-worker-2    │   │    n8n-worker-3    │   │    n8n-worker-4    │
│         │          │   │         │          │   │         │          │   │         │          │
│         ▼          │   │         ▼          │   │         ▼          │   │         ▼          │
│    n8n-runner-1    │   │    n8n-runner-2    │   │    n8n-runner-3    │   │    n8n-runner-4    │
│  (Code execution)  │   │  (Code execution)  │   │  (Code execution)  │   │  (Code execution)  │
└────────────────────┘   └────────────────────┘   └────────────────────┘   └────────────────────┘