Describe the problem/error/question
Greetings,
A newbie here, just investigating self-hosting and especially running in queue mode for more distributed workloads instead of a single point of failure where the server goes down in regular mode.
How am I meant to install community or private nodes on a queue mode environment with separate EC2 instances?
Let’s say I would like to install https://www.npmjs.com/package/n8n-nodes-text-manipulation and run it in one of my workflows.
I created the nodes directory for both the main instance (running in queue mode, responsible for the GUI, redis and postgres) and installed the package according to the guide Manual installation | n8n Docs
I also created nodes directory and installed the package on the worker instance (running only the n8n worker process).
Executing distribution looks to be working correctly. Webhook and scheduled executions are being run from the worker instance, and manually execution looks to be running on the local main instance with some built-in nodes in a test workflow.
However:
- Actually migrated the instance over from regular mode (and installed the community node from gui). When a previous workflow that relied on the text manipulation node ran, worker logs saying start job # and that’s it no error, but the main logs saying the node not found and the workflow executions all failing.
- Switch the main instance back to regular and uninstalled the community nodes (to rule out any conflicts with duplicate installation), and switched back to queue mode. Now the community node has disappeared, so a red exclamation mark on that node.
Any guidance on this setup, please? How would I install community nodes or private nodes in queue mode running, not on docker.
Please share your workflow
{
“meta”: {
“instanceId”: “8c228450815ea39980eb5c9e374275a13d739980d2f6d4cbb6d4e48be6d40968”
},
“nodes”: [
{
“parameters”: {
“authentication”: “headerAuth”,
“path”: “c8db099c-f18f-4993-be76-ddc46fef746e”,
“options”: {}
},
“id”: “9cc0b951-9ed6-4479-85e7-a45aac144eaa”,
“name”: “Webhook”,
“type”: “n8n-nodes-base.webhook”,
“typeVersion”: 1,
“position”: [
820,
360
],
“webhookId”: “c8db099c-f18f-4993-be76-ddc46fef746e”,
“credentials”: {
“httpHeaderAuth”: {
“id”: “j2kceMrJfqoTgVjw”,
“name”: “LoadTestAuthHeader”
}
}
},
{
“parameters”: {
“unit”: “seconds”
},
“id”: “de1de6f2-1ba0-4a04-b9bd-97b65cc82ba9”,
“name”: “Wait”,
“type”: “n8n-nodes-base.wait”,
“typeVersion”: 1,
“position”: [
1040,
360
],
“webhookId”: “a2f7a056-f62c-4ec5-89c8-0976d63c7667”
},
{
“parameters”: {
“keepOnlySet”: true,
“textsWithManipulations”: {
“textsWithManipulationsValues”: [
{
“dataSources”: {
“dataSource”: [
{
“text”: “asdfasdfasdfa”
}
]
}
}
]
}
},
“id”: “c5d9c099-5b54-4249-9eea-dc8f95694a62”,
“name”: “TextManipulation”,
“type”: “n8n-nodes-text-manipulation.textManipulation”,
“typeVersion”: 1,
“position”: [
1260,
360
]
},
{
“parameters”: {
“jsCode”: “// Loop over input items and add a new field called ‘myNewField’ to the JSON of each one\nfor (const item of $input.all()) {\n item.json.myNewField = 1;\n}\n\nreturn $input.all();”
},
“id”: “e3e0cc61-47f7-4372-a0f2-5aa992d50802”,
“name”: “Code”,
“type”: “n8n-nodes-base.code”,
“typeVersion”: 2,
“position”: [
1480,
360
]
},
{
“parameters”: {
“options”: {}
},
“id”: “978b16bf-bfa1-4b6f-aaa8-ff114e93a14d”,
“name”: “Split In Batches”,
“type”: “n8n-nodes-base.splitInBatches”,
“typeVersion”: 2,
“position”: [
1700,
360
]
},
{
“parameters”: {},
“id”: “f7051cc0-b420-4eef-a504-233bb54c05f6”,
“name”: “Filter”,
“type”: “n8n-nodes-base.filter”,
“typeVersion”: 1,
“position”: [
1920,
360
]
},
{
“parameters”: {},
“id”: “ea34ca88-0578-4ca0-a658-a5cb55760721”,
“name”: “When clicking "Execute Workflow"”,
“type”: “n8n-nodes-base.manualTrigger”,
“typeVersion”: 1,
“position”: [
860,
180
],
“disabled”: true
},
{
“parameters”: {
“options”: {}
},
“id”: “4c4f7dd5-bc88-4d05-afc1-490c09c3c302”,
“name”: “Date & Time”,
“type”: “n8n-nodes-base.dateTime”,
“typeVersion”: 2,
“position”: [
2140,
360
]
},
{
“parameters”: {
“values”: {
“string”: [
{
“value”: “aaaa”
}
]
},
“options”: {}
},
“id”: “1025cde2-8610-4af2-9ce8-8999632273ae”,
“name”: “Set”,
“type”: “n8n-nodes-base.set”,
“typeVersion”: 2,
“position”: [
2580,
360
]
},
{
“parameters”: {},
“id”: “3d570ce9-0a43-4032-8b50-5ecd38b85d04”,
“name”: “Execution Data”,
“type”: “n8n-nodes-base.executionData”,
“typeVersion”: 1,
“position”: [
2800,
360
]
},
{
“parameters”: {
“value”: “ghjikjhjkjhjklkj”
},
“id”: “cb7400e8-1b71-4e77-b4f8-d1af9595da95”,
“name”: “Crypto”,
“type”: “n8n-nodes-base.crypto”,
“typeVersion”: 1,
“position”: [
3020,
360
]
},
{
“parameters”: {
“unit”: “seconds”
},
“id”: “7d43eed8-e19e-4d4b-a2a1-8775df28c101”,
“name”: “Wait1”,
“type”: “n8n-nodes-base.wait”,
“typeVersion”: 1,
“position”: [
2360,
360
],
“webhookId”: “7ac8df54-1d51-4713-b3cd-55473abe544f”
}
],
“connections”: {
“Webhook”: {
“main”: [
[
{
“node”: “Wait”,
“type”: “main”,
“index”: 0
}
]
]
},
“Wait”: {
“main”: [
[
{
“node”: “TextManipulation”,
“type”: “main”,
“index”: 0
}
]
]
},
“TextManipulation”: {
“main”: [
[
{
“node”: “Code”,
“type”: “main”,
“index”: 0
}
]
]
},
“Code”: {
“main”: [
[
{
“node”: “Split In Batches”,
“type”: “main”,
“index”: 0
}
]
]
},
“Split In Batches”: {
“main”: [
[
{
“node”: “Filter”,
“type”: “main”,
“index”: 0
}
]
]
},
“Filter”: {
“main”: [
[
{
“node”: “Date & Time”,
“type”: “main”,
“index”: 0
}
]
]
},
“When clicking "Execute Workflow"”: {
“main”: [
[
{
“node”: “Wait”,
“type”: “main”,
“index”: 0
}
]
]
},
“Date & Time”: {
“main”: [
[
{
“node”: “Wait1”,
“type”: “main”,
“index”: 0
}
]
]
},
“Set”: {
“main”: [
[
{
“node”: “Execution Data”,
“type”: “main”,
“index”: 0
}
]
]
},
“Execution Data”: {
“main”: [
[
{
“node”: “Crypto”,
“type”: “main”,
“index”: 0
}
]
]
},
“Wait1”: {
“main”: [
[
{
“node”: “Set”,
“type”: “main”,
“index”: 0
}
]
]
}
}
}
Information on your n8n setup
- n8n version:1.4.1
- Database (default: SQLite):postgresql
- n8n EXECUTIONS_PROCESS setting (default: own, main):main with queue mode
- Running n8n via (Docker, npm, n8n cloud, desktop app):npm
- Operating system:ubuntu 22.04 on AWS graviton