How to add new node in a production setup

Hi Guys, I am struggling to find material about how to add custom nodes to a production setup running over a docker into an ec2 Linux instance.

I am really confused about how to proceed. Any link to docs is welcome.

Hey @Domenico_Rutigliano!

Welcome to the community :sparkling_heart:

Can you please share where exactly you’re stuck? Did you already create the node?

Once you have created the node, you will have to:

  1. Create custom image using the Docker file n8n/Dockerfile at master · n8n-io/n8n · GitHub (docker build)
  2. Push that image to a Docker registry (or any other registry) (docker push command)
  3. Reference your image in the docker-compose n8n/docker-compose.yml at master · n8n-io/n8n · GitHub
  4. Deploy

Hi, Harshil Thank you!

I have not created any custom node yet, I am not sure how to do it. I am also confused about your answer because I cant see anywhere how a custom image using docker is pulling my custom nodes.

I think you have two options here:

1 - Add the custom node to packages/nodes-base as all build-in nodes and create a custom docker image.

2 - Add the custom node to .n8n/custom as described here.

The downside with the first approach it’s that will have to create a custom image every time you want to upgrade to a new version.

I’d like to proceed with option 2 however it is not clear how to structure the nodes under the custom folder and how to make n8n recognize them and load them as available nodes.

Is there any documentation about how to install custom nodes in the custom folder?

The .n8n folder has already been mounted in docker via the docker-compose.yml and if I go in the docker image I can see that the folder is there with the fils too.

volumes:
- ${DATA_FOLDER}/.n8n:/home/node/.n8n

Let’s link below explains it.

Hey @Domenico_Rutigliano,

Were you able to figure it out? Let us know if you’re stuck somewhere :slight_smile:

No guys, I am still stuck. I have installed the npm packages in the custom folder but when n8n starts does not read them. And I do not know how to debut it.

What do you mean you have installed the npm packages in the custom folder? Do you mean your node depends on additional npm packages or do you mean you published your node as an npm package and you installed it there?

I am finding it a bit confusing but I am going to explain to you what I have achieved so far.

I have cloned this repo GitHub - n8n-io/n8n-nodes-starter: Example starter module for custom n8n nodes. in the /custom folder. I am creating the credentials and node folders in the custom folder and then I did run npm install to install all the dependencies. Not sure if I am doing it right.

That will sadly not work. You can add custom nodes mainly in two different ways:

  1. Very simple nodes (without dependencies) can be created with n8n-node-dev which then copies it into the /custom folder.
  2. You can create an own package with custom nodes (like this example one). That package has to then get npm installed alongside n8n.

Hi Jan,

Thank you this already sounds promising. I have now been able to install my custom node.

Hi Jan. I’m also requiring some more understanding about the process to deploy a custom node in production.

I’ve built my node based on the process described here:

I would now like to find out how exactly, to deploy it in production?

Do i just copy paste the dist/credentials and dist/node folders to ~/.n8n/custom in my Docker setup?

I did check out this https://github.com/n8n-io/n8n-nodes-starter as well but couldn’t find any instructions there.

Another thing that i’ve attempted to do, is:

  • in the .n8n/custom folder, add
    • package.json (modified to include only my-node's credentials and node location),
    • dist/credentials,
    • dist/nodes/my-node
    • index.js (empty)
      and then npm i --save.

This does seem to work for my local (docker) setup, and i’m able to see my-node in the n8n UI.
But it doesn’t work for my prod cloud (docker) instance. Error:


Error loading node "wmf" from: "/home/node/.n8n/custom/node_modules/wmf/dist/wmf.node.js",
Error loading node "msRest" from: "/home/node/.n8n/custom/node_modules/@azure/ms-rest-js/dist/msRest.node.js",
 ›   Error: There was an error: tempModule[nodeName] is not a constructor

Does you custom node have dependencies?

@RicardoE105
This is all that it uses:

import { BINARY_ENCODING, IExecuteFunctions } from 'n8n-core';

import {
	IBinaryData,
	IBinaryKeyData,
	IDataObject,
	ILoadOptionsFunctions,
	INodeExecutionData,
	INodePropertyOptions,
	INodeType,
	INodeTypeDescription,
	NodeApiError,
	NodeOperationError,
	LoggerProxy as Logger
} from 'n8n-workflow';

import { OptionsWithUri } from 'request';

along with the node-related files.

Update: i tried this Using the Node Dev CLI | Docs as well.

The files got auto-generated which i just updated with my code and then copy/pasted to the Prod instance.

But, still the same error:

2021-09-04T06:06:05.368Z | info     | Initializing n8n process {"file":"start.js"}
2021-09-04T06:06:05.371Z | info     |
n8n now checks for new versions and security updates. You can turn this off using the environment variable N8N_VERSION_NOTIFICATIONS_ENABLED to "false"
For more information, please refer to https://docs.n8n.io/getting-started/installation/advanced/configuration.html
 {"file":"start.js"}

2021-09-04T06:06:06.108Z | debug    | No codex available for: FormIoTrigger.node.js {"file":"LoadNodesAndCredentials.js","function":"addCodex"}
2021-09-04T06:06:06.113Z | debug    | No codex available for: FormstackTrigger.node.js {"file":"LoadNodesAndCredentials.js","function":"addCodex"}
2021-09-04T06:06:07.394Z | debug    | No codex available for: N8nTrainingCustomerDatastore.node.js {"file":"LoadNodesAndCredentials.js","function":"addCodex"}
2021-09-04T06:06:07.403Z | debug    | No codex available for: N8nTrainingCustomerMessenger.node.js {"file":"LoadNodesAndCredentials.js","function":"addCodex"}
 ›   Error: There was an error: Cannot find module 'n8n-workflow'
 ›   Require stack:
 ›   - /home/node/.n8n/custom/Directus.node.js
 ›   - /usr/local/lib/node_modules/n8n/dist/src/LoadNodesAndCredentials.js
 ›   - /usr/local/lib/node_modules/n8n/dist/src/index.js
 ›   - /usr/local/lib/node_modules/n8n/dist/commands/start.js
 ›   - /usr/local/lib/node_modules/n8n/node_modules/@oclif/config/lib/plugin.js
 ›   - /usr/local/lib/node_modules/n8n/node_modules/@oclif/config/lib/config.js
 ›   - /usr/local/lib/node_modules/n8n/node_modules/@oclif/config/lib/index.js
 ›   -
 ›   /usr/local/lib/node_modules/n8n/node_modules/@oclif/command/lib/command.js
 ›   - /usr/local/lib/node_modules/n8n/node_modules/@oclif/command/lib/index.js
 ›   - /usr/local/lib/node_modules/n8n/bin/n8n

I just followed these steps and it worked fine for me.

So I suggest, create a new test node via the CLI. Then build it locally (without any changes!) and then start n8n to confirm it loads fine. Then copy that build file to your production server and confirm that it also works there.

Like that, we can get closer to find out if the problem is in the code or somewhere else.

Hi Jan, i just tried the n8n-dev CLI approach, and was able to successfully run the Test node in my prod instance.

Where do you think i should check for bugs in my setup? Or rather, what’s the optimum way to use my already developed code with the CLI approach?

I would probably start to replace the code of the test node step by step with your existing one. First, change the name and the file name. Then add the imports, then the parameters, then the execute code. You will then see where it breaks.