I want to create a custom node for ActiveMQ queue/topic

leonardlin · September 24, 2019, 6:47pm

Hi

I’m very interested in n8n and would like to use it for my next project.

We are using ActiveMQ for events and currently we have to write listeners/connectors to many systems manually.

n8n seems like a good fit.

I’m interested to write a custom node that can connect and react to ActiveMQ events and contribute the node to the n8n community.

Before I start I wanted to ask if there are any limitations to how custom nodes work why there are no other queue-connectors yet.
Do they have timeout limitations? Is it not possible to keep a connection open to a queue for a long time?

Thanks for any input or hints that could help me in developing such a connector.

Best
Leo

jan · September 24, 2019, 7:08pm

Welcome, Leo!

Great to have you here! Happy to hear that you are interested in n8n and thinking about using it.

The reason that there are no queue connectors yet is just a lack of time. Implementing them should be not complicated. There are no known limitations in n8n which should cause any problems.

A good reference is the Interval-Node. It is very simple and the implementation would be pretty much the same. You can find the code here:
https://github.com/n8n-io/n8n/blob/master/packages/nodes-base/nodes/Interval.node.ts

You just have to implement the “trigger” function. This is the function which gets called when a workflow gets activated. There you can create the connection. Then every time a message gets received simply the this.emit function can be called with the message.
The function itself returns than simply the closeFunction which gets called when the workflow gets deactivated or n8n gets shut down (there you can close the connection to the queue). Additionally, also the manualTriggerFunction could be implemented but does not make much sense in this case. That function sends some example data when in development mode but in your case, it would not be needed as you would like to wait for an actual event and its data when testing/developing a workflow.

Hope that helps! If you have any other questions simply get back to me!

blue skies

Jan

leonardlin · October 30, 2019, 6:47pm

Hi Jan

Thanks for the quick intro.

In the development guidelines, it is mentioned that you should try to avoid third-party libraries.

Does this apply to the request/http library only, or in general?

for example:
I could leverage this library: GitHub - amqp/rhea: A reactive messaging library based on the AMQP protocol

alternatively I could try to use activemq-REST with websocket/keep-alive and periodic reconnects.

Thanks for any guidance.

jan · October 30, 2019, 8:51pm

For the request/http library that is a very hard rule. As I do really not see a reason to have multiple libraries that do the same.

For other libraries that is a little bit different. If it is possible to avoid I prefer to not add them if it does not have any disadvantages and the additional work involved is reasonable. But it does also not make sense to reinvent the wheel. So, in this case, I can see why adding an external dependency makes sense. Especially does it seem like one that does not have many dependencies by itself.

leonardlin · October 30, 2019, 9:08pm

I wrote something and deployed it to my local n8n installation.

When executing I get unsurprisingly:

Error: Cannot find module 'rhea'

I guess I have to checkout n8n/nodes-base from git and include it there?
There is no way nodes can load external libraries directly, right?

I was looking at your nodes, where you use an external library as well. e.g. redis

jan · October 30, 2019, 9:17pm

Yes, that would probably be best. Here is also a guide on how to create a node for n8n and how to set it up:

github.com

n8n-io/n8n/blob/master/CONTRIBUTING.md#create-a-new-node-to-contribute-to-n8n

# Contributing to n8n

Great that you are here and you want to contribute to n8n

## Contents

- [Code of Conduct](#code-of-conduct)
- [Directory Structure](#directory-structure)
- [Development Setup](#development-setup)
- [Development Cycle](#development-cycle)
- [Create Custom Nodes](#create-custom-nodes)
- [Create a new node to contribute to n8n](#create-a-new-node-to-contribute-to-n8n)
- [Checklist before submitting a new node](#checklist-before-submitting-a-new-node)
- [Extend Documentation](#extend-documentation)
- [Contributor License Agreement](#contributor-license-agreement)

## Code of conduct

This project and everyone participating in it are governed by the Code of
Conduct which can be found in the file [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).

This file has been truncated. show original

Yes, some nodes use already external dependencies. Want to however remove in the long-term some of them again. Like for example the “googleapis” as it should all be possible directly via REST. It is however still there as it was one of the first nodes I ever created for n8n and so it stayed behind. And because the library was now already there anyway, I also reused it in the Google Drive node.

leonardlin · October 31, 2019, 3:41pm

Sorry for the many questions.
I have n8n running in dev mode, but I can’t find the webpack.config.js

Because I want to add:

 devServer: {
    disableHostCheck: true
 }

I work on remote machines and use VSC remote workspace.
Currently I’m getting a

jan · October 31, 2019, 3:59pm

There is no webpack.config.js. But you can check the /n8n/packages/editor-ui/vue.config.js maybe you can set it there.

leonardlin · October 31, 2019, 4:08pm

never mind, stupid user error.

leonardlin · November 1, 2019, 7:46am

My first draft of the listener.
https://github.com/leonardlin/n8n/commit/230dc33752f93d754df34f5ec5dc485267eeeda3

Let me know if you have any comments.

Will now try to make a simple sender.
Hope to open a PR today.

jan · November 1, 2019, 8:30am

Great, just checked it out. Looking very good!

Some comments:

I would probably move host & port into the credentials and make them required. After all, it is a unit and they belong together. Even though host and port do not have to get saved encrypted it makes it easier for the user to define all of that once and then simply select them instead of having to define them again and again. The credentials can there also be imagined as connections to make it convenient for the users. Also if the host will change in the future will they have to change it only once in the credentials and not separate on every node in every workflow.
You are right, the “manualTriggerFunction” does not make sense here and can simply be omitted.
I think the linter is to set up to complain if callback functions are defined like this: function(a) { ... }. You should therefore normally use the short syntax (a) => {...}. But you can check by running npm run lint. The same when defining “port: port,” in line 98. The short form port, should be used.

Those are at least the things that jumped in my eye without downloading and testing it.

Thanks a lot for your work! Looking forward adding it to n8n!

leonardlin · November 1, 2019, 11:35am

I’m getting a missing “lint”

Is there another way to trigger the linter.

jan · November 1, 2019, 11:37am

Uh strange. Thought it was In the package.json. Sadly not in front of my computer. So can not check. But you should be able to add it manually with npm I tslint

leonardlin · November 1, 2019, 12:53pm

No worries,

I have opened a PR:
https://github.com/n8n-io/n8n/pull/101

I couldn’t get lint to work, I’m not good with npm/webpack stuff.
If you have some points I can try again and adjust the code.

jan · November 1, 2019, 7:12pm

Great, thanks a lot I will check it out tomorrow. Have a nice evening!

Alhussein · June 10, 2020, 9:29pm

We need nodes for send/trigger to Rabbitmq also
@jan said that the AMQP Sender & AMQP Trigger are designed to work with ActiveMQ, So why you do just use them??
I really enjoy your talking @jan & @leonardlin

leonardlin · June 10, 2020, 9:46pm

Hi @Alhussein

The situation with rabbitMQ amqp and ActiveMQ is a bit confusing.

Back in the days there was the protocol amqp 0.9.1

RabbitMQ supported amqp 0.9.1

Later amqp 1.0 was defined and some softwares moved to that version.
ActiveMQ was among them to support amqp 1.0

RabbitMQ vehemently sticks to version 0.9.1

The adapter I wrote is using amqp 1.0 and is tested with activeMQ

RabbitMQ won’t work with it.

However, you can copy the node and switch it-in an amqp 0.9.1 compatible library.

Alhussein · June 19, 2020, 2:01am

Hey @leonardlin it’s me again
I follow the step as possible as I can
I try to use ‘ampqlib’ and ‘amqp-ts’ external library to connect to rabbitmq but I faced an error which is say: amqplib not found
here what I did:
1- Install n8n-node-dev
2- clone n8n source code from github
3- run lerna bootstrap --hoist in the n8n folder
4- n8n-node-dev new >> node >> till created
5- I require amqplib and used it to simply just send msg to queue (the code is working as nodejs program)
6- I add amqplib to package.json file in n8n/packages/nodes-base/ folder
7- npm run build and npm run start fine
8- new node appear on ui editor fine, but when executed said that amqplib isn’t found
I try to use npm install I messed up
So what I missed here ???
How to add an external library to n8n code??
is it just simply added the desired library to package.json inside nodes-base folder??
thanks in advance

RicardoE105 · June 19, 2020, 3:21am

Hey @Alhussein, the packages have to be added to the package.json inside nodes-base as you mentioned.n8n/package.json at master · n8n-io/n8n · GitHub.

If that does not help, simply get back to me.

Alhussein · June 19, 2020, 12:24pm

Thanks so much @RicardoE105 It work fine now…
Yesterday I did as you said but I didn’t know why it’s wont work…
Thanks Again @RicardoE105, @jan, and @leonardlin
I really appreciate your Quick replies