Doubt about Elasticsearch node (Clusters)

Describe the issue/error/question

Hello,

Is it possible to configure n8n to connect to an ELK Cluster? I’ve experimented with just one node, and was planning on connecting n8n to an ELK Cluster.

I’ve tried using “,” (commas) on the URL string, but it didnt work.

Kind regards,
Rob

Information on your n8n setup

  • n8n version:
  • Database you’re using (default: SQLite):
  • Running n8n with the execution process [own(default), main]:
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]:

The Elasticserach node does use the regular REST API. Sadly do not know that much about Elasticsearch Clusters but I would have assumed you can use any node to connect to. If you want to connect via an IP address that changes, you would have to set up a DNS server or similar that takes care of the resolution.

Hi Jan,

Thanks for reaching back.

Perhaps it would be easier if we forget about the “Cluster” word. Let’s just say I have 3 servers running ELK, that are syncronized. My idea was to allow n8n to have a 3 server pool of options. If one was down, n8n could just use another one.

I’m more of a Python user, so I’ll use a snippet of Python to explain what I’m doing on other scripts, and that would be cool to replicate, if is possible, in n8n. Basically I’m using multiple servers, on the Elasticsearch API Client, and these servers will be contacted. This offers redundancy for the script in case one of the nodes fail. Obviously with just one node, if the node fails, any scripts running would fail due to the server being down.

        es = Elasticsearch(
                ["https://server1:9200", "https://server2:9200", "https://server3.com:9200"],
                http_auth=(user, pass),
                scheme="https"
        )

That being said, since I have 3 ELK Servers but n8n is just using one, if something happens to that server, all the Workflows using ELK would fail. I take it by your answer that perhaps it’s not allowed to have multiple URLs on the ELK Credential configuration page. (Tried with commas, but it didn’t work).

Your sugestion is viable, however, ELK implements its own kind of Load Balancing, so it would be redundant to create another Load Balancer.

Nevertheless, it may end up being the alternative, in order to be asured that if something came up on one of the servers, n8n would still manage to execute all Workflows.

Kind regards,
Rob

Hey @robjennings,

What about using 3 nodes one for each and do a response check to see if they are working and use the first available one?

You could have the first one and under settings for the node use continue on error and always output data then check the response and if it failed go to the second and do the same thing.

It isn’t pretty but it would do the job and you could link all the true branches on the if statement to check if anything came back to the same node to continue the workflow.

1 Like