Highly Available n8n

Thank you for developing this wonderful platform. It is great to see the project evolving.

I want to ensure that my self-hosted n8n endpoints are always available. I have read the following:

High availability for self-hosted n8n - Questions - n8n
Several n8n instances (main mode) on Kubernetes - Feature Requests - n8n

I am not sure if k8s is the answer. Obviously, a HA deployment would need a multi-cloud, multi-region architecture. Queue mode seems like the best (only) way to use all of the available cores on a server. The worker and webhook processor instances can be deployed on many servers which ensures redundancy. Redis can also be deployed in a highly-available manner. My concern is that the [Queue Mode] (Configuring queue mode | n8n Docs) instructions seem to contemplate a single Main instance and it is unclear how this would fail over. While I could deploy several Main instances and treat them as isolated islands, I am wondering if I can deploy them with a shared database backend to ensure that they are all in sync.

If I use Postgres as the shared backend, this leaves Postgres as the single point of failure. HA in Postgres is hard. Is CockroachDB a viable alternative to Postgres? CockroachDB uses the [Postgres wire protocol] (Migrate from PostgreSQL) and has only a few [syntax differences] (PostgreSQL Compatibility) from Postgres.

If CockroachDB will work as n8n’s backend, it may be the missing piece of the HA puzzle.

Edit: I have since come across the [Supported databases] (Supported databases and settings | n8n Docs) page which mentions CockroachDB. If CockroachDB indeed works, perhaps it should be mentioned/suggested in the Queue Mode documentation. Has anyone else used CockroachDB with n8n?

1 Like

Hi @BB9234movwZ4T5c, adding support for a new database requires changes to the n8n code base and subsequently testing/updating every new release (n8n usually has weekly releases).

This is a lot of work and it’s very easy to overlook some of the consequences a specific change might have for a given database. This is also the reason we ourselves have deprecated MySQL and MariaDB as backend databases in the past.

So I’d very much suggest you consider using PostgreSQL instead and consider configuring replication. PostgreSQL discusses high availability in their documentation here.

+1 for CockroachDB support. I too am looking at a distributed setup and this would be a dream setup because it would provide write capabilities on each node.

I appreciate the input, even if it’s perhaps not the news we’d hoped for, but I would like to understand the discrepancy between this response and the official docs which state:

If you can’t use any of the supported databases for some reason and you can code, consider submitting a pull request.

Source: Supported databases and settings | n8n Docs

It’s not clear from this quote on its own, but it applies to databases supported by TypeORM of which CockroachDB is one.
Furthermore it’s not merely another supported database, it has the same wire protocol so it’s very close to postgres.

The docs suggest that n8n is open to a PR/contribution to add support for this.
I for one would be interested in working on this. But this comment makes me wonder if such a PR would be welcomed.

Would the n8n team accept such a PR? Perhaps without going as far as officially supporting it (vs merely accepting changes that make it at possible).

Thanks in advance for your feedback

Joost

1 Like

Hey @joostdecock,

Welcome to the community :tada:

Anyone can put in a pull request for consideration but when it comes to adding another database type there is the ongoing maintenance tasks that need to be considered as well. We would need to run the automated tests on new releases and then deal with any support issues that come in if for some reason it doesn’t work.

At the moment there are only a couple of requests for CockroachDB so I suspect if you did put the PR in it likely wouldn’t be merged as we are not seeing the demand for it.

I’ll throw my voice in here. We’d definitely be using Cockroach if well supported.

Finding containerised postgres + repmgr + pgpool is OK-ish, but not ideal. Mainly the drag of a full clone when a node restarts :skull:. This is an area I actually prefer MySQL/MariaDB :grimacing:

1 Like

FWIW, If you’re comfortable hosting at AWS you can use Amazon’s Serverless Postgres clone as your backend. It’s at least as reliable as any other high availability SQL database / cluster, especially in a multi-AZ configuration. And it autoscales capacity, so the DB is always responsive no matter what the load. You do have some downtime for version upgrades, but if uptime is that critical then you probably have a blue/green configuration and you do your DB upgrades and testing before you switch that color into production.

1 Like

I’ve been looking at Litefs, BedrockDB and other distributed tools built on top of sqlite as an alternative to cockroachDB and other more exotic (read not natively supported by TypeORM) databases.

Nothing to report back yet, but they look promising.

Since Turso is fully compatible with SQLite, n8n already supports SQLite, and Turso allows you to install a local replica, it seems that Turso might be a viable option to provide geographic redundancy with the performance benefits of being able to read from a local replica (assuming that the delay in writing to the cloud and updating the local replica does not create any problems or latency).

We do only specifically support SQLite or Postgres so if you are using services that are “compatible” you may run into issues so I wouldn’t recommend this approach.

Hi Jon,

For your reference:

Hey @BB9234movwZ4T5c,

I still don’t recommend it, we have seen users try to run n8n on “compatible” databases before and it failed like Cockroach so officially we only support Postgres and SQLite.

In the future we may change this and support other database options but it isn’t something that will happen soon.

Thanks @Jon,

I just wanted to make sure that it is on your radar.

Hey @BB9234movwZ4T5c,

It was on the radar and we looked at using it but not for queue mode, we then decided against it for now (not sure why).