Redis TLS support for Bull in queue mode

The idea is:

Add Bull Redis configuration for supporting secure Redis(AWS Elsaticache) TLS connection.

My use case:

Scalable n8n cluster deployment in AWS

I think it would be beneficial to add this because:

Secure Bull Redis

Any resources to support this?

Patch

---
 packages/cli/config/schema.ts | 50 +++++++++++++++++++++++++++++++++++
 packages/cli/src/Queue.ts     | 35 +++++++++++++++++++++++-
 2 files changed, 84 insertions(+), 1 deletion(-)

diff --git a/packages/cli/config/schema.ts b/packages/cli/config/schema.ts
index 9d83f6e83..9f0c0ac50 100644
--- a/packages/cli/config/schema.ts
+++ b/packages/cli/config/schema.ts
@@ -344,6 +344,56 @@ export const schema = {
 					default: 10000,
 					env: 'QUEUE_BULL_REDIS_TIMEOUT_THRESHOLD',
 				},
+				secure: {
+					doc: 'Redis secure (TLS) flag',
+					format: Boolean,
+					default: false,
+					env: 'QUEUE_BULL_REDIS_SECURE',
+				},
+				tls: {
+					rejectUnauthorized: {
+						doc: 'Reject uanathorized certificate',
+						format: Boolean,
+						default: true,
+						env: 'QUEUE_BULL_REDIS_TLS_REJECTANUTHORIZEDT',
+					},
+					cert: {
+						doc: 'Cert in PEM format',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_CERT',
+					},
+					cert_file: {
+						doc: 'Cert file in PEM format, used only if "cert" value is not specified',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_CERT_FILE',
+					},
+					key: {
+						doc: 'Private key',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_KEY',
+					},
+					key_file: {
+						doc: 'Private key file',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_KEY_FILE',
+					},
+					ca: {
+						doc: 'Trusted CA certificates',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_CA',
+					},
+					ca_file: {
+						doc: 'Trusted CA certificates',
+						format: String,
+						default: '',
+						env: 'QUEUE_BULL_REDIS_TLS_CA_FILE',
+					},
+				},
 			},
 			queueRecoveryInterval: {
 				doc: 'If > 0 enables an active polling to the queue that can recover for Redis crashes. Given in seconds; 0 is disabled. May increase Redis traffic significantly.',
diff --git a/packages/cli/src/Queue.ts b/packages/cli/src/Queue.ts
index 9d4314e64..31b162799 100644
--- a/packages/cli/src/Queue.ts
+++ b/packages/cli/src/Queue.ts
@@ -1,5 +1,6 @@
 /* eslint-disable @typescript-eslint/no-unsafe-member-access */
 import Bull from 'bull';
+import fs from 'fs';
 import config from '../config';
 // eslint-disable-next-line import/no-cycle
 import { IBullJobData, IBullWebhookResponse } from './Interfaces';
@@ -8,6 +9,18 @@ import * as ActiveExecutions from './ActiveExecutions';
 // eslint-disable-next-line import/no-cycle
 import * as WebhookHelpers from './WebhookHelpers';
 
+interface SecureRedisConfig {
+	secure: boolean;
+	tls?: {
+		cert?: string | Buffer;
+		cert_file?: string;
+		key?: string | Buffer;
+		key_file?: string;
+		ca?: string | Buffer;
+		ca_file?: string;
+	};
+}
+
 export class Queue {
 	private activeExecutions: ActiveExecutions.ActiveExecutions;
 
@@ -17,7 +30,27 @@ export class Queue {
 		this.activeExecutions = ActiveExecutions.getInstance();
 
 		const prefix = config.getEnv('queue.bull.prefix');
-		const redisOptions = config.getEnv('queue.bull.redis');
+		const redisOptions = config.getEnv('queue.bull.redis') as SecureRedisConfig;
+		if (!redisOptions.secure) {
+			delete redisOptions.tls;
+		} else {
+			if (!redisOptions.tls) redisOptions.tls = {};
+			if (!redisOptions.tls.ca) {
+				if (redisOptions.tls.ca_file) {
+					redisOptions.tls.ca = fs.readFileSync(redisOptions.tls.ca_file);
+				}
+			}
+			if (!redisOptions.tls?.key) {
+				if (redisOptions.tls?.key_file) {
+					redisOptions.tls.key = fs.readFileSync(redisOptions.tls.key_file);
+				}
+			}
+			if (!redisOptions.tls?.cert) {
+				if (redisOptions.tls?.cert_file) {
+					redisOptions.tls.cert = fs.readFileSync(redisOptions.tls.cert_file);
+				}
+			}
+		}
 		// Disabling ready check is necessary as it allows worker to
 		// quickly reconnect to Redis if Redis crashes or is unreachable
 		// for some time. With it enabled, worker might take minutes to realize
-- 

Are you willing to work on this?

Also MemoryDB uses TLS and is a redis compatible DB server.

1 Like

This would be a nice feature for companies that require SSL and use Redis as an independent service, notably Azure Cache for Redis. As an example, until this feature is implemented, n8n cannot fit in our production environment.

We do support TLS now for Redis in queue mode which can be enabled by setting QUEUE_BULL_REDIS_TLS to true we don’t have support for settings certs or ignore issues when using self signed ones.

1 Like