N8n startup issue issues (migration?)

[adi]

Describe the problem/error/question

I run n8n in Kubernetes, with 3 deployments (main, webhook, and workers).
The main process starting up is failing now that I am testing out the latest image.

I was using 0.x versions with MariaDB and all was okay.

Now I have moved to PostgreSQL 15.4 and [email protected]

The startup seems to start okay, it connects to the DB, creates the schema etc, created entries in redis, but then at one point I get an error (shared below)

What is the error message (if any)?

Stopping n8n...

Error: There was an error shutting down n8n.

TypeError: Cannot read properties of undefined (reading 'removeAllQueuedWorkflowActivations')

I have tried quite a lot of debugging, but not sure what is causing this… The databases are clean with no entries, so there shouldn’t be anything to ‘remove’

Information on your n8n setup

• n8n version: 1.6.1
• Database (default: SQLite): PostgreSQL 15.4
• n8n EXECUTIONS_PROCESS setting (default: own, main): main - queue mode
• Running n8n via (Docker, npm, n8n cloud, desktop app): Docker (via Kubernetes)

[adi]

Some additional information:

• I enabeled the DB logging and reduced overall logging to DEBUG level, and it seems the migrations are not completing correctly

Question: What is the supported version of PostgreSQL and what permissions should the n8n db user have?
(To be fair I tried giving it superuser as well, still the same error)

The logs:

UserSettings were generated and saved to: /home/node/.n8n/config

2023-09-12T15:16:35.520Z | debug    | Lazy Loading credentials and nodes from n8n-nodes-base "{\n  credentials: 346,\n  nodes: 436,\n  file: 'DirectoryLoader.js',\n  function: 'loadAll'\n}"

query: SELECT * FROM current_schema()

query: CREATE EXTENSION IF NOT EXISTS "uuid-ossp"

query: SELECT version();

query: CREATE SCHEMA IF NOT EXISTS n8n

query: SET search_path TO n8n,public;

query: SELECT 1

query: SELECT 1

query: SELECT 1

query: SELECT 1

query: SELECT 1
(and repeating like this forever basically...)

[Jon]

Hey @adi,

I am not sure which versions we support but I have seen versions 12 to 15 work without issue. The n8n db user would need the usual set of permissions to manage the database but it doesn’t look like that is the issue.

Can you share your configuration so we can take a look? For now it may be worth trying with a single n8n instance pointing to the database if that works then add the workers and set it to queue mode to see how that goes.

[adi]

Hey @Jon

Okay, here is the list of env vars being set

- { name: "N8N_ENCRYPTION_KEY", value: "" }
- { name: "DB_TYPE", value: "postgresdb" }
- { name: "DB_POSTGRESDB_HOST", value: "" }
- { name: "DB_POSTGRESDB_PORT", value: "5432" }
- { name: "DB_POSTGRESDB_DATABASE", value: "" }
- { name: "DB_POSTGRESDB_SCHEMA", value: "n8n" }
- { name: "DB_POSTGRESDB_USER", value: "" }
- { name: "DB_POSTGRESDB_PASSWORD", value: "" }
- { name: "N8N_EDITOR_BASE_URL", value: "" }
- { name: "N8N_HOST", value: "" }
- { name: "N8N_LISTEN_ADDRESS", value: "0.0.0.0" }
- { name: "EXECUTIONS_MODE", value: "queue"}
- { name: "QUEUE_BULL_REDIS_HOST", value: ""}
- { name: "QUEUE_BULL_REDIS_PORT", value: "6379"}
- { name: "QUEUE_BULL_REDIS_PASSWORD", value: ""}
- { name: "QUEUE_BULL_REDIS_DB", value: ""}
- { name: "QUEUE_BULL_PREFIX", value: "n8n" }
- { name: "N8N_DISABLE_PRODUCTION_MAIN_PROCESS", value: "true"}
- { name: "WEBHOOK_URL", value: ""}
- { name: "N8N_LOG_OUTPUT", value: "console" }
- { name: "N8N_LOG_LEVEL", value: "debug" }
- { name: "N8N_LOG_FILE_MAXSIZE", value: "20" }
- { name: "N8N_LOG_FILE_MAXCOUNT", value: "60" }
- { name: "VUE_APP_URL_BASE_API", value: "" }
- { name: "N8N_HIRING_BANNER_ENABLED", value: "false" }
- { name: "N8N_DIAGNOSTICS_ENABLED", value: "false" }
- { name: "N8N_PORT", value: "80" }
- { name: "DB_LOGGING_ENABLED", value: "true" }
- { name: "DB_LOGGING_OPTIONS", value: "all" }

I also updated how I was running the deployment, based it similar to how it is shown here: https://github.com/n8n-io/n8n-kubernetes-hosting/blob/7ef567c315c9c3af8cbcc914a33f564c3fa72242/n8n-deployment.yaml#L28

[adi]

I just tried switching to regular execution mode from queue, removed the redis dependency, other workers/ webhooks and still the ‘SELECT 1’ loop is the same.

[adi]

Yeah after alot! of debugging, found the issue.

Was due to a switch from bullseye to alpine image, and the fact that the node user was not able to listen via port 80. Switching to 5678 works fine.

Also, the ‘SELECT 1’ which seemed like a hang message, seems to be just a casual poll / heartbeat query.

All good now.