Random Response Time Drops (n8n acting super slow)

Questions
1

I am building a workflow that uses OpenAI to write blog posts. This workflow passes through a lot of data. I scrape 10 web pages, create a database index, generate images with openAI. One execution can weight around 10MB. The workflow itself has around 20 nodes.

There are no error messages but sometimes the entire n8n platform response time can reach 20 seconds! It happens when I try opening the front page with the list of workflows, load executions, try executing a note, opening settings, basically any action can randomly take a super long time. The most annoying is waiting for “executing previous nodes” before a test node execution while all previous nodes already have data in them. After that huge delay, a chain of nodes can be executed quickly without any delay. And on the next request, here we go again…

It hasn’t happened before with the same workflow, it just became super slow one day. Sometimes it might execure/load page as normal, sometimes might take a solid 20 seconds. There are no other executions or intense tasks running on the server, I’ve been monitoring the server performance and the only bottleneck seems to be that one processor core randomly getting loaded 100% while others just chilling.

To be clear, there are no errors. Just the response time drop fairly often.

My assumption 1:
My prev executions are too heavy (although disk io nor memory were not the bottleneck) so I used these variables. Didn’t solve the issue.

EXECUTIONS_DATA_PRUNE=true
EXECUTIONS_DATA_MAX_AGE=336
EXECUTIONS_DATA_PRUNE_MAX_COUNT=1000

My assumption 2:
Due to 10 web pages cached while test execution, the platform has to send them back and forth when executing the next node in test mode. But if that’s true why opening settings can trigger that enormous delay?

Worth noting that when I use production webhooks, everything seems okay. Only the UI part that’s struggling (with a number of devices tested).

I’m using Docker, 4 core CPU, 4GM RAM and 20GB storage

Thank you in advance!

edit:

n8n version - 1.18
database - sqlite
OS - linux

2

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:
4

There are no error messages but sometimes the entire n8n platform response time can reach 20 seconds! It happens when I try opening the front page with the list of workflows, load executions, try executing a note, opening settings, basically any action can randomly take a super long time.

Hi @First_Rank, this sounds like the database could indeed be the problem, especially when this slowness corresponds to a workflow running in the background.

If so, you might want to consider using PostgreSQL instead of SQLite which will typically perform better in most setups. If this is not an option you could consider enabling WAL on your SQLite database.

Worth noting that when I use production webhooks, everything seems okay. Only the UI part that’s struggling (with a number of devices tested).

If this is only happening during manual executions this could suggest the amount of data during your test executions is simply too much for your browser to render. Are you also seeing this problem when processing smaller datasets (like one website)?

5

Thank you, Jam. I’ve replaced DB with Postgres, added a worker and redis and everything’s as smooth as butter! Turns out I’ve been using a demo version setup.

If someone interested, here’s my docker-compose. If you gonna use it, please keep in mind that I’m not opening ports but using an external network.
DB migration commands can be found here:
CLI commands | n8n Docs

version: '3.8'

services:
  x-shared: &shared
    image: n8nio/n8n:ai-beta
    restart: always
    volumes:
      - n8n_storage:/home/node/.n8n
    networks:
      - n8n_network
      - proxy_manager_default
    environment:
      - QUEUE_BULL_REDIS_HOST=redis
      - QUEUE_HEALTH_CHECK_ACTIVE=true
      - EXECUTIONS_MODE=queue
      - N8N_HOST=${HOST}
      - NODE_ENV=production
      - WEBHOOK_URL=https://${HOST}
      - N8N_RELEASE_TYPE=beta
      - NODE_FUNCTION_ALLOW_EXTERNAL=*
      - EXECUTIONS_DATA_PRUNE=true
      - EXECUTIONS_DATA_MAX_AGE=336
      - EXECUTIONS_DATA_PRUNE_MAX_COUNT=1000
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_DATABASE=n8n
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_USER=n8n
      - DB_POSTGRESDB_PASSWORD=${DB_PASS}
    depends_on:
      redis:
        condition: service_healthy
      postgres:
        condition: service_healthy
    links:
      - postgres
      - redis

  postgres:
    image: postgres:16.1-alpine3.18
    restart: always
    environment:
      - POSTGRES_DB=n8n
      - POSTGRES_USER=n8n
      - POSTGRES_PASSWORD=${DB_PASS}
    volumes:
      - db_storage:/var/lib/postgresql/data
    networks:
      - n8n_network
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -h localhost -U n8n -d n8n']
      interval: 5s
      timeout: 5s
      retries: 10

  redis:
    image: redis:6-alpine
    restart: always
    volumes:
      - redis_storage:/data
    networks:
      - n8n_network

    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 5s
      timeout: 5s
      retries: 10

  n8n:
    <<: *shared


  n8n-worker:
    <<: *shared
    command: worker
    depends_on:
      - n8n

networks:
  n8n_network:
  proxy_manager_default:
    external: true

volumes:
  db_storage:
  n8n_storage:
  redis_storage: