Code Node Execution Hangs When Offloaded to Workers (GKE/Queue Mode)

I’m seeking assistance with a persistent hang issue when attempting to run workflows that contain a Code Node using the Manual Execute button.

When the environment variable OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS is set to true on the Main Node, all manual executions of workflows containing a Code Node (specifically Python) immediately hang in the UI.

Successful Configuration Checks

I’ve confirmed the following critical items are correctly configured and working:

  1. Network Connectivity (Task Broker): Workers and Runners can successfully reach the Primary Node’s Task Broker URI (Confirmed via nc -vz).

  2. Queue Connectivity: Worker nodes successfully connect to and communicate via the Redis queue

  3. Runner Registration: The Runner Pods successfully connect to the Main Node and register their services. The Main Node logs show: Registered runner "launcher-python"

  4. Hanging Log State: The Runner logs show the process is stalled waiting for the task offer:

    INFO [launcher:py] Waiting for launcher's task offer to be accepted...
    
    
  5. Environment Variables: All runner-related variables are consistent across Main, Workers, and Runners, including:

    • N8N_RUNNERS_MODE: external

    • N8N_RUNNERS_AUTH_TOKEN (Confirmed identical via clean secret reference).

    • N8N_RUNNERS_TASK_BROKER_URI: http://n8n-primary-service:5679

Has anyone encountered this specific failure with OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS in a K8s queue setup, and is there a known variable that needs to be explicitly set on the Worker or Main node to resolve this possible internal task routing issue?

Information on n8n setup

  • n8n version: 1.115.3
  • Database: postgres
  • n8n setup: Main, Worker, Runner
  • Running n8n via: GKE (Google’s Kubernetes)
  • Operating system: Standard N8N containers