Waiting Workflow Execution

When one worker picked the execution from the queue and start executing it and if he founds wait node or waiting in workflow and if it is more than 65 seconds then he puts into the waiting state and update its waitTill time, So my question is after the waiting period is over all these workflow will be executed by main n8n instance or the worker ?
I have gone through the code and found out that in the start.js we are creating the WaitTracker instance so my assumption is that they will be executed by main instance

Hey @aashish.jain, I just tested this and can confirm it’s a worker instance that would execute the waiting workflow. You can also see this in your debug logs, queue_test-n8n-1 is my main instance container and queue_test-n8nworker-1 would be the worker one in my example case:

queue_test-n8n-1  | 2022-05-27T08:04:41.596Z | debug    | Wait tracker querying database for waiting executions {"file":"WaitTracker.js","function":"getwaitingExecutions"}
queue_test-n8n-1  | 2022-05-27T08:04:41.642Z | debug    | Wait tracker found 1 executions. Setting timer for IDs: 147 {"file":"WaitTracker.js","function":"getwaitingExecutions"}
queue_test-n8n-1  | 2022-05-27T08:04:56.214Z | debug    | Wait tracker resuming execution 147 {"executionId":"147","file":"WaitTracker.js","function":"startExecution"}
queue_test-n8n-1  | Started with job ID: 146 (Execution ID: 147)
queue_test-n8nworker-1  | 2022-05-27T08:04:56.319Z | info     | Start job: 146 (Workflow ID: 4 | Execution: 147) {"file":"worker.js","function":"runJob"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.323Z | verbose  | Workflow execution started {"workflowId":4,"file":"WorkflowExecute.js","function":"processRunExecutionData"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.325Z | debug    | Start processing node "Wait" {"node":"Wait","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.325Z | debug    | Running node "Wait" started {"node":"Wait","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.325Z | debug    | Running node "Wait" finished successfully {"node":"Wait","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.326Z | debug    | Start processing node "Function" {"node":"Function","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.326Z | debug    | Running node "Function" started {"node":"Function","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | Done!
queue_test-n8nworker-1  | 2022-05-27T08:04:56.524Z | debug    | Running node "Function" finished successfully {"node":"Function","workflowId":4,"file":"WorkflowExecute.js"}
queue_test-n8nworker-1  | 2022-05-27T08:04:56.525Z | verbose  | Workflow execution finished successfully {"workflowId":4,"file":"WorkflowExecute.js","function":"processSuccessExecution"}

Thanks for sharing it, but is it good approach?
As worker is fetching execution from the queue and while executing as per the waiting he puts the execution in the waiting state and then the main instance fire a query to the database to fetch out all the waiting execution and again push into the queue for the execution.

Thanks for sharing it, but is it good approach?

That’s not a question I could possibly answer. But perhaps @sirdavidoff can share some more background on why n8n’s queue mode is designed the way it is.

Considering that the query has to run anyway (it is also very fast because it has an index on the field), and adding jobs to the queue and getting from the queue is not expensive, I do not see any issue.
Can you please elaborate on what problem you see or have?
The alternative would be to keep the execution active which would be resource wise potentially much more resource-intensive. And we have to have some value defined when it should not keep the execution active and rather write it to the DB. After all could the wait be years. No idea if 65 seconds is perfect, maybe it should be 20, 40, 80, or 100 instead but honestly do not think it matters that much. Sounded originally like a reasonable choice and the perfect value probably depends on the exact use case and need. But expect that the most people have either waits which are either under 65 seconds or much longer.

1 Like

@jan Thanks for the update, Actually we are finalising our POC regarding the n8n so we have asked some queries here. We haven’t seen any issue regarding the same.
Please let me know if my understanding is correct : In the queue mode worker picks the job for execution and if he finds the waiting time more than 65 seconds in the workflow node then he puts the same into waiting state, In parallel WaitTracker is running and firing the query to the db to check which is waiting execution and according to the same he is pushing into the queue again.

It works like this:

  • Wait time < 65 seconds
    • n8n will keep the execution active and continue after the set waiting time
  • Wait time >= 65 seconds:
    • the execution will be saved to the DB with a specific wake-up time
    • n8n checks all 60 seconds if there are any executions that should start in the next 70 seconds (and is not already waiting)
    • for each of those executions, a timer gets started
    • once that timer triggers, the execution gets started again (in queue mode that means it will push it to the queue)

I hope that is helpful!