Should tasks run from “Execute Workflow” be run in workers, even if the “source” task is run interactively?
I just rebuilt my n8n install to use queue mode. As a test, I interactively ran a task that calls “Execute Workflow” about 40 times. If I understand queue mode correctly, the manually triggered workflow should run on the main instance. (And when I look at the executions list, there’s a little flask next to that execution.)
But shouldn’t the executed workflows would run on the worker(s)? When I look at the executions list, there’s no flask next to those executions.
When I look on Cloudwatch, my workers have almost no CPU. It looks like everything is running on the main instance.
Since this is a new setup I’m not sure if I misconfigured something or if I’m misunderstanding when the workers come into play.
Hi @Lee_S, it looks like your workflow might not require lots CPU resources itself, so the majority of your usage might actually come from using the UI to (for example) query your execution data.
If you want to be sure your workers are actually processing anything, perhaps you want to enable debug logging and take a look at your worker logs while running a workflow in production mode (= not manually executed)? You should see entries such as the ones below confirming your worker is indeed picking up work
With regards to the sub-workflows of a manually triggered parent I don’t think they would run on the workers. They are technically executed as part of the parent, but perhaps @krynble can confirm this with certainty?
I think the main thing here is that the executions do not show that icon for manual executions eventhough they are actually executed by a manually executed workflow.
I have noticed this as well in the past and it would indeed be good to have them also show the icon for manual executions.
As for the manual execution of subworkflows, as far as I have seen the subworkflows are indeed executed on main if it is a manual executions. And production workflow executions are also ran on the same worker as the main workflow as far as I have seen actually.
So, the “child executions” as we call them, when you use the “Execute workflow” node, are executed always in the same instance as the “parent execution”. This effectively means that manual executions are executed from start to end on the normal n8n instance, but if you make use of a “Schedule trigger” node, then the whole execution will take place in one of the workers (both parent and child executions will all happen in the same worker).
This effectively means that child executions are not sent to the queue to be picked up by another worker but rather run as an extension of the original execution.
Curious fact, the flask icon is shown only for the parent execution since this one has the “mode” attribute set to manual while the child executions have the execution mode as integrated, so that’s why you don’t see the vial.
Would it be possible to make manual sub flows also show with the vial? Of course if it is a lot work then we don’t want the team to work on this but if it is an easy fix that would be helpful in making things more clear in the executions.
I’ve been looking at my logs and am also seeing the same behavior. Manually started jobs and the children called with “Execute workflow” all run on my “main” instance.
So is it correct to say that if we want a job to “fan out” a number of children that run in parallel they need to be called with a webhook instead of with execute workflow?
So is it correct to say that if we want a job to “fan out” a number of children that run in parallel they need to be called with a webhook instead of with execute workflow?
Yes, if you want each sub-workflow to (potentially) run on a separate worker.
