Limit the number of executions of kafka trigger

Questions
1

I need to limit the number of execution of kafka trigger workflow, There are over 1k of massages are written to kafka and i have workflow to handle each massage. This issue is this workflow get executed to all the massages in kafka at the same time.
Example: 1000 massages are in kafka = 1000 executions at the same time.

Is there a possibility to limit these execution to x number, example: 1000 massages are in kafka = 3 executions once done start the next patch of 3 executions till all the massages are cleared.

I was thinking of deploying another n8n special for this workflow, if there is a way to limit the number of workflows executed by this n8n node.

2

Hi @sultan, this is not a built-in functionality I am afraid. n8n doesn’t have a build in queuing functionality waiting with an execution until one or more previous executions have finished.

I’ve seem some hacky approaches to not run a workflow if there already is an active execution, but this would not allow you to process an item at a later time I am afraid. Might still be worth investigating and seeing if you can build the logic yourself (here would be an example workflow fetching the current executions using an undocumented API and thus unsupported API), but I don’t have any great hints on how to get started I am afraid.