Uncontrollable RabbitMQ

Questions
#1

Hi.
I finally got around to writing this question.
I set up a little workflow:

WorkFlow 
{
  "nodes": [
    {
      "parameters": {
        "queue": "test-rabbit",
        "options": {
          "jsonParseBody": true,
          "onlyContent": true
        }
      },
      "name": "RabbitMQ",
      "type": "n8n-nodes-base.rabbitmqTrigger",
      "typeVersion": 1,
      "position": [
        250,
        480
      ],
      "credentials": {
        "rabbitmq": "RabbitMQ n8n"
      }
    },
    {
      "parameters": {
        "functionCode": "var time = new Date().getTime() + (5 * 1000);\n\nwhile (new Date().getTime() <= time) {}\n\nreturn 'Done!';\n\n"
      },
      "name": "FunctionItem",
      "type": "n8n-nodes-base.functionItem",
      "typeVersion": 1,
      "position": [
        470,
        480
      ]
    }
  ],
  "connections": {
    "RabbitMQ": {
      "main": [
        [
          {
            "node": "FunctionItem",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

I added a simple time loop to simulate a 5-second run and left it off.
Put about 30 messages in the queue.
I started the flow, and what do I see?
I see 30 floo running, even though there’s only one listener in the rebbit.

image
image2020×1150 166 KB

How?
No new messages should be accepted until the current ones are handled in success or error.
This leads to a large uncontrollable load on the server and high LA.

Home made message broker
#2

n8n starts a new workflow execution for each message it receives. It would currently also technically not be impossible in n8n to wait until the workflow execution finishes and only then start the next one. What would be possible is to implement some kind of a rate limiting that for example just all X seconds a new execution gets started.

If you need that kind of behavior please create a new topic under “Feature Request”.

#3

Rebbit, on the other hand, works by the push mechanism.
While the listener is busy, Rebbit can not send him a new message.
For this reason, it seems to me not right to make a limit to start the listener by time.

#4

WebHook can wait for the last node to execute, why can’t RebbitTrigger do that?

#5

It has technical reasons that are sadly too complex and time-intensive to explain in this context right now.

Not saying it would not be possible to do, just saying that it would need core changes that would need more time and are so very unlikely to happen now, as we simply have many more high priority things to work on. We have just limited resources and we have to make sure to work on the things that matter most.

#6

I know what you mean.
I hope you will someday have time to get this task out of the backlog.
I think that in the queue nodes documentation, you should warn users that with a large flow of messages, the load will be uncontrollable.