Merge and If nodes behavior

I am having some issues with the Merge node. I found a workaround but I want to share my scenarios to see if I can simplify the solution.

This is what I want to accomplish:

  1. Get some initial IDs
  2. “Set Global IDs” and “Set Custom IDs” nodes in example
  3. Execute a pre-processing command which generate multiple items
  4. “Get Items Function” node in example
  5. Each of the items from the previous step must be processed one by one (given the nature of the process, they cannot be processed in parallel,) so I use a SplitInBatches node to loop through them
  6. “Slow Serial Function” node in example
  7. I added an “IF” node to detect when all items have been processed
  8. After processing all the items I need to run a post-processing command, which needs as input the original data
  9. “Post-processing Command” node in example

The issue is with the “Merge for Post-processing” node, which is expected to execute only after the slow process consumed all items, so it receives as an incoming node the “IF” node when the “SplitInBatches” node has no additional data.

Scenario 1

  • Get Items Function” node is input1 of the “Merge for Post-processing” node.
  • IF” node is input2 of the “Merge for Post-processing” node.
  • “Merge for Post-processing” node is configured with “Pass-through” mode set to input1.

Issue: The final processing node is executed BEFORE all the items have been processed in the loop.

Code below:

{
  "name": "Merge Demo Scenario 1",
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "globalId1",
              "value": "g1"
            },
            {
              "name": "globalId2",
              "value": "g2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Global IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        480,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "number": [],
          "string": [
            {
              "name": "customId1",
              "value": "c1"
            },
            {
              "name": "customId2",
              "value": "c2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Custom IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        710,
        300
      ]
    },
    {
      "parameters": {
        "command": "echo \"discard this\""
      },
      "name": "Execute Other Required Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1030,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "return [\n  {\n    json: {\n      itemId: \"id100\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id101\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id102\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id103\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id104\",\n    },\n  },\n];"
      },
      "name": "Get Items Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1400,
        300
      ]
    },
    {
      "parameters": {
        "mode": "multiplex"
      },
      "name": "Merge Items with IDs",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        920,
        680
      ]
    },
    {
      "parameters": {
        "batchSize": 1
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        410,
        910
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        830,
        1160
      ]
    },
    {
      "parameters": {
        "mode": "passThrough"
      },
      "name": "Merge for Post-processing",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        1500,
        860
      ]
    },
    {
      "parameters": {
        "executeOnce": false,
        "command": "=echo \"Processed {{$node[\"Merge for Post-processing\"].json[\"itemId\"]}}\""
      },
      "name": "Post-processing Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1810,
        860
      ],
      "color": "#F90538"
    },
    {
      "parameters": {
        "functionCode": "const waitInSeconds = 1;\nitems[0].json.completed = true;\n\nreturn new Promise((resolve) => {\n // Simulate long runnig process\n setTimeout(() => {\n   resolve(items);\n }, waitInSeconds * 1000) \n});\n"
      },
      "name": "Slow Serial Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        650,
        910
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Set Global IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Global IDs": {
      "main": [
        [
          {
            "node": "Set Custom IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Custom IDs": {
      "main": [
        [
          {
            "node": "Execute Other Required Command",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Execute Other Required Command": {
      "main": [
        [
          {
            "node": "Get Items Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get Items Function": {
      "main": [
        [
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge Items with IDs": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Slow Serial Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 1
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge for Post-processing": {
      "main": [
        [
          {
            "node": "Post-processing Command",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Slow Serial Function": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": false,
  "settings": {},
  "id": "12"
}

Scenario 2

  • IF” node is input1 of the “Merge for Post-processing” node.
  • Get Items Function” node is input2 of the “Merge for Post-processing” node.
  • “Merge for Post-processing” node is configured with “Pass-through” mode set to input2.

Issue: The final processing node is NOT executed once the loop ends.

Code below:

{
  "name": "Merge Demo Scenario 2",
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "globalId1",
              "value": "g1"
            },
            {
              "name": "globalId2",
              "value": "g2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Global IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        480,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "number": [],
          "string": [
            {
              "name": "customId1",
              "value": "c1"
            },
            {
              "name": "customId2",
              "value": "c2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Custom IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        710,
        300
      ]
    },
    {
      "parameters": {
        "command": "echo \"discard this\""
      },
      "name": "Execute Other Required Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1030,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "return [\n  {\n    json: {\n      itemId: \"id100\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id101\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id102\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id103\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id104\",\n    },\n  },\n];"
      },
      "name": "Get Items Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1400,
        300
      ]
    },
    {
      "parameters": {
        "mode": "multiplex"
      },
      "name": "Merge Items with IDs",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        920,
        680
      ]
    },
    {
      "parameters": {
        "batchSize": 1
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        410,
        910
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        830,
        1160
      ]
    },
    {
      "parameters": {
        "mode": "passThrough",
        "output": "input2"
      },
      "name": "Merge for Post-processing",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        1500,
        860
      ]
    },
    {
      "parameters": {
        "executeOnce": false,
        "command": "=echo \"Processed {{$node[\"Merge for Post-processing\"].json[\"itemId\"]}}\""
      },
      "name": "Post-processing Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1810,
        860
      ],
      "color": "#F90538"
    },
    {
      "parameters": {
        "functionCode": "const waitInSeconds = 1;\nitems[0].json.completed = true;\n\nreturn new Promise((resolve) => {\n // Simulate long runnig process\n setTimeout(() => {\n   resolve(items);\n }, waitInSeconds * 1000) \n});\n"
      },
      "name": "Slow Serial Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        650,
        910
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Set Global IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Global IDs": {
      "main": [
        [
          {
            "node": "Set Custom IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Custom IDs": {
      "main": [
        [
          {
            "node": "Execute Other Required Command",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Execute Other Required Command": {
      "main": [
        [
          {
            "node": "Get Items Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get Items Function": {
      "main": [
        [
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Merge Items with IDs": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Slow Serial Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge for Post-processing": {
      "main": [
        [
          {
            "node": "Post-processing Command",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Slow Serial Function": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": false,
  "settings": {},
  "id": "13"
}

Workaround

I managed to get the final node to execute after the loop using Scenario 1 but adding a dummy function node that return no items in between the “IF” and the “Merge for Post-processing” node.

Is the behavior of the “Merge for Post-processing” node in Scenarios 1 and 2 expected? This is what I am assuming: 1) The merge node will not wait for the IF node if set to input2; and 2) the merge node will not execute if the IF node is set to input1. If both are expected, it may be worth to include more details in the documentation.

Any suggestions to improve the workflow are appreciated :slight_smile:

Thanks!

PS: Below is the code of the workaround:

{
  "name": "Merge Demo with Dummy Function",
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "globalId1",
              "value": "g1"
            },
            {
              "name": "globalId2",
              "value": "g2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Global IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        480,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "number": [],
          "string": [
            {
              "name": "customId1",
              "value": "c1"
            },
            {
              "name": "customId2",
              "value": "c2"
            }
          ]
        },
        "options": {}
      },
      "name": "Set Custom IDs",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        710,
        300
      ]
    },
    {
      "parameters": {
        "command": "echo \"discard this\""
      },
      "name": "Execute Other Required Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1030,
        300
      ]
    },
    {
      "parameters": {
        "functionCode": "return [\n  {\n    json: {\n      itemId: \"id100\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id101\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id102\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id103\",\n    },\n  },\n  {\n    json: {\n      itemId: \"id104\",\n    },\n  },\n];"
      },
      "name": "Get Items Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1400,
        300
      ]
    },
    {
      "parameters": {
        "mode": "multiplex"
      },
      "name": "Merge Items with IDs",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        1290,
        680
      ]
    },
    {
      "parameters": {
        "batchSize": 1
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        410,
        910
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        790,
        1140
      ]
    },
    {
      "parameters": {
        "mode": "passThrough"
      },
      "name": "Merge for Post-processing",
      "type": "n8n-nodes-base.merge",
      "typeVersion": 1,
      "position": [
        1500,
        860
      ]
    },
    {
      "parameters": {
        "functionCode": "return [];"
      },
      "name": "Dummy Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1160,
        1120
      ]
    },
    {
      "parameters": {
        "executeOnce": false,
        "command": "=echo \"Processed {{$node[\"Merge for Post-processing\"].json[\"itemId\"]}}\""
      },
      "name": "Post-processing Command",
      "type": "n8n-nodes-base.executeCommand",
      "typeVersion": 1,
      "position": [
        1810,
        860
      ],
      "color": "#F90538"
    },
    {
      "parameters": {
        "functionCode": "const waitInSeconds = 1;\nitems[0].json.completed = true;\n\nreturn new Promise((resolve) => {\n // Simulate long runnig process\n setTimeout(() => {\n   resolve(items);\n }, waitInSeconds * 1000) \n});\n"
      },
      "name": "Slow Serial Function",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        650,
        910
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Set Global IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Global IDs": {
      "main": [
        [
          {
            "node": "Set Custom IDs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Custom IDs": {
      "main": [
        [
          {
            "node": "Execute Other Required Command",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Execute Other Required Command": {
      "main": [
        [
          {
            "node": "Get Items Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Get Items Function": {
      "main": [
        [
          {
            "node": "Merge Items with IDs",
            "type": "main",
            "index": 0
          },
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge Items with IDs": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Slow Serial Function",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Dummy Function",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge for Post-processing": {
      "main": [
        [
          {
            "node": "Post-processing Command",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Dummy Function": {
      "main": [
        [
          {
            "node": "Merge for Post-processing",
            "type": "main",
            "index": 1
          }
        ]
      ]
    },
    "Slow Serial Function": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": false,
  "settings": {},
  "id": "11"
}