SplitInBatches eats items

Hey there,

one of our workflows seems to eat items.
We created a loop with splitInBatches but it does only run through approximately 80% of the items created.

Do you guys have any clue what the reason might be? Three people tried to figure it out, and we were unable to find the problem.

Here you see the 199 iterations.
Screenshot 2020-09-02 at 11.19.37

Here you can see that at the same execution, it was actually 258 items.

We would appreciate any suggestion very much!

The loop defining part of the workflow is attached:

{
  "nodes": [
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        1302,
        543
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": true,
              "value2": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}"
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1820,
        730
      ]
    },
    {
      "parameters": {},
      "name": "End",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        2010,
        720
      ]
    },
    {
      "parameters": {
        "httpMethod": "POST",
        "path": "start",
        "options": {}
      },
      "name": "Webhook1",
      "type": "n8n-nodes-base.webhook",
      "typeVersion": 1,
      "position": [
        940,
        540
      ]
    },
    {
      "parameters": {
        "functionCode": "return items[0].json.body.text_1.split(',')\n.filter(item => item.trim())\n.map(item => { return {json: { id: parseInt(item.trim())}}; });\n\n\n\n"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1090,
        540
      ]
    },
    {
      "parameters": {},
      "name": "A LOTTA NODES ",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1630,
        540
      ]
    }
  ],
  "connections": {
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "A LOTTA NODES ",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "End",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Webhook1": {
      "main": [
        [
          {
            "node": "Function1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "A LOTTA NODES ": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

I changed the first node to create some dummy data (250 items) to be able to reproduce the issue. Anyway if I do that it works totally fine for me:

Here the workflow:

{
  "nodes": [
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        800,
        300
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": true,
              "value2": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}"
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1150,
        500
      ]
    },
    {
      "parameters": {},
      "name": "End",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1340,
        480
      ]
    },
    {
      "parameters": {
        "functionCode": "const returnItems = [];\nfor (let i=0;i< 250;i++) {\n  returnItems.push({json: {i}});\n}\nreturn returnItems;\n"
      },
      "name": "Function1",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        600,
        300
      ]
    },
    {
      "parameters": {},
      "name": "A LOTTA NODES ",
      "type": "n8n-nodes-base.function",
      "typeVersion": 1,
      "position": [
        1000,
        300
      ]
    }
  ],
  "connections": {
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "A LOTTA NODES ",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "End",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Function1": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "A LOTTA NODES ": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Hi @jan, i am facing a similar issue (with an extra layer of SplitInBatches) as stated above.

Here’s my workflow:

{

  "name": "412",

  "nodes": [

    {

      "parameters": {},

      "name": "Start",

      "type": "n8n-nodes-base.start",

      "typeVersion": 1,

      "position": [

        240,

        450

      ]

    },

    {

      "parameters": {

        "batchSize": 1,

        "options": {}

      },

      "name": "SplitInBatches",

      "type": "n8n-nodes-base.splitInBatches",

      "typeVersion": 1,

      "position": [

        1090,

        630

      ]

    },

    {

      "parameters": {

        "conditions": {

          "boolean": [

            {

              "value1": true,

              "value2": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}"

            }

          ]

        }

      },

      "name": "IF",

      "type": "n8n-nodes-base.if",

      "typeVersion": 1,

      "position": [

        1460,

        1030

      ]

    },

    {

      "parameters": {},

      "name": "End",

      "type": "n8n-nodes-base.noOp",

      "typeVersion": 1,

      "position": [

        1780,

        810

      ]

    },

    {

      "parameters": {

        "functionCode": "const returnItems = [];\nconst a = $node[\"SplitInBatches1\"].json.j;\nconsole.log(a);\nconsole.log($node[\"SplitInBatches1\"].context[\"noItemsLeft\"]);\nfor (let i=0;i<250;i++) {\n  let y = i+a;\n  //console.log('y: ',y);\n  returnItems.push({json: {'x':y}});\n}\nreturn returnItems;\n"

      },

      "name": "Function1",

      "type": "n8n-nodes-base.function",

      "typeVersion": 1,

      "position": [

        910,

        630

      ]

    },

    {

      "parameters": {},

      "name": "A LOTTA NODES ",

      "type": "n8n-nodes-base.function",

      "typeVersion": 1,

      "position": [

        1290,

        630

      ]

    },

    {

      "parameters": {

        "functionCode": "const returnItems = [];\nfor (let j=0;j<10;j++) {\n  returnItems.push({json: {j}});\n}\nreturn returnItems;\n"

      },

      "name": "Function",

      "type": "n8n-nodes-base.function",

      "typeVersion": 1,

      "position": [

        470,

        450

      ]

    },

    {

      "parameters": {

        "batchSize": 1,

        "options": {}

      },

      "name": "SplitInBatches1",

      "type": "n8n-nodes-base.splitInBatches",

      "typeVersion": 1,

      "position": [

        700,

        450

      ]

    }

  ],

  "connections": {

    "SplitInBatches": {

      "main": [

        [

          {

            "node": "A LOTTA NODES ",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "IF": {

      "main": [

        [

          {

            "node": "SplitInBatches1",

            "type": "main",

            "index": 0

          }

        ],

        [

          {

            "node": "SplitInBatches",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "Function1": {

      "main": [

        [

          {

            "node": "SplitInBatches",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "A LOTTA NODES ": {

      "main": [

        [

          {

            "node": "IF",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "Function": {

      "main": [

        [

          {

            "node": "SplitInBatches1",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "SplitInBatches1": {

      "main": [

        [

          {

            "node": "Function1",

            "type": "main",

            "index": 0

          }

        ]

      ]

    },

    "Start": {

      "main": [

        [

          {

            "node": "Function",

            "type": "main",

            "index": 0

          }

        ]

      ]

    }

  },

  "active": false,

  "settings": {

    "timezone": "Asia/Calcutta",

    "executionTimeout": -1

  },

  "id": "2"

}

The “Function” node here passes a list of 10 items.
Hence, expected result is 10 iterations of “SplitInBatches1”.
Whereas, as we can see, only 2.

Also, the node “SplitInBatches” does not run for the 2nd time.
I can’t find out why :confused:

n8n version: 0.95.1

I’ve used & modified your code example here as my actual use-case is a very similar structure.

Thanks.

Yes with two layers of SplitInBatches there are issues to be expected. What the node does is that it takes the data it receives, saves it all, and then releases a few of them every time it gets called. If there are no items left it simply does not return anything anymore and the execution stops. In your case do you expect the node to do two different things depending on the input data. If the data comes from the IF-Node you want it to behave like I described above, if it however receives data the second time from the Function1 Node you expect it to take the incoming data as new init-data.
For it to work, do you have to tell the node that it should “reset”. Here a post from a few months ago which describes how that works:

1 Like

Yes, it finally worked after i:

  1. set the reset option of SplitInBatches to {{($node[“SplitInBatches”].context[“noItemsLeft”])}}
  2. the IF condition to: {{($node[“SplitInBatches”].context[“noItemsLeft”])}} == True
1 Like