How to Iterate Over Items using Split In Batches

Hello,
I am facing some problems in the flow.

Case: I am trying to update hotel prices using http call for next 365 days from sheets data . I am taking the values for date and prices from spreadsheet. Used set node to set the value of date and price. Since there are 365 items in set node I am using split in batches to iterate every item separately so that I don’t hit the API limit. How should I make sure all 365 days prices have been updated because in the flow shown below only 1 day price is getting updated. On looping the output of http node to input of split in batches my flow kept running and iterated over 500+ items which is not possible.
Screenshot attached below for reference.
@harshil1712

Hey @harshil1712
I tried connecting the last node of my flow to split batch but same item is getting updated. Just to check the data flow I added Gsheets in the end to visualise data flow.
I have attached screenshot below of my flow and Gsheet.


Set1 Node

@Mustafa_Ali The SplitInBatches always has to be used in conjunction with an IF node, like this:

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        450,
        300
      ]
    },
    {
      "parameters": {
        "operation": "getAllPeople",
        "returnAll": true
      },
      "name": "Customer Datastore",
      "type": "n8n-nodes-base.n8nTrainingCustomerDatastore",
      "typeVersion": 1,
      "position": [
        650,
        300
      ]
    },
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        850,
        300
      ]
    },
    {
      "parameters": {
        "values": {
          "string": [
            {
              "name": "misc",
              "value": "={{$json[\"name\"]}}, {{$json[\"notes\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        1050,
        300
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1270,
        510
      ]
    },
    {
      "parameters": {},
      "name": "NoOp",
      "type": "n8n-nodes-base.noOp",
      "typeVersion": 1,
      "position": [
        1470,
        490
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Customer Datastore",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Customer Datastore": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Set",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "NoOp",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Is this flow correct? Because it ran for 2 records only. I have total 365 items in my sheets.

Could you make sure that the IF node is referencing the SpliInBatches node correctly?

image

If it is, then do make sure that the Set1 node is providing the items properly.

Hey @Mustafa_Ali,

Can you please share your workflow? I tested out the same logic and it is working fine for me.

@shrey-42, you don’t have to necessarily use the IF node. It depends on the use case. If you want to continue the workflow after all the batches are processed, that’s where you can use the IF node.

Won’t be able to share the complete work flow dude to data sensitivity. Share some part my flow.

{
  "nodes": [
    {
      "parameters": {
        "batchSize": 1,
        "options": {
          "reset": true
        }
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        1530,
        350
      ]
    },
    {
      "parameters": {
        "amount": 0.07,
        "unit": "seconds"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.wait",
      "typeVersion": 1,
      "position": [
        2190,
        110
      ],
      "webhookId": "799d26d3-2223-40c8-87c1-d9702bab1036"
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "number": [
            {
              "name": "Price",
              "value": "={{$node[\"Google Sheets2\"].json[\"Destination Property Price\"]}}"
            }
          ],
          "string": [
            {
              "name": "Date",
              "value": "={{$node[\"Google Sheets2\"].json[\"Date\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set2",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        1750,
        350
      ]
    },
    {
      "parameters": {
        "conditions": {
          "boolean": [
            {
              "value1": "={{$node[\"SplitInBatches\"].context[\"noItemsLeft\"]}}",
              "value2": true
            }
          ]
        }
      },
      "name": "IF",
      "type": "n8n-nodes-base.if",
      "typeVersion": 1,
      "position": [
        1990,
        120
      ]
    }
  ],
  "connections": {
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Set2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set2": {
      "main": [
        [
          {
            "node": "IF",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "IF": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

@harshil1712 Thanks for the info! Didn’t know about this.

1 Like

Hey @Mustafa_Ali,

In your Split In Batches node, you had the Reset option set to true. This option resets and starts splitting the incoming items from the start, as a result every time it will return the first item in your case. The workflow will help you in achieving what you want to do:

{
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "name": "SplitInBatches",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 1,
      "position": [
        650,
        300
      ]
    },
    {
      "parameters": {
        "amount": 0.07,
        "unit": "seconds"
      },
      "name": "Wait",
      "type": "n8n-nodes-base.wait",
      "typeVersion": 1,
      "position": [
        1150,
        100
      ],
      "webhookId": "799d26d3-2223-40c8-87c1-d9702bab1036"
    },
    {
      "parameters": {
        "keepOnlySet": true,
        "values": {
          "number": [
            {
              "name": "Singe Price Unit",
              "value": 500
            }
          ],
          "string": [
            {
              "name": "Date",
              "value": "={{$json[\"Date\"]}}"
            }
          ]
        },
        "options": {}
      },
      "name": "Set2",
      "type": "n8n-nodes-base.set",
      "typeVersion": 1,
      "position": [
        850,
        300
      ]
    },
    {
      "parameters": {
        "authentication": "oAuth2",
        "options": {
          "valueRenderMode": "FORMATTED_VALUE"
        }
      },
      "name": "Google Sheets",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 1,
      "position": [
        450,
        300
      ],
      "credentials": {
        "googleSheetsOAuth2Api": "Google [email protected]"
      }
    },
    {
      "parameters": {
        "authentication": "oAuth2",
        "operation": "update",
        "key": "Date",
        "options": {
          "valueRenderMode": "FORMATTED_VALUE"
        }
      },
      "name": "Google Sheets1",
      "type": "n8n-nodes-base.googleSheets",
      "typeVersion": 1,
      "position": [
        1050,
        300
      ],
      "credentials": {
        "googleSheetsOAuth2Api": "Google [email protected]"
      }
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Google Sheets",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "SplitInBatches": {
      "main": [
        [
          {
            "node": "Set2",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set2": {
      "main": [
        [
          {
            "node": "Google Sheets1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Google Sheets": {
      "main": [
        [
          {
            "node": "SplitInBatches",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Google Sheets1": {
      "main": [
        [
          {
            "node": "Wait",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Hey @harshil1712

I am using the same logic set Reset value is also true. In set2 node which values should I use as expression (Output of Gsheets or SplitInBatches).


Hey @Mustafa_Ali,

If you checked my workflow, I don’t have the Reset option set. I have deleted that option (by default it is false). We don’t need the Reset option. As explained earlier, this option will reset the batches, and in every iteration, you will receive the first item. We don’t to receive the the first item in every iteration. Instead, we want the next item in a new iteration.

Great. Its working now. Thanks for help.
Just curious how does how does split in batches iterate to next item when it takes input from last node?

That’s wonderful!

To answer your question, the Split In Batches node doesn’t take input from the last node. It uses the output from the previous node. The last node only tells the Split In Batches node that it should execute and split the next batch.

1 Like