Can't get some nodes to iterate over all input items

[ajayjohn]

I have read in other forum threads that n8n nodes are built to iterate over all the items returned by the previous node. However, I am unable to get that to work consistently and am quite confused about what I could be doing wrong.

Given below is the flow in question. It fetches all Jira entries matching a certain filter, processes it via a function node that transforms some values in each entry fetched and finally, passes it on to a series of nodes that check if a matching entry exists in ClickUp and take some resultant actions on it.

The challenge I am facing is that the ‘Process issue details’ node does pass on an array of items correctly to the ‘Fetch matching tasks’ node. (Seen below)

However, this node only processes the first record in the array. (Seen below)

Surprisingly, if I replace the ‘Fetch matching tasks’ ClickUp node with an HTTP Request node, it perfectly processes every single item array it receives. Why does this not work with the ClickUp node?
Could someone help me with this issue, please?

P.S: I know that I have not shared too many details of the node’s configuration, but I can do so as requested.

[jan]

I just checked the code of the node and it looks all correct for me. Can you please share an example workflow with the “Process issue details” node that should output some mock data which is similar to the one you receive as output there and the “Fetch matching tasks” node. I can then have a look and test what is going wrong. Thanks!

[ajayjohn]

Thank you, Jan!
I just messaged you the node details (since it contained a few IDs for my instance that I did not want to post publicly). Just let me know if you require anything else.

[jan]

Thanks a lot received it. But looks all good to me and works fine. I had to create a very simple version of what you did which creates two items and then filters for department and gets one item each (one for department 0 and one for department 1).

It totally correctly does make two requests and I end up with two items (one from each department).

Here my simple example workflow:

[ajayjohn]

Thank you for looking into this, @jan! When I ran your code, it certainly did work perfectly for me too.
That prompted me to investigate if the issue is caused by at least one input item yielding no output in the ClickUp node.

Turns out that is definitely the case. If I remove all constraints and ensure that all input items yield an output, I see that the ClickUp node gets executed for every input record. However, if some input items will yield no output records, the ClickUp node does not yield any output (even for the few items that should have yielded an output). Is this normal behavior?

[jan]

No If it is as you describe then it is definitely a bug. I will have a look.

[jan]

Can sadly not reproduce what you describe.
Added now to my above example 3 more items with department IDs which do not exist. One in the beginning, on in between the two existing and one at the end. It still totally correctly returns two items like before (the two that actually exist).

[ajayjohn]

Ah, it was my mistake. I incorrectly understood how the n8n handles an array of items. I assumed that when an array of items is received, the node will fire once for each item (as opposed to processing all items and sending a single consolidated array as its output). I was able to achieve what I needed by using a ‘Split in batches’ node and and an if loop, instead.

Thans again, @jan and I apologize for sending you on a wild goose chase to find an issue that did not exist

[jan]

Ah really great to hear that you found the issue.

No problem! Have fun!

[fo2rist]

It still does looks like an issue to me (the looping behavior of a node is unexpected).
I have a similar configuration with one Google Drive node fetching folders and another fetching subfolder for them. When the first Drive node is connected to the HTTP node, the HTTP node loops through the input array, and fires one request per item. But when the second drive node gets in between it executes once and stops. That’s not consistent with HTTP node behavior and not expected (e.g. Integromat iterates through the list).
Let me know if I’m doing something wrong or missing something.

{
  "name": "Move files from One Folder to Another",
  "nodes": [
    {
      "parameters": {},
      "name": "Start",
      "type": "n8n-nodes-base.start",
      "typeVersion": 1,
      "position": [
        250,
        300
      ]
    },
    {
      "parameters": {
        "authentication": "oAuth2",
        "operation": "list",
        "useQueryString": true,
        "queryString": "=name contains \"t-kittens-\" AND mimeType = 'application/vnd.google-apps.folder'",
        "options": {
          "fields": [
            "*"
          ]
        }
      },
      "name": "Google Drive",
      "type": "n8n-nodes-base.googleDrive",
      "typeVersion": 1,
      "position": [
        450,
        300
      ],
      "credentials": {
        "googleDriveOAuth2Api": "Google Drive Read/Write access"
      }
    },
    {
      "parameters": {
        "authentication": "oAuth2",
        "operation": "list",
        "useQueryString": true,
        "queryString": "='{{$json.id}}' in parents AND mimeType = 'application/vnd.google-apps.folder'",
        "options": {}
      },
      "name": "Google Drive1",
      "type": "n8n-nodes-base.googleDrive",
      "typeVersion": 1,
      "position": [
        660,
        300
      ],
      "alwaysOutputData": false,
      "credentials": {
        "googleDriveOAuth2Api": "Google Drive Read/Write access"
      }
    },
    {
      "parameters": {
        "url": "https://webhook.site/UUID",
        "options": {},
        "queryParametersUi": {
          "parameter": [
            {
              "name": "item",
              "value": "={{$json.name}}"
            }
          ]
        }
      },
      "name": "HTTP Request",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        870,
        300
      ]
    }
  ],
  "connections": {
    "Start": {
      "main": [
        [
          {
            "node": "Google Drive",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Google Drive": {
      "main": [
        [
          {
            "node": "Google Drive1",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Google Drive1": {
      "main": [
        [
          {
            "node": "HTTP Request",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "active": false,
  "settings": {
    "timezone": "America/Los_Angeles",
    "saveManualExecutions": true,
    "executionTimeout": -1
  },
  "id": "1"
}

[jan]

Welcome to the community @fo2rist!

Yes, you are absolutely right. Many of the “list” or “getAll” methods are currently inconsistent. Meaning they often execute only once. Is for example the same on the Google Sheets node.

The reason for that is that 90% of the people expect that to happen. They do one “list” operation at the beginning of a workflow or if they do it in the middle they want to replace the whole data. But because that is inconsistent and n8n has now the option “executeOnce” we want to change it, to make all nodes consistent in the future.
So it will be changed in the future but I do not have an ETA yet. The reason is that changing it would be a very big breaking change for many people. If we take for example a user which has a for him correctly executing workflow and he “sends” 1000 items in Google Drive with list set and gets 1000 back, he would in future receive 1 million (1000 x 1000). So we have to be very careful there.

What you can do in the meantime is to use the SplitInBatches Node.

[fo2rist]

Thanks a lot, Jan!