Graphical Iterator Node

When trying to use an array currently its only outputting the first result, there are some programming trickery functionNodes that get it working, but some people just struggle with that and it wastes a lot of time.

Other similar services like Zapier and Integromate have an Iterator node that takes the multiple arrays and keeps iterating over and over until done.

Having this would be really handy.

As long as the data is structured correctly, all nodes (with a few exceptions) will do the iteration for you behind the scenes. If the functionality you need is no available, you can always use the HTTP Request node with a function node to map the data to the right structured.

I have never used iterator on those platforms but I would bet iterators are the same as the SplitBatches Node in n8n.

very well could be split batches, let me have a look. cheers

maybe, but not sure it can handle nested arrays, or i dont know how to tell it to do it( is more likely)

This is what i was after.

return items[0].json.objects.map(item => {
  return {
json: item
  }
});

Cant we just add that to a mappable node button , so we can link that data of the node before :wink:

2 Likes

Agreed that it would be useful. I overheard @maxT talking about the possibility of Data Transformation nodes in n8n the other day :male_detective:‍♂

2 Likes

Thanks for your post! The way n8n handles items means a “loop” or iterate node is less likely the solution but instead a Map/ Transform node is required (i.e. to get nested objects into what n8n considers an “item” - i.e. a top-level object in the array being sent into a node, since n8n performs actions per-item by default).

Just wanted to say that totally hear you on this one - I am currently performing research on best way to remove 70-80% of the need to use Function node for general data mapping/ transformation. We’re proceeding with a bit more caution on this node since we don’t want to put out some ill-conceived hacky solutions, then be supporting half-baked mapping nodes (for example, we’re supporting both Interval + Cron trigger; but in hindsight they could probably be one node just with various options). Once a node is released, we try hard not to make breaking changes or deprecate them so that users’ active workflows are not impacted.

On that note, if you can think of great examples of data mapping no-code patterns in other tools, please do share! Or simply listing the types of abstract transforms you envision needing to do often is also very valuable.

Cheers!

2 Likes

Honestly look at how integromat does it, if they didn’t charge for every single action/iteration I would still be there. Some of these scenarios are doing API lookup on an API lookup which when you think has over 1000+ results in the first lookup which then does another 1000 API lookups, gets expensive to use their cloud system per action.

The way they map data between nodes is excellent.

4 Likes

Would be good to see a graphical iterator node added to the product roadmap.

Hi @RedPacketSec — thanks for your post.

Data transformation and data mapping in n8n could definitely be improved, and we’re working on a master plan for that.

Integromat have put a lot of thought into how they do it, but our approach is fundamentally different to theirs: we have iteration built into every node by default. So as you mentioned, all that needs doing is to split the things you want to iterate over out into items. Adding a node for this will be part of the data transformation overhaul.

2 Likes

nice to hear, iteration is a pain point for me sometimes especially nested arrays

Any news on the data mapping and data transformation master plan?

It’s in progress! First steps:

  • Node to perform common operations on lists of items (e.g. split a single item out into multiple)
  • Extending expressions to be much more useful
  • Improving the ‘Set’ node
3 Likes

nice!

1 Like

How we can use this iteration?
For exemple i have this expression that I use to select a string.
{{$node["NocoDB"].json["job"]["core"][0]["version"]}}
How I can set it up so that the action will be repeated for every object in the CORE array ?

You probably need to map the data within the core array. If you provide more details (in another post) we can help you better.