Merge (using Code node) for nested loops separately

Questions
, ,
1

Describe the issue/error/question

Hello guys,

I wonder how you merge data using the ‘Code’ node when there’s a loop inside another loop and you want to merge data separately for each nested loop.

So I created this example workflow where I get data (Customer Datastore) and do 2 runs (represented by ‘some_data_im_getting’ node) on the first 2 items. So in total 2 runs outside loop and 4 runs inside loop (2 for each item).

In the example workflow, the ‘merge_loop_runs’ node should output 2 items in the 1st run of the outside loop (only Jay Gatsby), and 2 items in the 2nd run (only José Arcadio Buendía). Now it outputs 2 items in the 1st run (Jay Gatsby) and 4 in the 2nd run (Jay Gatsby and José Arcadio Buendía).

To make things more complicated, in my real workflow the 2 runs of the nested loop is a limit - so it could be the nested loop runs only once (there’s one additional condition in the ‘IF’ node that can end the loop).

In theory, I know the i parameter in ‘merge_loop_runs’ node should be equal to 0 for the 1st run of outside loop and to sum of init - 1 (where init variable is from ‘increment_init’ node of previous runs) for every consequent run but I don’t know how to put it in code (or if it’s even possible).

Thank you.

What is the error message (if any)?

No errors.

Please share the workflow

Share the output returned by the last node

In the example workflow above, the expected output of the last node for the second run of the ‘split_in_batches_customer_datastore’ loop is just 2 last items instead of 4.

Information on your n8n setup

  • n8n version: 0.215.2
  • Database you’re using (default: SQLite): SQLite
  • Running n8n with the execution process [own(default), main]: own
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker
2

Very interesting question, @antitalent — thanks for taking the time to describe it so well.

Here’s a workflow that should do what you’re looking for. Changes I’ve made:

  1. Only try to gather the data when the outer loop is finished (so you only need to do it once)
  2. Slightly different code for gathering the runs (see last node)
  3. Added a property outer_run_index to your ‘set limits’ node. In the final node, you could use this to split the items based on which outer loop iteration they came from (or your ‘init’ to split them on inner loop iterations)

Is this what you’re looking for?

3

Nice solution @sirdavidoff, thank you for that. It looks very promising.

However, what I had forgotten to mention is that after the merge is done I need to perform another loop for each outer loop item (but I think it’s irrelevant for this particular issue) and finally update each outer loop item (i.e. the Customer Database item in this example workflow).

Basically what I’m saying is I need to “group” the output items of your ‘Gather runs’ node to 2 items based on the outer_run_index property - that’s what you’re saying in your 3rd point right? Sounds easy, and I played around with ‘Item Lists’ node but never made it work. I guess code is the only way to do it?

4

There are two options that might work, but it’s hard to say without knowing what else you’re doing in the workflow:

  1. The ‘summarize’ operation of the item lists node, with each field you want to keep added and set to ‘append’
  2. The ‘remove duplicates’ operation of the item lists node. You could sort the items beforehand if it matters which of the duplicated items you keep
5

Yes, summarize! I never noticed you can scroll down for another option. Thank you @sirdavidoff.

image
image728×513 27.6 KB