I’m trying to create a workflow in n8n that compares two lists (let’s call them Input 1 and Input 2) and outputs only the items from Input 1 that don’t have a match in Input 2. To achieve this, I’m using the Merge node in “Remove Matches” mode.
Here’s the issue:
Input 1 produces multiple executions, each containing one or more items.
Input 2 produces a single execution, which contains the entire reference list I want to compare against.
The problem with the Merge node is that it only compares the first execution from Input 1 with the execution from Input 2, completely ignoring the other executions from Input 1. As a result, I can’t properly compare all the items from Input 1 with Input 2.
What I need:
Combine all items from Input 1 (even if they arrive as separate executions) into a single array within one execution.
Compare this unified array with the list from Input 2 using the Merge node in “Remove Matches” mode.
Output only the items from Input 1 that don’t match anything in Input 2.
What I’ve tried:
Using the Aggregate node to combine all items from Input 1 into a single execution.
Using a Function node with $input.all() to manually consolidate all executions from Input 1.
Experimenting with different configurations of the Merge node.
Unfortunately, none of these methods have worked. The Merge node continues to process only the first execution of Input 1, and I’m unable to properly aggregate the items to make the comparison work.
Question:
Is there a native way in n8n to handle this? I need to ensure all items from Input 1 are merged into a single execution before being compared with Input 2. Any advice, suggestions, or best practices would be greatly appreciated!
Thanks for posting here and welcome to the community!
I think it depends on what kind of data triggers these multiple executions. Can you share the workflow that you have so far and ideally some sample data to understand a bit better, what the dynamics of your use case are?
Thanks
Tip for sharing your workflow in the forum
Pasting your n8n workflow
Ensure to copy your n8n workflow and paste it in the code block, that is in between the pairs of triple backticks, which also could be achieved by clicking </> (preformatted text) in the editor and pasting in your workflow.
```
<your workflow>
```
Make sure that you’ve removed any sensitive information from your workflow and include dummy data or pinned data as much as you can!
Ria, I have 2 inputs in a “Merge” node, that keeps only the input 1 items that are non-matching with input 2.
In input 1, I have a HTTP node that receives a simple json with data from Meta Ads Leads. This node works in batches because there are many origins (different meta forms) and pagination. So, I have like 5 executions with 50 items each (I use a loop to get next pages).
In input 2, I have a “Baserow - Get many rows” node, that execute only one time with 250 items.
The merge node matches only first execution of each one, so, it matches the first 50 with the 250.
Is it enough or you do need more details and a flow?
Thanks, that helps to get a more clear picture.
Is there any reason why you’re doing the pagination as an additional loop outside of the HTTP request node?
Does the pagination option of the node not work for you? Because if you were doing the pagination within the HTTP node itself it would gather all the results into one output.
Something like this (e.g. when response includes a NextURL:
Thank you so much for your response! I really appreciate the help.
I wasn’t using the pagination feature within the HTTP Request node simply because I wasn’t aware it existed! Now that I know about it, I’ve adjusted my approach accordingly.
However, I’m struggling to get the paginationCompleteWhen condition to work properly. Here’s what I’m trying to do:
={{
new Date($('Define dados').item.json.data_limite_evolution).getTime()
<
($response.body.messages.records.last().messageTimestamp * 1000)
}}
The first variable is the minimum timestamp to ensure that I don’t fetch data older than this value. Converting an ISO date to Unix time works correctly here.
The second part is where I’m stuck. I need to compare it against the last message’s timestamp, but no matter how I format it, the condition never works.
In the preview, the value appears correctly, but at runtime, I get the error: “Cause detailed Missing sourceData (probably an internal error).”
This is my full HTTP Request node configuration:
Even though I’m now using the built-in pagination, my original question is still relevant:
If I needed to compare values across different executions (instead of consolidating them into one execution), how could I do that in n8n?
