Merge node creating duplicate records

[ThomasLu_EarthDaily]

Describe the problem/error/question

I am trying to combine data from two nodes by matching fields in the Merge node, but the node is creating duplicates and outputting more items than I am putting in.

I believe part of the issue stems from me splitting out a field to perform that many HTTP requests which I am trying to enrich/join the previously split items, via the following steps:

1. Split out PhysicalCredentials
2. Perform an HTTP request for each PhysicalCredential’s CardTemplate
3. Combine records matching the ID from HTTP node with CardTemplate from Split node

Although I can remove duplicates after, I’m just curious as to why this is happening.

What is the error message (if any)?

No error message. Erroneous output.
Workflow outputs have been pinned to demonstrate issue

Please share your workflow

Share the output returned by the last node

Information on your n8n setup

• n8n version: 1.67.1
• Database (default: SQLite): default
• Running n8n via (Docker, npm, n8n cloud, desktop app): n8n cloud

[ria]

Hi @ThomasLu_EarthDaily

The issue is that you already have duplicate entries in both your Inputs (namely ef8fc32a-6f04-427a-b962-0e61e538d17c). I was able to recreate the same scenario (see below).

I am actually not sure if this is a bug or by design, as the merge node is not meant to remove duplicates from input. But I can see how the expectation in a scenario like yours would be to completely merge all data into unique output items.

@Jon what’s your take on this?

[ThomasLu_EarthDaily]

Oh, wow… I can’t believe I missed seeing that.
I was just so stuck on “3 records, 3 HTTP requests… expecting 3 outputs”

I’ll just drop a Code node before the HTTP request to get the distinct CardTemplates.