Nasty "Remove Duplicates" node bug?

Hey guys,

I’m experiencing some very weird behavior from the “Remove Duplicates” node and I created a whole new workflow just to test what was going on. It seems the node is discarding brand new items as duplicates even though they have not been seen before. I’m scratching my head over here thinking what am I doing wrong?

Here’s the test workflow (I can’t get it to show up here for some reason):

What I’m basically doing here is using the “remove items processed from previous executions” setting and clearing deduplication history on first new execution. Then, I’m giving it an input which is outputs correctly and then I’m using the set node to give it the same input but with 2 new/different items.

The node still discards everything even though there are two new items.

My background use case that lead to the discovery of this issue:

My use case here is that I’m using the node within a loop with an API that may return some duplicate items along with new items every now and then, and I want to break the loop as soon as the node no longer receives new items from the API and they’re all duplicates. Obviously because of this issue the loop keeps breaking immaturely because the remove duplicates node discards everything on second input even though there are new items.

What am I doing wrong here?

Thanks!

P.S: Running latest self-hosted n8n via Cloudron.

Any thoughts, guys? Tried to seek some AI help, Google help, trial and error but still same issue. Is it a bug or am I missing something? Really appreciate the help.

This forum is mostly community-powered, please don’t do that.

1 Like

Apologies, removed the mentions. Just thought it was missed. Is it possible to get paid help from the n8n team for self-hosted when the community isn’t able to solve a specific issue?

Thanks!

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.