How to continue workflow when Remove duplicates doesn't output anything?

[thoresson]

I’m building a workflow that ingests a couple RSS feeds, iterates over the items and stores new ones to a database. At first, I iterated over allt 17 feeds and fetched the items. But this results in 352 articles, that seems to be a bit much for Remove Duplicates to handle in one go.

So instead I’m trying this: Iterate over the feeds one after another, removing duplicates for them one by one. But with this setup, the workflow stops as soon as there is a feed with no new items.

I tired to activate “Always output data” to the Remove Duplicate node, but that introduced another problem: A JSON array with a lot of empty entries, like this, is the output in the Done branch from the Loop Over Feeds node.

[
  {},
  {},
  {},
  {},
  {},
  {},
  {},
  {},
  {},
  {}
]

And I don’t find a good solution to remove all empty values from the JSON, either in the step between remove dupes and the loop, or between the loop and the following step.

How can I get rid of them?

(Or, would you do this some other way all together? Would it be better to get rid of the dupe check and let my Supabase node handle the duplicates, only creating new rows for new items?)

Thanks!

[Franz]

A Filter node (always output data) combined with an If node could solve the issue.
The Filter node checks if $json Object is not empty, but it will output an empty object if nothing remains. The If node then checks whether the object is empty.

[thoresson]

Excellent, thanks!