Hi @vinnysud, would the URLs posted on Slack be unique?
If so, I’d just keep the processed URLs somewhere (anything capable of storing data, whether that’s Baserow, Airtable, Google Sheet, PostgreSQL…).
Then when your workflow fetching URLs from Slack runs, just fetch the processed URLs as well and use the Merge Node’s Remove Key Matches operation to remove all the already processed URLs from your Slack data source.
Going forward you could consider setting up a Slack app and have it send a webhook to n8n for every new message posted (and you can then simply add them to Clickup if they contain a URL), so you wouldn’t need to worry about past data.
Hey @MutedJam, thanks for the help.
That was indeed my first intent (I managed to save either on a local spreadsheet, or Google Sheet).
But I didn’t do a great usage of the Merge node so far: I’ll give a second try.
For the webhook, it looks like a great solution, but the workflow is stored on my local machine (which is not 24h/24 connected), so I guess I would miss some elements? Or is it capable of sending the elements that I missed during deconnection? Anyhow, I try your first technique and improve on top of this.
I’ll keep you updated.
Ok so that was a bit my issue yesterday with the Merge node: for some reason, it does not seem to work the way I’m expecting it: I tried to plug slack entries and and Google sheet and use removekeymatches, with no success. So I made a test and plugued the same entries in both inputs:
But still it keeps all the data and does not delete a single entry
So the way it works is that you pass on two different sets of items to the node (e.g. one coming from Slack, one from a Google Sheet having all the URLs already processed). It will then let only these items pass that are part of input 1 (Slack) but not of input 2.
Here’s a dummy workflow showing this. You can copy it straight into your n8n canvas if you want to give it a go:
You can see the top branch (“Mock All URLs”) returns 5 items whereas the bottom branch (“Mock Known URLs”) returns 3 items. When the workflow reaches “Process New URLs”, it will only have the two URLs that are in “Mock All URLs” but not in “Mock Known URLs”. I hope this makes sense