New node type: check for new data

Feature Requests Nodes
1

Following up on this conversation, it would be great to have a generic node that can check for new records in data when polling sources.

Common uses are:

  • Check if a record ID is unique (useful for non-numeric or non-increasing ID’s)
  • Check if a record value is larger than the last one (for auto-increment IDs, timestamps etc)

The node could have two outputs:

  • New records (a list of items that are new since the last time this scenario was run)
  • No new data (not something I’d personally need, but it might be useful for some use cases?)
2

Agree, this would be so useful since staticData is problematic for the beginners :slight_smile:

3

Looks like the groundwork for this has already been laid by @fahhem:

I didn’t see this PR before, only came across this in this thread but I think it pretty much fulfills this request.

2 Likes
4

Hi !

As per the thread I ignorantly created here, this feature is pretty much a smarter version of something I was thinking about. Just voted it, hope someone can pick it up again!

I would just add one option to what has been described already (which I would recap as “filter items and only let new ones through”). That would be an option to “filter all results out on first run” so that only new items coming in after activation of the workflow would be processed. This way it would be easy to have workflows running on “incoming emails in Gmail”, for example (without having to resort to imap setup if you have set up Gmail with OAuth already). In other words, it would turn a cron node + polling node combo into a pseudo-trigger to start a workflow when real triggers don’t exist or aren’t possible.

Cheers,
Pierric.