How to handle large data sets?

Hi @sasanka_deshapriya

Of course there is ways to get things done with large data sets. But we should always try to limit the amount of data where possible.
There is probably a way to filter the data coming from googlesheet for the last 7 days. Not sure though, as I always use a database if I am working with data like this.

To answer the question. You should probably get the data and then call a subworkflow to process it a smaller batch of items at a time. something like this
image
Make sure to clear your data at the end so it doesnt send all data back to the main workflow. you can do this by just adding a set node that keeps only set data. (and set it to run only once to be complete)