For each node, there should be an option to “Clear runs” in order to limit memory usage, as previous runs might only be relevant for debugging purposes.
Welcome to the community!
Could you give a bit more information on what this feature request is about. It sounds like a good idea but not sure if I understand correctly. And would of course be best for the n8n team to understand it as well.
Hi @BramKn ,
When nodes within a workflow have multiple runs, they store information about each run in memory. Over time, especially with data-intensive nodes or frequent executions, this can consume significant amounts of memory, leading to performance issues. While this storage is useful in some cases, it is not always necessary.
- Node-Level Toggle: Introduce a toggle at the node level to decide whether or not the data from previous runs should be stored. This provides granularity, allowing users to determine which nodes should retain data and which shouldn’t.
- Global Workflow Setting: Provide a global setting within the workflow to clear data from all nodes after their execution. This would act as an override for the individual node settings.
- Improved Memory Usage: This feature would allow workflows to run more efficiently, especially in scenarios where they become memory-intensive due to data storage from previous runs.
- Flexibility: Users can decide on a case-by-case basis when data retention is necessary and when it is not, allowing for optimal performance in various use cases.
- Encourages Modular Design: By providing an efficient means to manage memory usage, users can be discouraged from splitting workflows into “sub-workflows” just for the sake of memory optimization. This can lead to cleaner, more maintainable designs.
I like this idea as well.
Would be really useful (particularly in loops) to offload the memory from individual nodes when the use-case demands.
Although, it would probably have to retain the ‘last run’ as that would be necessary for the functioning of the workflow.
So, maybe also have an option to “retain” data for last “n” runs only.
It is a good idea indeed. But as @shrey-42 mentioned it would be needed to atleast keep the last run for the workflow to function.
There are some more challenges here as well as it will be changing the core of n8n I expect.
In the mean time you can use a split in batch and sub workflows or if you fancy community nodes I also have added the option to automatically use sub workflows to the https://www.npmjs.com/package/n8n-nodes-splitinbatches-advanced community node.
I completely agree with your perspective. Retaining at the latest run is crucial for the integrity of the workflow. The idea of having an option to define how many runs to retain is great!
Thank you for sharing that node. I really like that you can combine the results!