my workflow slows down and fails when I use crazy or big data
Describe the problem/error/question
What is the error message (if any)?
Please share your workflow
(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)
Share the output returned by the last node
Information on your n8n setup
- n8n version:
- Database (default: SQLite):
- n8n EXECUTIONS_PROCESS setting (default: own, main):
- Running n8n via (Docker, npm, n8n cloud, desktop app):
- Operating system:
This is expected
Reading below should help
Hi @Ogunmoded_Okiki, welcome!
When you process a large dataset in a single execution, n8n either runs out of memory and the workflow slows down or crashes. To avoid this, instead of processing everything at once, split it down in batches in loops or use sub workflows to process different chunks, and so memory would be cleared quickly, and this problem would most probably not occur. Your hardware is also a limitation.
@Ogunmoded_Okiki welcome to the n8n community
What usually causes slowdown is not “using Data Table” by itself, but saving many unnecessary execution data; processing large payloads within the workflow; loops with thousands of items; large files/binaries; many inserts/updates row by row; workflows that load everything into memory at once.
Use this rule of thumb:
up to thousands/tens of thousands of simple records → Data Tables can work
hundreds of thousands/millions, reports, heavy search → external database