We have a sheet with over 600,000+ rows and 40+ columns data and wanted to understand if n8n’s internal database will be able to efficiently process that data during workflow operations or better to use some external database like Supabase / MySQL. What is your experience with such large data processing via n8n. And how to efficiently setup workflows to handle such large data processing.
1 Like
Hi @weblineindia to be fairly honest n8n can really do that, the problem can be related to how you fetch that sheet if you are using Google cloud console API , processing that kind of file will really fry the API and would instantly get you a silenced notice for maybe 1-2 hours, what i recommend is a proper database (Supabase) to handle all this, i have tried processing managing data with google sheets using n8n it is fairly easy and it goes nice but soon my sheet got over 120k rows and a huge amount of data my API started getting 403 and 404 and basically sometimes temp ban from google for spamming, and what i have done after that is i have exported everything in an csv and then imported that csv into the supabase and everything is working very smoothly ever since. So please consider getting yourself a Supabase subscription, ALSO if on n8n cloud consider a higher plan than PRO and on VPS go with the best config possible, data even over 10MB can really cause workflows to stop or produce errors so get the best machine and everything would sail smoothly.
1 Like
Thanks for the insights. How about n8n’s internal database (data tables). Will it be able to handle such data?
1 Like
@weblineindia it can but we are talking about safe processing of data and ofc you do not want to loose your data if something breaks in future updates, using Supabase even in free tier is the best option!
1 Like
Hi @weblineindia For 600,000+ rows, n8n’s internal database will crash. Use Supabase or MySQL for the heavy lifting and only bring small, filtered batches into n8n using Limit and Offset (e.g., 1,000 rows at a time). To prevent memory issues, disable ‘Save Successful Executions’ and perform all sorting/filtering via raw SQL queries instead of n8n nodes.
Hi @weblineindia
I think the bigger issue here is less about the database itself and more about how much data n8n is asked to process at once. In my experience, n8n can handle large datasets, but not if you try to load and process everything in memory at once. That’s usually where things start to break down, not necessarily the database itself. Even if you use the internal database or an external one, the important part is to process the data in chunks. I would avoid workflows that pull all 600k rows into a single execution. Instead, I’d paginate or batch the data and process it incrementally, for example using Loop Over Items or splitting the workload across multiple executions.
Another thing I’ve seen help a lot is pushing as much filtering and aggregation as possible upstream. So instead of pulling raw data into n8n and transforming it there, let the database return only what you actually need.
1 Like
Good points above. I’ll add some specifics from running PostgreSQL with n8n in production.
n8n’s internal Data Tables - avoid for your use case. It’s SQLite under the hood, not designed for 600k+ rows with 40 columns. You’ll hit memory issues fast, especially on self-hosted.
My recommendation: PostgreSQL over Supabase (or Supabase if you want the managed layer on top). Here’s the pattern I use:
- Never pull the full dataset into n8n. Always filter at the DB level first:
SELECT id, col1, col2 FROM your_table
WHERE status = 'pending'
LIMIT 500 OFFSET 0
- Use the Loop Over Items node to batch process in chunks of 200-500 rows
- Add proper indexes on the columns you filter/sort by - this alone makes a huge difference at 600k rows
- Disable Save Successful Executions in n8n settings to reduce internal DB pressure
One thing to watch: n8n loads ALL items from a node into memory before passing to the next node. So even with external DB, if you SELECT 600k rows at once, your workflow will crash. Batching is non-negotiable at this scale.
I’ve processed millions of rows this way without issues. The key is treating n8n as the orchestrator, not the data store.
1 Like