Performance issue of the N8N workflow when need to operate more than 50 000 data

Hello community!
I have the issue with my N8N workflow which retrieves the more than 50 000 elements from the external API and save them to the CRM.

WHEN
I run the workflow and Node retrieves the 50 000 Contracts from External API
THEN
Workflow hanging after 10 000 elements and can’t finish successfully.
Here is the workflow:

N8N Version: 1.17.1
Running n8n via Docker
Operating system Linux

It looks like your topic is missing some important information. Could you provide the following if applicable.

  • n8n version:
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main):
  • Running n8n via (Docker, npm, n8n cloud, desktop app):
  • Operating system:

Hi @Radik_Balagutdinov

Try splitting the load by using a split in batch node with a subflow to process the data in smaller batches.
There is a couple of examples on the forum already, so you should be able to find one. :slight_smile:

2 Likes

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.