Hi everyone,
I’m processing 10k+ records from an API in n8n using HTTP + Merge nodes. Even with pagination and rate limits, I’m still hitting execution time issues.
Any tips for speeding this up or proven patterns for handling big datasets?
In general, if you are using a workflow/orchestration tool to do the “heavy lifting” you should probably offload that to a separate service that can be tuned more appropriately to have the capacity it needs to complete the work, or designed to work asynchronously so that the main process (workflow) can pause/wait until the work is done.
Post more details about what exactly you are doing and someone might have a more specific suggestion.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.