Why HTTP node taking so much time to GET the contacts from any connector like Mailchimp or Kalviyo?

Describe the problem/error/question

It has been observed that the HTTP node takes too much time to load (GET) contacts from any connector when the number of contacts exceeds 70K. The node keeps executing without any error message. Is there any way to improve the node’s performance with additional settings?

What is the error message (if any)?

Keep loading

Please share your workflow

Its a general behaviour with every workflow.

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Loading

Information on your n8n setup

  • n8n version: 0.228.2
  • Database (default: SQLite):
  • n8n EXECUTIONS_PROCESS setting (default: own, main): default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • Operating system: MAC

Hi @Gouravdev, are you storing the execution data? For larger amounts of data, this can significantly slow down executions (especially if the database is already under load). As a first step, you may want to try disabling caching of execution data.

Assuming you are processing your 70K contacts in batches you might also want to use the Split in batches node instead of using the built-in batching of the HTTP Request node. This will give you a bit more transparency, as it allows you to view the progress (it wouldn’t speed up things though):

ezgif.com-gif-maker

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.