I want to transfer a large amount of data from postgresql to elasticsearch. I am aware that I should be using the queue mode. In what way will my records be fetched? Do they get fetched row-wise, or are they batch processed? I am currently testing out n8n

Describe the issue/error/question

What is the error message (if any)?

Please share the workflow

(Select the nodes and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow respectively)

Share the output returned by the last node

Information on your n8n setup

  • n8n version:
  • Database you’re using (default: SQLite): SQLite
  • Running n8n with the execution process [own(default), main]: default
  • Running n8n via [Docker, npm, n8n.cloud, desktop app]: Docker

Hi @Pruthvi_HP, welcome to the community :tada:

I don’t think you’d need queue mode here based on the description. Does your PostgreSQL data have a numeric ID or another column you can use to order and paginate through?

If so, you could simply use an approach as suggested over here to only process your data in chunks.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.