I want to transfer a large amount of data from postgresql to elasticsearch. I am aware that I should be using the queue mode. In what way will my records be fetched? Do they get fetched row-wise, or are they batch processed? I am currently testing out n8n

Questions
2

Hi @Pruthvi_HP, welcome to the community :tada:

I don’t think you’d need queue mode here based on the description. Does your PostgreSQL data have a numeric ID or another column you can use to order and paginate through?

If so, you could simply use an approach as suggested over here to only process your data in chunks.

1 Like