Guys I have an elastic search deployment, now I need to get all documents from a specific index, so I used the get all method and the response I get consists of only 10 documents but currently I have more.
can you clarify is it a problem with n8n elastic search node @MutedJam @RicardoE105
This is the workflow
Hm, this seems unexpected me tbh @LENINDALLAS. From looking at the code it seems the ReturnAll option simply means no limit parameter is set and Elastic’s default is used (which appears to be 10).
Unfortunately, I don’t understand Elasticsearch well enough to fix this myself (it should require specifying a sorting parameter and then use the search_after parameter for pagination, but there might be other implications). So I’ll add this to our internal bug tracker for a closer lok.
How can I get all documents available in a particular index from elastic search node
I think the best you can do for the moment is use the maximum size limit of 10,000.
To get all documents will require updating the node, using the scan & scroll API for versions <7.8 and search-after for versions >= 7.8.
Depending on how large the documents are though, it might be infeasible to retrieve 10,000+ documents to avoid memory issues.