Guys I have an elastic search deployment, now I need to get all documents from a specific index, so I used the get all method and the response I get consists of only 10 documents but currently I have more.
can you clarify is it a problem with n8n elastic search node @MutedJam @RicardoE105
This is the workflow
Hm, this seems unexpected me tbh @LENINDALLAS. From looking at the code it seems the ReturnAll option simply means no limit parameter is set and Elastic’s default is used (which appears to be 10).
Unfortunately, I don’t understand Elasticsearch well enough to fix this myself (it should require specifying a sorting parameter and then use the search_after parameter for pagination, but there might be other implications). So I’ll add this to our internal bug tracker for a closer lok.
How can I get all documents available in a particular index from elastic search node
I think the best you can do for the moment is use the maximum size limit of 10,000.
To get all documents will require updating the node, using the scan & scroll API for versions <7.8 and search-after for versions >= 7.8.
Depending on how large the documents are though, it might be infeasible to retrieve 10,000+ documents to avoid memory issues.
Yeah got it, implemented the same with pagination.
Elastic search node does not let the user know that when return all document is selected it will return only few data