I have an elastic search node which list all the documents available in all the indexId passed to it, Now I was to return an array, which consists of an object inside the object I want the IndexId and all the document id present in that indexId.
Hey @LENINDALLAS,
Do you have some example data to play with? Don’t forget to remove anything important from it.
This is the workflow constructed, Id split function does the work but the next elasticsearch node does not output any data even when the value is hard coded.
Not able to iterate the values
Hi @LENINDALLAS, I was having a look into this just now but wanted to make sure I look at the exact data structure you’re having on your end. Would you be able to share the JSON output (minus anything confidential or private, of course) from your id split function node?
Hi @MutedJam
[
{
"index": ".ent-search-db-lock-20200304",
"id": "canvas-workpad-template:workpad-template-aefa8b2b-24ec-4093-8a59-f2cbc5f7c947"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "canvas-workpad-template:workpad-template-061d7868-2b4e-4dc8-8bf7-3772b52926e5"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "canvas-workpad-template:workpad-template-6181471b-147d-4397-a0d3-1c0f1600fa12"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "canvas-workpad-template:workpad-template-029bdeb3-40a6-4c90-9320-a5566abaf427"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "canvas-workpad-template:workpad-template-890b80e5-a3eb-431d-b8ed-37587ffd32c3"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "ingest-package-policies:a4496a2f-f9d5-4d80-b985-0e76845228de"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "ingest-agent-policies:policy-elastic-agent-on-cloud"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "ingest-agent-policies:0fe01d90-2bdf-11ec-a193-67e6c2df97e5"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "osquery-manager-usage-metric:live_query"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "index-pattern:ff959d40-b880-11e8-a6d9-e546fe2bba5f"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "account_id:61666230d1fec7da7221e911|user_oid:61666230d1fec7da7221e912"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "account_id:61666230d1fec7da7221e911|user_oid:616e8ea4d1fec7e40929b148"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "000000000000000000000001"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "4ebf6c6b21f4-1076901e-07fe-438a-ad26-8b2ed5b84681"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "61666231d1fec7da7221e913"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "616e8ea4d1fec7e40929b149"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "61666230d1fec7da7221e910"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "uFC1KUAbOMxhlUefvjAh33YPt0b7jUl+XeCsdsANzSI="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "y8Taq/C+gmeLUfAtkymyJYeCe7naL/Lt+z9yp7ZUwXs="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "AEzOK1k4WDOzaZWTTwCrUncI++wDJGNEPNImQ4ZbKkw="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "NczwzoT//iZpg8Wju3jOCpu0gCCirgtxWJqgBLx4cMQ="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "uPDngBhYdUUBV83RDtsK59l7c2uBqzs5kGO5gGuiYv8="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "n+oTYZMo1H+qJH6ivxREasWAyLYSlhLnK/CcVPseDQg="
},
{
"index": ".ent-search-db-lock-20200304",
"id": "uBdjl3wB5U-7ae04F21M"
},
{
"index": ".ent-search-db-lock-20200304",
"id": "uRdjl3wB5U-7ae04F21M"
}
]
This is the response data from the id split function, now I want to perform the get method in the next node by passing index Id and document Id.
Thank you
Many thanks for your example! It looks like the indices you are retrieving are system ones (seeing their names start with a dot . as per What's the difference between dot indices and non-dot indices - #2 by dadoonet - Elasticsearch - Discuss the Elastic Stack). Digging a bit more on the Elastic forums suggests the best course of action would be to avoid such indices.
So if you only want to fetch documents from your own indices rather than system ones, these documents would typically have the index in the document itself. For example:
Meaning if you are only looking for such non-system documents, you could use a much simpler logic like the one below to get IDs and indices for these documents:
Would this already work for you?
Thank you so much @MutedJam Your explanation helped the workflow get way simpler and I understood about the concept a little more
@MutedJam The output got at the last set node does not include index id, it returns only the id.
Can you suggest some ways to get index Id
Hi @LENINDALLAS, if the index is also unavailable in your actual documents (rather than the system ones), my suggestion would be to use the Split node to create a separate batch for each index like so:
This will also work for the aforementioned system indices. You would need to flip through each execution to see its data:
Hey @MutedJam the output is having both index Id and id fields, but since the set node gets executed many number of times the response sent for the webhook call consists only the last set node run value (2nd screenshot attached), could you provide a way, that the response data consists of all the data together.
Sorry, I was looking into the Elasticsearch part of your question and forgot about the webhook 
This is still possible though. I’d suggest creating two separate workflows in this case to separate fetching indices and delivering the response from fetching each document. You could then use the logic from https://n8n.io/workflows/1160 to merge the data from each execution back together.
Workflow 1 (fetching indices and handling the webhook):
Workflow 2:
Note that workflow 1 refers to workflow 2 via an ID in the Execute Workflow node which likely differs on your end. So you’d need to adjust that. When put together this should give you all the indices and documents:
Hope this helps!