How to pass a single downloaded file to pinecone vector store at once rather than sending all in a bulk

The following is my Workflow.

  • I have a Google Drive Node in which the folder test1 is chosen. I am extracting all the files from it, ignoring the folders.
  • The output is sent to another Google Drive Node where it is downloading the files.
  • I want to pass these to Pinecone vector Store where I am inserting the documents to a namespace. I clear the namespace everytime before the namespace is created.
  • The namespace is created as the name of the file i.e. the Google file name.

Issue:-
I am getting 8 files from the Google Drive Download Node. All the 8 files are passed to the Pinecode Vector Store at once. Ideally I want each of the file to be passed to it.

The output of the Google drive Download Node seems like:-

[
  {
    "id": "1VV3DFJbmENOfFYSVNuIGZZQX_bsaTDgI",
    "name": "Diffblue Supporting Slides (1).pdf"
  },
  {
    "id": "1XNhFBe_3Y5OD4v_lBzOitc3TjtJHhapb",
    "name": "Accion Integration COE Doc.pdf"
  },
  {
    "id": "17-ZYLf8tjAdMLDyGBYGcrE7Px9VsCyox",
    "name": "2.txt"
  },
  {
    "id": "1Z7DOvdNX2mMtP3EE1WSeXSkDZVAq9bGR",
    "name": "1.'txt'"
  },
  {
    "id": "1frN_6x883svdsBH2ZWNzP2hbX8va8COH",
    "name": "Copy of Spring and SpringBoot Notes.txt"
  },
  {
    "id": "1EwLwGO1WOsvy6Dm9j0luC2n2kdasXmOL",
    "name": "Copy of Commands Intellij.txt"
  },
  {
    "id": "1mIdDxsPIvTanyHU7vScJWzfochs3Ylgo",
    "name": "Spring and SpringBoot Notes.txt"
  },
  {
    "id": "1SSZnovuT-ltvEmCaWCATufXlcM_aN68p",
    "name": "Commands Intellij.txt"
  }
]

How can I ensure that one item at a time is passed to the Pinecone node rather than all at once? This will let me create multiple Namespaces rather than one.

Looping through the items helped me in resolving the issue.
Here is the solution:-

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.