I am using n8n to store information in two databases. One is a vector database and another is a normal one, that I fill with structured data. Both databases, however, I fed with the same data.
I keep them apart because I want an easy way to display the information in the “normal” database on a website, but also want to provide the possibility of chatting with the vector database via RAG chatbot.
But to make sure that I can cross-reference the entries between the two databases, I store the IDs of the normal database as metadata of the vector database. Now I also want to get the IDs of the embeddings back (maybe as an array) to store in the normal database. But I have no idea how to best go about it. When storing the “supabase vector store” node, I can write the embeddings into the database, but it doesn’t send back the IDs. So I have to get them some other way.
But I have to admit that querying databases is far from my expertise. Does anyone have ideas on how to get the IDs and could maybe guide me through it? It would be soooooo much appreciated
Not quite. I want the IDs stored in the metadata. These are the IDs of another table (data) and I want to use them, to find the relevant entries. Once found, I would like the chatbot to answer the user’s question based on the information in the vector database and append it, with information from “data” (e.g. a URL). That way the user can check the sources from which the vector database derives it’s information.
Ah in this case you might want to identify a common unique ID that can be populated in both data bases, similar to how a relational database uses foreign keys referencing another table.
I did. I have a bigint in my vector database und a UUID in my normal database (data). Both have references to each other. But the problem is, that I cannot access the UUID of my normal data rows using the agent AI + vectore store node. Thus I can’t add the source links (which are stored in “data”) to my agent output.
So for each chat with the agent you want part of the result to be stored in your “normal” database to the respective UUID?
That should be doable with giving the AI Agent a tool to access your database / or alternatively outsource this process into a subworkflow which then gets called by your agent.
Do you wanna share your workflow for a better representation of the scenario?
Tip for sharing your workflow in the forum
Pasting your n8n workflow
Ensure to copy your n8n workflow and paste it in the code block, that is in between the pairs of triple backticks, which also could be achieved by clicking </> (preformatted text) in the editor and pasting in your workflow.
```
<your workflow>
```
Make sure that you’ve removed any sensitive information from your workflow and include dummy data or pinned data as much as you can!
Let me explain :slight_smile. Important info: I have the same data stored twice - As a normal, human-readable database and as a vector database. And below is the workflow I’m imagining.
User asks the Agent a question
Agent searches the vector database for the right answer
Agent also identifies the corresponding entries in the normal database (using their UUID provided in the metadata of the vector database)
Agent extracts additional data (e.g. a URL) with this UUID from the normal database
Agent provides an Answer to the User consisting of an individual text (using data from the vector database) and the URL of the sources, where this information was derived from.
