Describe the problem/error/question
My flow is designed for preparing an RAG file. The input is text that is converted into JSON. I want to store this JSON as an OBJECT in Qdrant. However, my flow starts splitting the JSON by attributes and creates many useless vectors. The entire JSON can be stored if I pass it as text, but then Qdrant treats it as text, whereas I want to store it specifically as a JSON object. From the forum, I see that others have encountered a similar issue, but no clear solution has been provided http://community.n8n.io/t/how-to-pass-json-data-to-a-qdrant-vector-store-node-properly-with-code-node-or-edit-field-node/114415.
What is the error message (if any)?
There are no errors, but the data is split into different Qdrant points.
Please share your workflow
The forum does not allow properly pasting the flow, so the script from Edit Fields5 is below.
{{
(() => {
let input = $json.data
// Парсим уровень
const lvlMatch = input.match(/# Уровень:\s*(\d+)/);
const lvl = lvlMatch ? parseInt(lvlMatch[1], 10) : null;
// Парсим тип (первая буква заглавная)
const typeMatch = input.match(/# Тип:\s*(.+)/i);
const type = typeMatch ? typeMatch[1].charAt(0).toUpperCase() + typeMatch[1].slice(1).toLowerCase() : null;
// Парсим классы и названия
const classNameRegex = /^- ([^:]+): (.+)$/gm;
let classes = [];
let match;
while ((match = classNameRegex.exec(input)) !== null) {
classes.push({ class: match[1].trim(), name: match[2].trim() });
}
// Парсим статы
const statRegex = /^- (.+) - ([\d.]+)%$/gm;
let stats = [];
while ((match = statRegex.exec(input)) !== null) {
stats.push({ name: match[1].trim(), chance: parseFloat(match[2]) });
}
// Константный statCount
const statCount = { "1": 0, "2": 0, "3": 100, "4": 0 };
// Формируем итоговый массив
const result = [];
for (const cls of classes) {
for (const stat of stats) {
result.push({
lvl,
name: cls.name,
class: cls.class,
type,
stat,
statCount
});
}
}
return result;
})()
}}
Share the output returned by the last node
I want to achieve the result below (screen1). But I get screen2
screen1:
screen2:
Information on your n8n setup
- n8n version: 1.105.4
- Database (default: SQLite): qdrant
- n8n EXECUTIONS_PROCESS setting (default: own, main):
- Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
- Operating system: Ubuntu 24.04
I cannot edit my message because the site gives an error, so I am attaching test data for script verification below.
# Тип: броня
# Уровень: 1
# Названия и классы:
- Маг: Накидка языка природы
- Жрец: Мантия божественного света
- Друид: Накидка прорицателя
- Шаман: Накидка древнего колдуна
- Мистик: Накидка единства
# Статы:
- Сила +6~12 - 7.72%
Qdrant is a vector(!) database. Each record ( aka point ) is a vector + an optional JSON payload. The payload is there to store metadata (any JSON), which you can filter on, retrieve, and return with search results. Qdrant is not a general JSON database, and you can’t store a point without a vector.
If you need to store json as metadata for filtering - you can totally do that, as for search process…
You embedding model (nomic-embed-text) expects textual data (string) as input to convert it to a vector, which then is passed to the qdrant database.
Qdrant stores entire json as text and it is expected, it is what is returned to the LLM when appropriate vectors are found. You do want it to be text. This is not what qdrant uses to find matches, it is only to return it back.
Hi.
1. I still haven’t received an answer to my question. It doesn’t matter for what purpose I want to store a json for vector description. I need it and I’m asking how to do it.
2. Qdrant vector store builds a vector from text and puts the same text in metadata. It doesn’t make any sense.
For example, I store a book description in a vector, but when I find a book by its description, I want to know who its author is. To do this, I build a vector from the description and put the author in metadata. However, the standard qdrant vector store component does not seem to allow this.
You can store the whole thing as a vector, why do you need to store author in the metadata?.. Anyway, “you need it” and “you are asking how to do it”… so here we go…
Here we store the book’s description as a vector and author (and title) as metadata.
After that we see two new points:
Then we retrieve the info with AI Agent:
The chat:
What doesn’t standard qdrant vector store component seem to not allow? What is the bug?
Thank you for the solution! It can really solve the given problem!
I managed to find a different solution to my problem. It might be useful to someone else as well.
Create RAG
Extract data
Basically, it is a custom-made equivalent of the Qdrant Vector Store.
The bug is that Qdrant Vector Store cannot extract content from a point if it is not a string. This may be an undocumented feature, but it is not obvious at all. Not to mention that storing the original data on which the vector is based is a very specific task, applicable mostly to AI. For semantic search, it’s just foolish.
In any case, I’m glad that I was able to find both an out-of-the-box solution and a custom-made one. Thank you.
