Structure-aware RAG ingestion: how I get high-accuracy retrieval on technical docs

I built this for the service delivery team at an MSP I worked at. The engineers support a stack of different platforms, and every time they hit something unfamiliar in the middle of a ticket they were searching the web or digging through vendor portals for the right steps. So I ingested the technical documentation for every platform they support into a vector database and gave them a way to ask it questions in plain language through an LLM. It answered straight from the docs through a RAG MCP, so the exact instructions were right in front of them instead of a web search.

The thing that made the answers actually good was not the model. It was how the documents get prepared before they ever reach the vector store. Two decisions did most of the work: structure-aware chunking, and tagging metadata on the whole document before it gets split. I cleaned the workflow up, made it generic, and open-sourced it.

What it does

Drop a document in a Google Drive folder (I used Google for the template because it's widely used) and it gets normalized, tagged, chunked, embedded, and upserted into Qdrant automatically. No manual steps.

  • Watches a Drive folder and fires on every new file.
  • Converts any format to Markdown: PDF, Word, PowerPoint, Excel, CSV, HTML, plain text, images, and native Google Docs/Sheets/Slides. A small bundled MarkItDown service does the conversion over HTTP.
  • Pulls document-level metadata with a cheap LLM (title, type, summary, topics, keywords, named entities).
  • Chunks on the document's own heading hierarchy, not fixed-size windows.
  • Deletes any existing vectors for that file first, then inserts, so a re-upload never leaves stale chunks behind.

How it is built

Every node on the canvas has a sticky note explaining it, so the workflow documents itself.

  • Google Drive Trigger. Fires on a new file in the watched folder.
  • Config (Set). Holds the Qdrant collection name in one place; the store nodes read it from here.
  • Download File. Pulls the bytes. Native Google files are exported first (Docs to Word, Sheets to CSV, Slides to PowerPoint) so they convert cleanly.
  • MarkItDown Convert (HTTP Request). POSTs the file to the converter and gets back clean Markdown.
  • Information Extractor + Chat Model. The metadata schema. This is the one part you customize per document type.
  • Assemble Metadata (Code). Merges the extracted fields with the Drive fields and a content fingerprint into one metadata object per document.
  • Delete Existing Vectors (HTTP Request). Clears old points for this file_id.
  • Chunk by Structure (Code). The heart of it (see below).
  • Qdrant Insert with a Default Data Loader, an OpenAI embeddings node, and a large pass-through splitter.

The two decisions that made it work

Structure-aware chunking, in three tiers. Instead of cutting every N characters, the Code node splits on structure and adapts to how much structure the source actually has. If the file keeps real Markdown headings (Word, Google Docs, HTML), those become the section boundaries. If it is a flat PDF with no heading markup, it recovers the boundaries from short title-like lines. Only if neither works does it fall back to fixed-size windows, and even then it cuts on line breaks, never mid-sentence. Each chunk carries its heading path (for example Guide > Setup > Auth) and that path is prepended to the embedded text, so the vector itself knows what section it came from. No more chunks that stop halfway through a sentence, no more tables sheared off from their header row.

Metadata tagged before the split. Because the whole document is tagged before it is chunked, every chunk inherits those tags. That is what lets the retrieval layer filter to the right documents before the semantic search even runs. Instead of searching the entire store and hoping, you narrow to something like "policy docs about refunds" first, then search inside that slice. It is the single biggest lever on answer quality once the chunking is clean.

Where it fits, and where it does not

This is for structured, technical content. SOPs, policies, API docs, specs, platform runbooks, anything with clear headings and subjects. It is not the right tool for creative writing or long prose where there is no real structure to tag or split on. For a stack of technical documentation, it is my default.

Reusable

The pipeline is fixed; the only thing you customize per document type is the metadata (three nodes that stay in sync). There is a setup prompt in the repo you can paste into an LLM to generate those three edits for your own documents, so you are not hand-editing schemas. The converter is swappable too: anything that returns Markdown over HTTP works.

The whole thing is open source (MIT), with setup docs and the Dockerized converter: Document Fabric on GitHub

Happy to break down any single node if it is useful. The Chunk by Structure Code node is the interesting one and I am glad to walk through the tiering logic.

Cheers,

Michael

1 Like

One thing I left out of the post, since it lives on the query side and not in this ingestion workflow: after all this, I put a reranker on top of the retrieval, and that was the last big jump.

The flow at query time is: filter on metadata first, run the vector search over that slice, pull back the top handful of candidates, then hand those to a cross-encoder reranker that scores each one against the actual question and reorders them. The embedding search is good at “roughly about this,” but it will sometimes rank a loosely related chunk above the exact answer. The reranker is what fixes that last mile. It reads the question and the chunk together and pushes the real answer to the top.

Clean structure-aware chunks first, metadata filtering second, reranker third. Each one only pays off if the one before it is in place. Stack all three and the results have been hard to argue with.

I ran the same pipeline over developer documentation and built a RAG database my coding agents query while we build automations. Right now it is about 112,000 chunks covering around 40 platforms and frameworks: LangChain and LangGraph, LlamaIndex, CrewAI, Pydantic AI, FastAPI, Temporal, Prefect, n8n, Qdrant and ChromaDB, plus the infra layer like Cloudflare, Kubernetes, Docker, Traefik, and Terraform. When I am building on any of those, the agents pull the current docs instead of guessing from stale training data.

One change I made here: I moved the embeddings off the paid OpenAI API and onto a self-hosted Qwen3 model running locally, at 4096 dimensions. Two reasons. Embedding 112,000 chunks, and re-embedding whenever the docs update, adds up fast on a per-token API, and I would rather not ship all of that through a third party. Running it locally, the only cost is my own hardware and the data never leaves.

Same three layers, same order: structure, metadata, reranker. The metadata filter really earns its keep here. Narrowing to the right platform and version before the search is what stops the agent from blending three major versions of a framework into one wrong answer.