I hope someone here can lend some advice on a system I’m building. I’ve got a Retrieval-Augmented Generation (RAG) system up and running that is very reliable—it hits about 99% accuracy on the questions I test it with!
Now I need to tackle scaling up the content. I’m adding documents specific to different aircraft models, and each model has roughly 20,000 pages of data. My main question is: Should I set up a brand new RAG instance and a separate vector database for each aircraft model?
If I combine them all into a single database, the total size would balloon to between 100,000 and 150,000 pages. Since everything is Dockerized on our internal server, I have plenty of computing power. My only real worry is the maximum practical size or performance limit of that single large database.
A rapid dip in accuracy may be observed when scaling up– the common RAG issue. You may have to look into reranking. Separate vector dbs seems like a bit much, partitions should work if you have logical separations like this.
Hey @abdouli Based on what I’ve been learning, you should totally set up separate vector stores for each aircraft model! If you put all those 100,000+ pages into one giant database, you might run into problems with performance, like retrieval getting slower. Plus, if you keep the documents separated, it makes sure you don’t accidentally mix up data from one plane with another, which would mess up your awesome 99% accuracy. Since you already have everything Dockerized, running multiple smaller databases will actually be a lot cleaner for the overall system architecture and easier to manage as you keep adding more stuff.
It’s the best way to keep your RAG super fast and accurate.