Needing assistance - Local files into vector DB

Questions
1

I am needing some assistance with getting my local files into my Postgres Vector DB. My goal is to monitor my local server for specific file changes (added, updated, deleted) and update the vectors. If a file is updated, I want the older version to be removed before the updated file is added. I can find flows for monitoring Google Drive and inserting into Supabase. But nothing for local files to Postgres.

I attempted to get ChatGPT to assist but that was a joke. The training dataset is too old and it wasn’t aware of the correct nodes. And it had me pretty much using Code Nodes to attempt to accomplish the task. In the end we were making repeated changes that were taking me in circles.

I’ve even tried simplifying the flow with a manual trigger to simple take the files from the specified directory and insert them into the DB. The path error still remains. I have verified that permissions are correct and accessible from within the containers. Volumes are mounted properly. I can manually list the contents of the path from within the n8n container.

Intended flow: Local File Trigger → List Path → Create ID → Check File Status → Delete if needed → Extract file text → Insert into vector

Command failed: cat {{$json[“file_path”]}} cat: can’t open ‘{{[file_path]}}’: No such file or directory

This flow didn’t start out like this. These are changes suggested by ChatGPT. Pretty much went from no code to “Hey, Let’s code EVERYTHING” :smile:

Information on your n8n setup

  • n8n version: 1.59.4
  • Database (default: SQLite): psql (PostgreSQL) 16.4 (Debian 16.4-1.pgdg120+1)
  • n8n EXECUTIONS_PROCESS setting (default: own, main): Default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
  • **Operating system:
  • Locally hosted n8n and Postgres
    Distributor ID: Ubuntu
    Description: Ubuntu 24.04.1 LTS
    Release: 24.04
    Codename: noble
    **
2

Yeah I don’t think the Execute Command node can take n8n expressions so your “cat {{$json[“file_path”]}}” is just that and the {{ ... }} doesn’t get replaced. To fix (If you want to keep your template as-is), try replacing all Execute Command nodes with “Edit Fields (Set)” nodes which can handle expressions.

I created a similar flow for handling local file changes here - Build a Financial Documents Assistant using Qdrant and Mistral.ai | n8n workflow template - which may be of interest. It doesn’t use PostgreSQL however but it should be pretty straightforward to change.

1 Like
3

Thank you for your reply. I was actually looking at some of your templates that I came across last night. I was debating between the Ultimate RAG Setup and the Financial Documents Assistant. I think the Financial Documents Assistant will probably work for my use case. Nice work by the way. You’ve created some very nice workflows.

1 Like
4

This is incorrect. You just have to have Expression clicked instead of Fixed.and then you can use {{}} to access n8n data.

1 Like
5

Ah thanks for the correction!

6

I have changed the workflow that I am using to your finacial doc flow. However I am having an issue with the Local File Trigger. The trigger is only returning one file in the top directory path. I am wanting to monitor a folder and all sub-folders and files. But even in the top folder it’s only returning one file when there are 11 files and 10 sub-folders. Do I need to approach this from a different node?

‘’’
{
“meta”: {
“instanceId”: “558d88703fb65b2d0e44613bc35916258b0f0bf983c5d4730c00c424b77ca36a”
},
“nodes”: [
{
“parameters”: {
“triggerOn”: “folder”,
“path”: “/automation/docs”,
“events”: [
“add”,
“change”,
“unlink”,
“addDir”,
“unlinkDir”
],
“options”: {
“awaitWriteFinish”: true,
“depth”: -1,
“usePolling”: true
}
},
“id”: “36eefd90-b45f-4093-85d7-d81630198ea8”,
“name”: “Local File Trigger”,
“type”: “n8n-nodes-base.localFileTrigger”,
“position”: [
-160,
1040
],
“typeVersion”: 1
}
],
“connections”: {},
“pinData”: {}
}
‘’’