I am using the self-hosted version of n8n which the latest release is 1.76.3. I am also not using it for production. Simple personal use. As soon as this issue is resolved I intend on removing the SQL execution. However, at this time, I am having too many workflow failures due to this issue. With this in place, it has temporarily resolved the connection issue.
I just updated to 1.77.3 and now I am getting this error:
Error inserting: Cannot use a pool after calling end on the pool
My workflow is split into two. The main flow gets a sitemap, splits the bulk urls into individual urls, then starts a loop. The sub-flow gets the content and then inserts into pgvector store. Previously I was only able to get through about 100 pages before the flow would error/stall/crash.
I was hopeful that the issue had been repaired until I hit about page 500. Then the new issue came up.
I think I may have fixed my issue by adding a retry to the pgvector insertion. I was able to make it through 1166 page insertions with no failure.
problem persists, still running into problems when adding a bunch of documents into the pgvector node.
the fix PR is waiting for a review for 5 months?
I am using this in my postgres container as init command.
postgres -c max_connections=500 -c idle_session_timeout=5000
this specific issue disappeared.
of course my timeout and connections are calibrated on my db size and queries, which are very lightweight.
if you have further suggestions please share. at the beginning of each rag cycle (but could do that via crontab) I also run
vacuum full;
analyse ;
reindex database db-name ;
after setting up postgres like this, no more DB problems without need to restart the postgres container.
as a last resort we could create an error workflow that in case of postgres errors restarts the container
let me know if you have suggestions for further improvements on postgres performance, I realized there is substantial room for improvement.
here is a reference for all possible parameters, but I am no expert: