ChatGPT not referencing embeddings from Vector Database

[bluedog]

Describe the problem/error/question

I have a separate workflow which uses ChatGPT to summarise news articles, then pushes the summary plus the article metadata to Pinecone to store as embeddings:

An example of the content pushed to Pinecone:

I created a second workflow which would run at 6am and create a summary of the key points from the last 24 hours’ content in the Vector database:

What is the error message (if any)?

This workflow will frequently respond with something like "It seems I’m unable to retrieve the latest cybersecurity news summaries at this time. However, you can stay informed about recent incidents and trends by checking reputable cybersecurity news sources like Krebs on Security, Threatpost, or the Cybersecurity & Infrastructure Security Agency (CISA).

If you have specific incidents or topics in mind, I can assist you with details or provide guidance based on existing knowledge. Let me know how you would like to proceed!"

My understanding of the workflow I’ve set up is that it should just be referencing the data sent to Pinecone, but it seems to think it needs to reference outside sources and isn’t using the Vector Store I’ve connected it to.

Looking at the Vector Store Tool2 node, the input (what I presume is what the AI Agent node is providing to the vector store as a query) is very basic (e.g. “Provide a summary of the latest cybersecurity news and incidents over the past 24 hours.”) and the response from the connected Model in the 3 runs it attempted is “I don’t know.”

Clicking through to the connected OpenAI Chat Model1 node, it’s clear the content from the Vector Store is being provided along with the question, but the Model still only replies “I don’t know.”

Please share your workflow

See above

Share the output returned by the last node

It seems I’m unable to retrieve the latest cybersecurity news summaries at this time. However, you can stay informed about recent incidents and trends by checking reputable cybersecurity news sources like Krebs on Security, Threatpost, or the Cybersecurity & Infrastructure Security Agency (CISA).

If you have specific incidents or topics in mind, I can assist you with details or provide guidance based on existing knowledge. Let me know how you would like to proceed!

I expected that workflow should reference the embeddings in the Vector Database and create a summary using ChatGPT of the contents, but it seems to be interpreting the prompt as a request to resolve external resources, despite being provided the content by the Vector Store.

Information on your n8n setup

• n8n version: 1.77.3
• Database (default: SQLite): Default
• n8n EXECUTIONS_PROCESS setting (default: own, main): Default
• Running n8n via (Docker, npm, n8n cloud, desktop app): Docker
• Operating system: Windows 11 Pro

[bluedog]

Please note I’m completely new to AI, embeddings, and automation with n8n, I’m figuring it out as I go along.

I’m assuming embeddings are the way to go here, but given my use case, I feel like I might’ve been better off with a traditional relational database, and just storing the ChatGPT summaries as strings?

The reason for my thinking is that - as you would’ve seen from the content being pushed to Pinecone - it includes the original article metadata like the title, url, date published, etc. The intent of that is that I could ask ChatGPT to provide the summary as well as links to the original articles which were referenced.

Looking at the Pinecone DB, even though those fields like the publish date and url do make it in - there’s no relationship between those objects, or to the original summary.

That being the case - is it even possible for the AI Agent to even know what the summaries from the last 24 hours were, let alone, what the original title and URL for the article are?

As I mentioned - not even sure I’m on the right path here, any guidance on how to best address my original use case of creating a daily summary of the key topics in the article summaries would be greatly appreciated!

[Richard_Osborne]

I think you’re right, you might be better off with a simple database to store the articles and then have a step where you pull all records from the past 24 hours and give them to an LLM to summarise. Depending on the number of articles you might want to think about using Gemini 2.0 (use the OpenAI node and the Gemini OpenAI endpoint) because it has one of the biggest context limits and it’s cheap.

The problem with vector searching in your case is that it’s not really designed for simple storage and retrieval, it’s more like “Show me articles that mention AI news in the context of marketing” and the vector search can find the top semantically similar records in your vector store.