Context token limit exceeded in AI Agent chatbot workflow

[Ashaff_Hussain]

Describe the problem/error/question

I’m trying to fetch meeting transcripts from pgvector DB that contains the transcripts in embedding and the metadata as a separate column. However when I query the table, I get a max token 128k limit exceeded error. How do I fix this so that only the most relevant data is retrieved

What is the error message (if any)?

Bad request - please check your parameters

This model’s maximum context length is 128000 tokens. However, your messages resulted in 192821 tokens (192665 in the messages, 156 in the functions). Please reduce the length of the messages or functions.

Please share your workflow

<
{
“nodes”: [
{
“parameters”: {
“public”: true,
“options”: {}
},
“type”: “@n8n/n8n-nodes-langchain.chatTrigger”,
“typeVersion”: 1.1,
“position”: [
-1160,
-120
],
“id”: “97270982-f1a1-48cf-9292-ae220831b307”,
“name”: “When chat message received”,
“webhookId”: “25ecfd07-3a20-41d7-8004-6a059ca87f84”
},
{
“parameters”: {
“trigger”: [
“app_mention”
],
“channelId”: {
“__rl”: true,
“value”: “C088WK0UPHU”,
“mode”: “id”
},
“options”: {}
},
“type”: “n8n-nodes-base.slackTrigger”,
“typeVersion”: 1,
“position”: [
-1160,
40
],
“id”: “db57e7d6-37a2-436f-aeb0-72ad0fe2fc52”,
“name”: “Receive Slack Message”,
“webhookId”: “b0766072-795e-4932-8899-2d36bd2d3786”
},
{
“parameters”: {
“select”: “channel”,
“channelId”: {
“__rl”: true,
“value”: “C088WK0UPHU”,
“mode”: “id”
},
“text”: “={{ $json.output }}”,
“otherOptions”: {}
},
“type”: “n8n-nodes-base.slack”,
“typeVersion”: 2.2,
“position”: [
-480,
40
],
“id”: “3f9ed387-4ec3-4c45-a084-165cfae90808”,
“name”: “Return Results”,
“webhookId”: “000ca34c-9d58-45ca-b7d6-7d01ad7ea965”
},
{
“parameters”: {
“promptType”: “define”,
“text”: “={{ $json.chatInput||$json.text}}”,
“options”: {
“maxIterations”: 2
}
},
“type”: “@n8n/n8n-nodes-langchain.agent”,
“typeVersion”: 1.7,
“position”: [
-900,
40
],
“id”: “8edce352-d3dd-4699-94e2-5a0c1c5ffe94”,
“name”: “AI Agent”,
“alwaysOutputData”: true,
“retryOnFail”: true
},
{
“parameters”: {
“model”: {
“__rl”: true,
“value”: “gpt-4o”,
“mode”: “list”,
“cachedResultName”: “gpt-4o”
},
“options”: {
“temperature”: 0.5
}
},
“type”: “@n8n/n8n-nodes-langchain.lmChatOpenAi”,
“typeVersion”: 1.2,
“position”: [
-1000,
240
],
“id”: “09746fba-476e-4d46-918e-c72e0acde342”,
“name”: “OpenAI Chat Model”
},
{
“parameters”: {
“mode”: “retrieve-as-tool”,
“toolName”: “Call_Transcripts”,
“toolDescription”: “You are connected to a Postgres pgvector store containing meeting transcript data along with associated metadata (including meeting id, title, summary, participants, created_date, transcript text, and summary points). A chat request has been received that asks for meeting transcript data matching specific criteria. Your task is to:\n\nFilter by Metadata:\nFirst, scan the metadata fields (e.g., title, summary, participants, created_date, summary points) to isolate records that match the criteria specified in the chat request.\nEnsure that the filtering leverages existing indexes on the metadata, so that the system does not perform a full scan on all transcript data.\n”,
“tableName”: “grain_transcripts”,
“topK”: 400,
“includeDocumentMetadata”: false,
“options”: {}
},
“type”: “@n8n/n8n-nodes-langchain.vectorStorePGVector”,
“typeVersion”: 1,
“position”: [
-640,
220
],
“id”: “fb488195-00a9-425b-8bf4-1a1fbeedd1f0”,
“name”: “Postgres PGVector Store1”
},
{
“parameters”: {
“options”: {}
},
“type”: “@n8n/n8n-nodes-langchain.embeddingsOpenAi”,
“typeVersion”: 1.2,
“position”: [
-580,
400
],
“id”: “a5a0c5fe-ee22-46f8-95c7-4d54f7b56fff”,
“name”: “Embeddings OpenAI1”
}
],
“connections”: {
“When chat message received”: {
“main”: [
[
{
“node”: “AI Agent”,
“type”: “main”,
“index”: 0
}
]
]
},
“Receive Slack Message”: {
“main”: [
[
{
“node”: “AI Agent”,
“type”: “main”,
“index”: 0
}
]
]
},
“AI Agent”: {
“main”: [

]
},
“OpenAI Chat Model”: {
“ai_languageModel”: [
[
{
“node”: “AI Agent”,
“type”: “ai_languageModel”,
“index”: 0
}
]
]
},
“Postgres PGVector Store1”: {
“ai_tool”: [
[
{
“node”: “AI Agent”,
“type”: “ai_tool”,
“index”: 0
}
]
]
},
“Embeddings OpenAI1”: {
“ai_embedding”: [
[
{
“node”: “Postgres PGVector Store1”,
“type”: “ai_embedding”,
“index”: 0
}
]
]
}
},
“pinData”: {
“When chat message received”: [
{
“action”: “sendMessage”,
“chatInput”: “list all calls availabe for analysis. Use the meta data column in the grain_transcripts table in the PGvector database”
}
]
},
“meta”: {
“templateCredsSetupCompleted”: true
}
}
/>

Share the output returned by the last node

Information on your n8n setup

• n8n version: - 1.76.1 (Self Hosted)
• Database (default: SQLite): - Postgresql PG vector
• n8n EXECUTIONS_PROCESS setting (default: own, main):
• Running n8n via (Docker, npm, n8n cloud, desktop app): - npm
• Operating system: MAC OS

[solomon]

Hey @Ashaff_Hussain,

Instead of sending the full transcript directly to the AI Agent, try to simplify the text.

For example:

[2024-11-12 15:42:00]
John: What are we planning for the next quarter?
[2024-11-12 15:42:50]
Anna: We have the goal of selling 30% more than we did this quarter, due to increases in demand in sector X and a bigger sales team.

Could become this:

John: What are we planning for the next quarter? Anna: We have the goal of selling 30% more than we did this quarter, due to increases in demand in sector X and a bigger sales team.

That way you would save tokens and even a 2h meeting would probably fit into the context window.

Try to clean up the transcript, removing all the unecessary information and line breaks. The AI will understand it.

To do the clean up, you can send a sample of the transcript to ChatGPT and ask for JavaScript that would run in a Code node in n8n. I always do that and it works incredibly well.
.

If my reply answers your question, please remember to mark it as a solution.