Is it possible to dump a huge textfile into the model context at the beginning?

Questions
1

Describe the problem/error/question

I have a preprocessed text file from a PDF that I would like to insert directly into the LLM context. I know, RAG is generally what is used here, but Gemini offering a million context window makes me want to try this first before going down RAG. I am processing a document regarding laws, and it is about 100k Tokens. I just want to put it into the model at the beginning of any chat and let the system prompt do the heavy lifting as the user asks questions.

What is the error message (if any)?

Please share your workflow

Chat → AI Agent so far with Gemini as the Model.

(Select the nodes on your canvas and use the keyboard shortcuts CMD+C/CTRL+C and CMD+V/CTRL+V to copy and paste the workflow.)

Share the output returned by the last node

Information on your n8n setup

  • n8n version: Latest
  • Database (default: SQLite): Default
  • n8n EXECUTIONS_PROCESS setting (default: own, main): default
  • Running n8n via (Docker, npm, n8n cloud, desktop app): Docker locally
  • Operating system: Windows
2

Yes, you should be able to put it in the system message of the agent:

CleanShot 2025-03-11 at 17.23.38
CleanShot 2025-03-11 at 17.23.38436×648 28 KB

3

The system prompt is sent with every message to the user (at least, according to the logs). If I have a 100k system message this will balloon. Ideally I just need it once in the message/context history with the system message driving the AI to focus on prior context. Kind of like attaching a PDF/Text at the beginning and then having a conversation flowing from it.

Is it possible to send a ‘mock’ message before the user’s first message to get it into their message history?

4

Yeah you could force send a message as “user” or “assistant” type in some cases, and that would remain as an initial message within history if setup right.

5

How do I do this in N8N? Is there a way to send a message before the user’s first message?

6

I believe there is a better way, but you could make the request in an openai node initially for example, grabs its session ID, and provide that to the agent node’s memory instead of its own session ID.

7

The system prompt is sent with every message to the user

The system prompt gets written to the message history on the first interaction and is retrieved for each subsequent interaction. It isn’t added to the memory again on subsequent interactions, but is already there.

Memory is how you give context to the model for subsequent interactions. If the system message wasn’t sent each time, the model wouldn’t have access to it and wouldn’t be able to answer anything about it.

Hope that helps?

8

You could however manipulate the memory before calling the agent if you want to. If you’re using the simple memory (what used to be called ‘Window buffer memory’), you can use the ‘Chat memory manager’ node to do this.

9

Oh interesting, thanks for that info, I personally assumed it was sent with all requests even with the agent node.

10

It is effectively sent with all requests since it’s at the start of the memory and the contents of memory are added in for all requests. But it’s not added an extra time to the memory for each interaction; it’s only in there once.

1 Like