I am pasting a markdown document into my system instructions and was expecting to use Gemini’s implicit cache to reduce cost and speed (my document is very static).
Yet, looking at Gemini logs, each new session creates a new version of the cache but each subsequent call in the same session use cached tokens. What is n8n doing to create this new cache at the start of each session? At the moment I am using the in-built memory for testing.
I have tried both a workflow with agent node and the new agent functionality and I get the same behaviour.
This is Gemini’s implicit cache behaving normally, n8n isn’t creating anything.
Implicit caching matches on the exact token prefix of the request. Inside one session every call sends the same system prompt plus the same growing history, so from call 2 onward the prefix matches and you see cached tokens. A new session is a different conversation, so the request no longer matches what the previous session sent, and Gemini builds a fresh cache entry. On top of that the implicit cache only lives about 5 minutes, so even two sessions with identical prompts an hour apart won’t share it.
Since your doc is static and big, explicit caching is the real fix. You create a cachedContent on the Gemini API once, with your markdown inside it and a TTL you choose, then reference that cache in the request. n8n has no node for the setup step but it’s just one HTTP Request call to create it, and after that you pass the cache name into the agent’s request. Every session then rides the same cache and you get the reduced cached-token price across all sessions.
One thing worth double checking if you stay on implicit: if anything session-specific (a session id, a timestamp) ends up inside the system prompt, the prefix changes on every request and you’ll never get a hit.
Hi @pomegran, a new n8n session starts a fresh chat history. Gemini manages the implicit cache, and two sessions can get a hit if their requests start with enough of the same content. Google recommends sending similar prompts close together, but doesn’t document a fixed five-minute lifetime for implicit caching.
I’d test the first turn of two fresh sessions back to back with the same model, Markdown, tools and opening question. In the Gemini logs, compare usageMetadata.promptTokenCount and usageMetadata.cachedContentTokenCount. Which model are you using, and roughly how many tokens is the Markdown? If the shared text is below that model’s caching minimum, the growing chat history could explain why you see hits only on later turns.
If you want to reuse one explicit cache, the Google Gemini Chat Model node in n8n 2.47.7 has no cachedContent field. For a model that supports explicit caching, you can create the cache once, then use an HTTP Request node to call generateContent with the returned cache name while it’s valid.
Gemini’s implicit cache isn’t tied to n8n sessions at all. It matches on the exact prefix of the request, so a “new cache per session” usually means the first request of each session doesn’t share an identical prefix with the previous one. A few things I’d check, in order:
Anything dynamic in the system message. If the system prompt contains an expression that changes per execution or per chat (e.g. {{ $now }}, the session ID, the user’s name, a date), the prefix differs and the cache misses. Put the static markdown doc at the very top of the system message and move anything dynamic below it (or into the user message).
Timing. Implicit caching is best-effort and short-lived. Within one session your calls are seconds apart, so they hit; if your test sessions are several minutes apart, the earlier cache may simply have expired. Try starting two new sessions back to back and compare.
Isolate n8n vs Gemini. Send the same system instruction + a short first message twice with an HTTP Request node straight to generateContent, from two “sessions” (just two executions). If those hit the cache but the agent doesn’t, the agent is changing the prefix (tools/memory/formatting); if they also miss, it’s Gemini’s side.
If you need a guaranteed cache: use explicit caching instead. Create the cache once with an HTTP Request node (POST /v1beta/cachedContents with your doc as the system instruction and a ttl), store the returned name, then call generateContent with cachedContent set to it. As far as I know the Gemini Chat Model node doesn’t expose cachedContent, so this means calling the API via HTTP Request rather than the agent node. With a static doc it’s the reliable way to get the discount on every session.
If you can share the system message (with the doc trimmed out), happy to look at what might be shifting the prefix.
Thanks for all the responses. Apologies but I can’t paste in the text as it has company data.
I have been down all these routes regarding checking the system prompt which is completely static. The only things that changes is the input so I can see no reason why it’s not using the same cache - nothing visible in the Gemini API logs anyway. A static system prompt for the first turn returns this in usageMetadata:
I have diffed the system prompt from the logs and they are identical. The only thing that changes is the input (for history) and a new session from n8n (whatever that does in background). I really don’t understand why it always ignores cache on the new n8n session - every follow-up turn in a session uses cache.
I understand the workarounds around building the cache - just don’t understand this behaviour. Even the Gemini logs don’t make it any clearer
So after more investigation (and I was surprised about this and implicit caching!) is that the first input influences Gemini caching.
I started a new session in n8n and said “hello” as my first input. This did not use cache.
I started another new session and said exactly the same first input, “hello” and this actually used cache.
So even though my system prompt is around 35,000 tokens, the very first input drives how the cache is used.
I tested this out and checked the Gemini logs:
Session 1
Input 1: Hello (not cached)
Input 2: How do I donate? (cached)
Session 2
Input 1: Hello (cached)
Input 2: What courses do you run (cached)
So a workaround would be to use the model to generate the greeting message at the start of each session with the same triggered input e.g. “start_message” that the user does not see (rather than the widget doing this as my widget currently works). Would need to work out the numbers as to whether this would be worth it. People often start conversations and don’t say anything and close the chat widget meaning a wasted input.
Still, at least I know now why I was getting this behaviour. The first input does influence how cache is used - I thought it was purely on size of the prompt.