Utilize cache for AI - Anthropic

Feature Requests
1

The idea is:

Anthropic allows for cached data which brings cost down by 90% for example when creating context while vectorizing big documents. Anthropic allows for this, check link below. In use-cases where you need to use big document you can save a lot of money and global resources

My use case:

Saving money when using AI in n8n

I think it would be beneficial to add this because:

Everyone, bcs everyone wants to save money.

Any resources to support this?

https://www.anthropic.com/news/prompt-caching

Are you willing to work on this?

Yes

2

i definitely support this idea, will look forward towards its implementation

1 Like
3

For more complex AI Agents, this is a must have. I have a complex sales agent powered by Anthropic however it sometimes does 28 runs before coming up with a solid output. That comes at a pricey tag atm. Any ideas if this will be implemented at all?

2 Likes
4

This! Add only caching to system message for example as it’s used almost all time!
EDIT:
I’ve checked the code and adding a feature that will enable caching the system prompt will be so easy. Please do it!

2 Likes
5

This would be so immensely valuable to reduce token usage and latency :pray:

3 Likes
6

If someone can implement this I can pay for ready implementation :smiley:

2 Likes
7

Yes this would be extremely useful to implement this on the model level so that it can be used across all modules.

Another use case on my end: I use very long system prompts with the Information Extractor module to then only output one word but input like 300 and then run through 1000s of rows always entering the same system prompt.

2 Likes
8

This is a must have for agent building. Otherwise costs are almost 10x.
Please implement it. Thanks!!

2 Likes
9

yes. must have.

1 Like
10

It definitly is a must have, could be an option : prompt caching (on/off) in the anthropic chat model node

1 Like
11

Ive managed to solve the prompt caching problem using a quick custom proxy server for the anthropic api then just swapping out the base url in the anthropic node.

I created a simple FastAPI server that forwards the request on to anthropic and adds the cache headers to the system prompt. meaning no changes are required to n8n and prompt caching is working as expected!

it’s defo more of a work around than a permanent solution but it was easy to setup and is working well.

1 Like
12

It’s quite easy to do prompt caching with an HTTP Request node as documented here: