Add Prompt Caching Support for Anthropic Models in OpenRouter Chat Model Node

Feature Requests
1

OpenRouter Chat Model

The idea is:

Add prompt caching configuration to the OpenRouter Chat Model node to enable caching for Anthropic models (Claude) through OpenRouter’s API. This would allow users to mark specific messages for caching using Anthropic’s cache_control mechanism, which OpenRouter supports.
The implementation would include:

  • Detection of Anthropic models (e.g., anthropic/claude-4-5-sonnet)
  • Configuration option to enable/disable prompt caching
  • TTL selection: 5 minutes (default) or 1 hour
  • Automatic application of cache_control markers to selected messages (e.g., system message, last N user messages)
  • Support for both simple and advanced caching strategies

My use case:

I’m building workflows with large context windows (character cards, documentation, RAG data) that need to be sent with every request. Currently, I’m paying full price for tokens that rarely change between requests.
With prompt caching:

  • Large documentation queries: I have a 50,000 token documentation context that I query multiple times. Without caching, each query costs full price for all 50k tokens. With caching, after the first request, I’d pay 0.1x the price for those tokens (90% savings).
  • Multi-turn conversations: Long conversations where the history grows with each turn. Caching previous messages would reduce costs significantly as the conversation gets longer.
  • RAG workflows: When retrieving and sending large chunks of data to Claude for analysis, the retrieved context often stays the same across multiple questions.

I think it would be beneficial to add this because:

  1. Significant cost reduction: Cache reads are charged at 0.1x the original price for Anthropic models through OpenRouter. For workflows with large context windows, this can reduce costs by 80-90%.
  2. Faster response times: Cached content is processed faster, leading to quicker responses.

Any resources to support this?

Are you willing to work on this?

Yes, I’m willing to contribute to this feature if guidance is provided on the preferred implementation approach.

2

This is a great idea that really fits the needs of workflows with large contexts. Caching would significantly reduce costs and speed up responses, especially for Anthropic models via OpenRouter. The option to configure TTL and target specific messages for caching makes it very flexible. In my view, it’s a valuable feature to implement quickly for better handling of large token volumes.

1 Like