A cost pattern that bit me on a batch workflow, sharing in case it saves someone a surprise bill.
I had a Loop Over Items node feeding a Basic LLM Chain, classifying ~400 rows. The prompt looked cheap: a paragraph of instructions plus one row of data per call. What I forgot is that the instructions aren’t sent once — they ride along on every single iteration. So the fixed system/instruction block, times 400 items, was quietly the majority of the token spend, and the actual per-row data was the small part.
The thing that made it obvious was boring bookkeeping: I looked at one call’s input tokens, saw how much was the static instructions vs the row, and multiplied by the item count. The static part dominated, and it scaled with the number of items even though it never changed.
What helped, none of it clever:
- Trim the instruction block hard for looped nodes specifically. A 300-token preamble you’d never notice on a single run becomes 300 × N in a loop.
- Where the task allows it, batch several rows into one call (e.g. 10 rows per prompt) instead of one row per call — you pay the fixed instructions once per 10 rows instead of once per row. Fewer, fatter calls, same instructions amortized.
- Keep few-shot examples out of the looped prompt unless they measurably lift accuracy — they’re the most expensive thing to re-send N times.
Rough magnitude for me: batching 10 rows per call on that 400-item run cut total tokens to roughly a third, same output quality, because the instruction overhead got paid 40 times instead of 400.
Do others batch inside Loop Over Items for AI nodes, or is there a cleaner n8n-native way to amortize a fixed prompt across items that I’m missing? Curious how people structure this when the per-item work is small but the instructions are big.
إعجاب واحد (1)
Your 3x number matches what I get independently, and the ratio is the part people miss — on a typical classification preamble the instructions are 90%+ of every call, so the data you actually care about is a rounding error in the bill.
I wanted that number before committing to a run, so I built a small thing that reads the prompt template and reports it:
How it works:
- Splits the template at the first placeholder — everything before it is static, everything after varies per row.
- Counts both halves and prices the per-item loop against a batched one.
- Reports the break-even and what batching removes in redundant instruction copies.
On a 218-token classifier across 400 rows it shows instructions at 96% of each call, and batching at 10 cutting input tokens from 90,400 to 11,920.
The thing that surprised me while building it: prompt caching looks like the obvious fix here and usually does nothing. A cacheable prefix has a minimum length — 1,024 tokens on Sonnet, 4,096 on Haiku 4.5 — and a normal classification preamble is under it. Below the minimum, caching is skipped silently. No error, and cache_creation_input_tokens stays 0, so a workflow can look cached and bill uncached indefinitely.
Where it breaks: the minimum isn’t monotonic across generations, so moving to a cheaper model to save money can quietly remove a cache discount you were relying on. And caching matches a prefix, so a placeholder near the top of the template means there’s no stable prefix at all and no tuning will produce a hit — instructions first, row data last. The tool checks both and says which side of the line you’re on.
Token counts are estimated offline within ~15-20% unless you set an API key, which is fine for a go/no-go call and not fine for quoting a client.
Prompt caching on the Anthropic side handles exactly this, and it stacks with your batching instead of replacing it. Mark the static instruction block in the system parameter with cache_control: {type: "ephemeral"}, the first call pays full price, and every call inside the cache window (5 minutes by default) pays roughly a tenth of the input price for that same prefix. I’ve only used it through a raw HTTP Request node against the Anthropic API directly, not through the Basic LLM Chain node, so I can’t say whether n8n’s built-in node passes the cache_control field through, worth checking before assuming it does. On 400 near-identical loop calls that alone would likely beat your row-batching on token cost, though batching still wins once you care about fewer round trips too.
Prompt caching on the Anthropic side handles exactly this, and it stacks with your batching instead of replacing it. Mark the static instruction block in the system parameter with cache_control: {type: "ephemeral"}, the first call pays full price, and every call inside the cache window (5 minutes by default) pays roughly a tenth of the input price for that same prefix. I’ve only used it through a raw HTTP Request node against the Anthropic API directly, not through the Basic LLM Chain node, so I can’t say whether n8n’s built-in node passes the cache_control field through, worth checking before assuming it does. On 400 near-identical loop calls that alone would likely beat your row-batching on token cost, though batching still wins once you care about fewer round trips too.
Both of these are the right follow-ups, and they point at the same thing from opposite ends.
@denismagel — prompt caching is exactly the escape hatch for this shape, and you’re right that it stacks with batching rather than replacing it. The one caveat I’d add is that caching doesn’t make the cost go away, it moves it between buckets: the first call pays a cache-write (which is priced above normal input — roughly 1.25× on Anthropic), and every hit inside the window pays a cache-read (~0.1×). So on a 400-iteration loop it’s a big win, but the break-even depends on how many iterations land inside the 5-minute window and how big the static prefix is — a loop that stalls between rows can pay the write premium repeatedly and quietly claw back the savings. Worth measuring rather than assuming. On whether n8n’s built-in LLM node forwards cache_control on the system block — I haven’t confirmed it does, and your instinct to check before relying on it is the safe call; the raw HTTP Request node against the Anthropic API is the path I’d trust to actually set it.
@Serhii — splitting the template at the first placeholder is the correct decomposition, and it’s the same one I lean on: the static prefix is deterministic (you can count it exactly before the run, no distribution), and only the per-row tail needs an estimate. Nice that your break-even tool reports both halves. If it’s useful, npx @wartzar-bee/tokenscope (Apache-2.0) does the priced version of that split — it takes real usage and breaks the dollar figure into input / output / cache-write (~1.25×) / cache-read (~0.1×), which is what you need to actually answer denismagel’s caching question with a number instead of a hunch (“did the cache window pay for its write premium on this loop, or not?”).
The trap that bit me on top of all this: the static prefix isn’t fixed across time. The day someone edits the instruction block or adds a tool, that 90%-of-every-call floor jumps — and because it re-bills every iteration of every loop, it inflates the whole workflow’s cost at once, arriving as an ordinary template edit. I gate it in CI (@wartzar-bee/ci-guardrail, also Apache-2.0, fails the build past a max-usd ceiling) so a prompt tweak that doubles the static half gets caught on the PR instead of on the invoice.
Question back at both of you: has either of you found a clean way to hold the cache warm across a slow loop — i.e. keep iterations inside the 5-minute window when row processing itself takes seconds — or do you just accept the occasional re-write when the loop stalls?
Answering my own open question from above, because it turns out to answer yours too: the built-in node does pass it through. LmChatAnthropic has a “Prompt Caching” option under Options, default Disabled, with two values, 5 Minutes and 1 Hour. It sets invocationKwargs.cache_control = { type: ‘ephemeral’, ttl: } before handing the model off, so the raw HTTP Request detour I described isn’t needed any more.
That reframes the warm-cache question rather than solving it: you don’t keep the window open, you pick the longer one. A loop whose rows take seconds sits comfortably inside an hour, so the stall case stops being a re-write problem at all. The trade sits on the write side, the longer TTL is priced above the 5-minute one, so it only pays back if the run is actually long enough to use it. A 400-row loop that finishes in two minutes is cheaper on 5m.
One thing worth checking before measuring either: the option caches the system prompt, the tool definitions and the conversation history together. If your loop rebuilds the tool list per iteration, the prefix isn’t stable and you pay writes no matter which TTL you picked.
Hi, I’m the founder of Rynler. Your 400-row example suggests one extra metric: total billed cost divided by correct, usable classifications, counting rejected outputs too. Batching already helped in your test, so I wouldn’t assume changing providers improves it.
I’ve prepared a plain-chat HTTP Request example with label validation, without agent tools or explicit cache controls. Would it be useful to adapt it to 10 non-sensitive rows from your workflow and compare it with your current setup? The example has been checked locally; this Rynler comparison has not been run in production.