A cost pattern that bit me on a batch workflow, sharing in case it saves someone a surprise bill.
I had a Loop Over Items node feeding a Basic LLM Chain, classifying ~400 rows. The prompt looked cheap: a paragraph of instructions plus one row of data per call. What I forgot is that the instructions aren’t sent once — they ride along on every single iteration. So the fixed system/instruction block, times 400 items, was quietly the majority of the token spend, and the actual per-row data was the small part.
The thing that made it obvious was boring bookkeeping: I looked at one call’s input tokens, saw how much was the static instructions vs the row, and multiplied by the item count. The static part dominated, and it scaled with the number of items even though it never changed.
What helped, none of it clever:
- Trim the instruction block hard for looped nodes specifically. A 300-token preamble you’d never notice on a single run becomes 300 × N in a loop.
- Where the task allows it, batch several rows into one call (e.g. 10 rows per prompt) instead of one row per call — you pay the fixed instructions once per 10 rows instead of once per row. Fewer, fatter calls, same instructions amortized.
- Keep few-shot examples out of the looped prompt unless they measurably lift accuracy — they’re the most expensive thing to re-send N times.
Rough magnitude for me: batching 10 rows per call on that 400-item run cut total tokens to roughly a third, same output quality, because the instruction overhead got paid 40 times instead of 400.
Do others batch inside Loop Over Items for AI nodes, or is there a cleaner n8n-native way to amortize a fixed prompt across items that I’m missing? Curious how people structure this when the per-item work is small but the instructions are big.
1 « J'aime »
Your 3x number matches what I get independently, and the ratio is the part people miss — on a typical classification preamble the instructions are 90%+ of every call, so the data you actually care about is a rounding error in the bill.
I wanted that number before committing to a run, so I built a small thing that reads the prompt template and reports it:
How it works:
- Splits the template at the first placeholder — everything before it is static, everything after varies per row.
- Counts both halves and prices the per-item loop against a batched one.
- Reports the break-even and what batching removes in redundant instruction copies.
On a 218-token classifier across 400 rows it shows instructions at 96% of each call, and batching at 10 cutting input tokens from 90,400 to 11,920.
The thing that surprised me while building it: prompt caching looks like the obvious fix here and usually does nothing. A cacheable prefix has a minimum length — 1,024 tokens on Sonnet, 4,096 on Haiku 4.5 — and a normal classification preamble is under it. Below the minimum, caching is skipped silently. No error, and cache_creation_input_tokens stays 0, so a workflow can look cached and bill uncached indefinitely.
Where it breaks: the minimum isn’t monotonic across generations, so moving to a cheaper model to save money can quietly remove a cache discount you were relying on. And caching matches a prefix, so a placeholder near the top of the template means there’s no stable prefix at all and no tuning will produce a hit — instructions first, row data last. The tool checks both and says which side of the line you’re on.
Token counts are estimated offline within ~15-20% unless you set an API key, which is fine for a go/no-go call and not fine for quoting a client.
Prompt caching on the Anthropic side handles exactly this, and it stacks with your batching instead of replacing it. Mark the static instruction block in the system parameter with cache_control: {type: "ephemeral"}, the first call pays full price, and every call inside the cache window (5 minutes by default) pays roughly a tenth of the input price for that same prefix. I’ve only used it through a raw HTTP Request node against the Anthropic API directly, not through the Basic LLM Chain node, so I can’t say whether n8n’s built-in node passes the cache_control field through, worth checking before assuming it does. On 400 near-identical loop calls that alone would likely beat your row-batching on token cost, though batching still wins once you care about fewer round trips too.
Prompt caching on the Anthropic side handles exactly this, and it stacks with your batching instead of replacing it. Mark the static instruction block in the system parameter with cache_control: {type: "ephemeral"}, the first call pays full price, and every call inside the cache window (5 minutes by default) pays roughly a tenth of the input price for that same prefix. I’ve only used it through a raw HTTP Request node against the Anthropic API directly, not through the Basic LLM Chain node, so I can’t say whether n8n’s built-in node passes the cache_control field through, worth checking before assuming it does. On 400 near-identical loop calls that alone would likely beat your row-batching on token cost, though batching still wins once you care about fewer round trips too.