What we actually measured.

The 8.6% figure above is the median saving among OPTIMIZED executions only — it is not an average saving, not a typical saving for every prompt, not an overall corpus saving, and not a guaranteed saving. The middle half of those executions saved between 4% and 13.3% (mean 8.2%). See “Result classes, in plain English” below for what the other executions did.

Every one of the 1,400 executions in this benchmark falls into exactly one of three outcomes:

A large ALREADY_EFFICIENT count is not a weakness in this result — it's the point. If the deterministic pipeline has nothing safe or useful to remove, it does not manufacture a saving to show one anyway. That's the same behaviour you'll see if you run an already-tight prompt through the product itself.

• 70 test prompts (“fixtures”): 52 main, 8 stress-test, and 10 deliberately already-efficient
• 4 optimisation styles × 5 model targets, run against every fixture
• 1,400 total executions, all through the same real optimizer entry point the product uses — never a separate or simplified code path
• A versioned, source-controlled corpus (currently v1.0.0) — a future corpus version is never silently compared against this one without saying so
• A frozen release-benchmark system: the V1 optimizer's transformation behaviour cannot change without a new versioned benchmark run and a passing regression suite first
• Automated checks (“hard gates”) that fail the benchmark outright on any real corruption, any unjustified token overhead, or any already-efficient prompt being expanded instead of left alone

We'd rather you read this section than skip it.

• This is Token Optimiser's own committed test corpus — 70 fixtures — not a sample of every possible real-world prompt.
• Savings vary materially by prompt. Most of the executions in this corpus (69%) were already efficient enough that nothing was changed.
• Token savings in this benchmark do not measure or prove any change in downstream answer quality — that's a different question this benchmark doesn't answer.
• Estimated monetary savings shown in the product depend on token-pricing assumptions, not your actual provider invoice — see the product's own methodology notes for how that estimate is calculated.
• Model-target differentiation is currently limited in V1: this corpus does not show a meaningfully different savings outcome between Claude, ChatGPT, Gemini, and Grok targets.
• None of the figures on this page should be read as a guaranteed saving for any prompt you run.

The three result classes above exist because a tool that always claims to have improved your prompt isn't telling the truth on every run. Token Optimiser is built to tell the three outcomes apart and show you which one you got — including the conservative case where it keeps your original prompt exactly as written rather than risk a change to something that mattered.

If you want practical techniques for spotting removable tokens yourself before reaching for a tool, see our guide to reducing prompt tokens.