Claude Code CLI 2.1.282 Analysis

Measured 2026-09-25 21:01 UTC from this cycle’s benchmark runs, written up from those measurements alone, then checked by an independent reviewer. Where the wording and the tables disagree, the tables are correct.

Verdict

At the high reasoning setting, one task costs this much per session right now, cheapest to most expensive:

• haiku-4-5 $0.0144
• sonnet-5 $0.0295
• opus-5-5 $0.0344
• opus-4-8 $0.0482
• opus-5 $0.0740
• fable-5-1 $0.0771
• fable-5 $0.1051

At the medium setting:

• sonnet-5 $0.0270
• opus-5-5 $0.0299
• opus-4-8 $0.0457
• opus-5 $0.0587
• fable-5-1 $0.0646
• fable-5 $0.0931

Across the last 10 cycles at the high setting, each family moved like this, and stayed inside these dollar bands:

• fable-5: no clear direction, $0.1035 to $0.1139
• fable-5-1: drifted up, $0.0707 to $0.0771
• haiku-4-5: no clear direction, $0.0138 to $0.0148
• opus-4-8: no clear direction, $0.0460 to $0.0490
• opus-5: drifted up, $0.0667 to $0.0740
• opus-5-5: too few cycles to say, $0.0296 to $0.0344
• sonnet-5: no clear direction, $0.0275 to $0.0314

Across the last 10 cycles at the medium setting:

• fable-5: no clear direction, $0.0909 to $0.0987
• fable-5-1: no clear direction, $0.0626 to $0.0646
• opus-4-8: no clear direction, $0.0435 to $0.0469
• opus-5: no clear direction, $0.0527 to $0.0604
• opus-5-5: too few cycles to say, $0.0242 to $0.0305
• sonnet-5: no clear direction, $0.0267 to $0.0282

This cycle moved from the previous CLI version (2.1.281) to the current one (2.1.282), and the families with a same comparison moved very little:

• haiku at the high setting rose 0.8%
• sonnet at the high setting fell 1.2%
• sonnet at the medium setting fell 0.8%

In all three cases the individual runs behind those averages overlap heavily, so none of the moves stands out from normal run-to-run variation. Something may have shifted; this measurement cannot see it.

The spread between families is far larger than anything that moved this cycle: at the high setting, haiku-4-5 costs 86.3% less per task than fable-5.

Which cost figure is which – setting high

The same sessions summarized three ways. They disagree on purpose, and none is the corrected version of another.

• Mean $/session: multiplies out to real spend, which makes it the budgeting number, and it is what the Cost Trends chart plots.
• Median $/session: shrugs off an unusually cheap or expensive run, but it does not sum to real spend.
• Suite cost: the price of one full pass over every prompt in the catalog, the per-prompt medians added up, so it answers what a whole run costs rather than what one session costs.

family	models compared	mean $/sess	mean delta %	median $/sess	median delta %	suite $	suite delta %	prompts
haiku	haiku-4-5	0.0143 -> 0.0144	+0.8%	0.0141 -> 0.0157	+10.8%	0.1209 -> 0.1196	-1.1%	9
sonnet	sonnet-5	0.0299 -> 0.0295	-1.2%	0.0327 -> 0.0328	+0.2%	0.2483 -> 0.2412	-2.9%	9

Suite cost uses only the prompts both cycles ran, so the two sides always add up the same catalog.

Which cost figure is which – setting medium

The same sessions summarized three ways. They disagree on purpose, and none is the corrected version of another.

• Mean $/session: multiplies out to real spend, which makes it the budgeting number, and it is what the Cost Trends chart plots.
• Median $/session: shrugs off an unusually cheap or expensive run, but it does not sum to real spend.
• Suite cost: the price of one full pass over every prompt in the catalog, the per-prompt medians added up, so it answers what a whole run costs rather than what one session costs.

family	models compared	mean $/sess	mean delta %	median $/sess	median delta %	suite $	suite delta %	prompts
sonnet	sonnet-5	0.0273 -> 0.0270	-0.8%	0.0316 -> 0.0288	-8.8%	0.2231 -> 0.2238	+0.3%	9

Suite cost uses only the prompts both cycles ran, so the two sides always add up the same catalog.

What these terms mean

New to this report? Here is the vocabulary used below, in plain words. Skip ahead if you already know it.

• Session (also “task”): one complete run of one prompt by one model, start to finish. Every cost in this report is per session.
• Family: a model line, such as opus or sonnet. One family can cover more than one version, for example opus-4-8 and opus-5.
• Cache, warm and cold: the tool reuses recent context instead of sending it again, which is far cheaper. A session that gets to reuse it is “warm”; one that has to send everything fresh is “cold”. Whether a given session lands warm or cold is partly luck, and that luck moves the cost.
• Cache luck (or cache-hit luck): the cost swing caused by how many sessions happened to land warm rather than cold, rather than by any real change in the tool or the model.
• Blended cost: cost measured across every session, warm and cold alike. This is the real money figure, and it is what the cost feed shows. “Blended” says which sessions are counted, not how they are averaged; see “Which cost figure is which” above for the averaging.
• Warm-only cost: cost measured across warm sessions only. Removing the cold ones removes most of the luck, so this is the fair test of whether cost actually changed.
• Pooled: all sessions for a family added together into a single number, rather than split out per prompt.
• Mean cost per session: the ordinary average across every session. It multiplies out to real spend, so it is the budgeting figure and the one this report leads with.
• Median cost per session: the middle session once they are lined up by cost. It ignores an unusually cheap or expensive run, but it does not add up to real spend.
• Suite cost: the price of one full pass over every prompt in the catalog, found by taking each prompt’s median and adding those up. It is a total for the whole catalog, not an average per session, so it is a larger number than either figure above and is not comparable to them.
• Cell: one family and one prompt combination, for example haiku on the rename-refactor prompt. Cells hold few sessions, so a single cell is noisy.
• n: how many sessions sit behind a number. A larger n is more trustworthy.
• p-value, and “significant”: the same prompt run twice costs different amounts, so a p-value asks whether two sets of runs can be told apart at all. Line up every run from both cycles: if one cycle’s runs sit consistently higher, p is near 0 and the difference is called “significant”; if the runs are mixed together, p is near 1 and the two sets look like the same thing measured twice. Precisely, it is the chance of seeing a gap at least this lopsided if the two cycles were truly identical. A high p is not weak evidence of a change, it is no evidence either way.
• API-equivalent cost: dollars calculated at published API rates. It is not necessarily dollars billed, because a subscription plan may already cover the usage.

Pooled blended comparison

This is the real per-task spend: every session counted, including the unlucky ones that start with a cold cache. It is the mean dollars per session a user actually pays, and it is the figure the cost feed carries. Each setting is compared against its own previous run at the same setting.

Setting high, nightly-2.1.281 to nightly-2.1.282:

family	models compared	n (prev -> cur)	prev $/sess	cur $/sess	delta %	Mann-Whitney p
haiku	haiku-4-5	53 -> 53	0.0143	0.0144	+0.8%	0.919
sonnet	sonnet-5	53 -> 53	0.0299	0.0295	-1.2%	0.929

Setting medium, nightly-medium-2.1.281 to nightly-medium-2.1.282:

family	models compared	n (prev -> cur)	prev $/sess	cur $/sess	delta %	Mann-Whitney p
sonnet	sonnet-5	53 -> 53	0.0273	0.0270	-0.8%	0.874

Only families with a matching run in the previous cycle at the same setting can appear here, which is why the two settings list different families.

Pooled warm comparison (significance control)

Whether a change is believable is a separate question from what it cost. The warm view drops the sessions that had to build their cache from scratch, so a lucky or unlucky draw of cold starts cannot masquerade as a change in price. Comparing the individual warm run costs from each cycle, every p-value below sits far above the 0.05 line, so none of this cycle’s moves can be told apart from ordinary run-to-run variation.

Setting high:

family	models compared	n (prev -> cur)	prev $/sess	cur $/sess	delta %	Mann-Whitney p
haiku	haiku-4-5	45 -> 45	0.0132	0.0134	+2.1%	0.878
sonnet	sonnet-5	45 -> 45	0.0273	0.0267	-2.1%	0.884

Setting medium:

family	models compared	n (prev -> cur)	prev $/sess	cur $/sess	delta %	Mann-Whitney p
sonnet	sonnet-5	45 -> 45	0.0247	0.0245	-1.0%	0.865

What the reasoning setting costs

The same release was measured twice, once at each reasoning setting. This is the only place the two are held against each other. The change column measures medium against high, so a negative percentage means medium is the cheaper of the two, and the comparison is of the individual session costs from each setting.

model	high mean $	medium mean $	change	p	n
fable-5	0.1051	0.0931	-11.5%	0.111	53/53
fable-5-1	0.0771	0.0646	-16.1%	0.199	53/53
opus-4-8	0.0482	0.0457	-5.2%	0.460	53/53
opus-5	0.0740	0.0587	-20.6%	0.050	57/53
opus-5-5	0.0344	0.0299	-13.1%	0.374	53/53
sonnet-5	0.0295	0.0270	-8.4%	0.250	53/53

Medium is the cheaper setting for all six models measured both ways:

• fable-5 is 11.5% cheaper at medium
• fable-5-1 is 16.1% cheaper at medium
• opus-4-8 is 5.2% cheaper at medium
• opus-5 is 20.6% cheaper at medium
• opus-5-5 is 13.1% cheaper at medium
• sonnet-5 is 8.4% cheaper at medium

None of those six differences clears the 0.05 line, including opus-5 at p=0.050, so for every model here the gap between the two settings cannot be told apart from run-to-run variation. The consistent direction across all six is worth watching, but this cycle’s data cannot confirm it.

haiku-4-5 was measured at the high setting only, so it has no cross-setting figure.

Notable cells (secondary)

At the high setting, one prompt moved more than the rest: sonnet on rename-refactor, which asks the model to rename a function everywhere it appears, covering its definition, its export, and every call site across several files. The figures behind that cell:

• warm cost fell 18.5%
• 6 runs before and 6 runs after
• p=0.347, well above the 0.05 line

This is one cell out of 18 compared at once, and with that many simultaneous comparisons roughly one in twenty will look striking by chance alone, so a single cell is never a finding on its own. Treat it as something to check again next cycle. At the medium setting, no cell moved beyond the noise threshold.

Caveats

• Two views of cost are reported, and they answer different questions. The headline figure is the real per-task spend across every session, cache luck included, and it matches the dashboard. The warm-only figure strips out that random cache luck and is the control for whether a change is real. Judge “did cost change” by the warm comparison, not by the size of the headline move: a headline move with no significant warm move may simply be cache luck.
• The two cycles being compared ran on different CLI versions (2.1.281 against 2.1.282). Measuring that version-to-version effect is the point of the benchmark, but it also means the difference cannot be pinned on the model itself without a separate test that holds the CLI version fixed.
• The per-cell movers are one lens over 18 comparisons made at the same time. At the usual 0.05 threshold, roughly one in twenty such comparisons will look significant purely by chance, so a single standout cell is not a result.
• Token capture for Haiku is known to be incomplete, so every Haiku cost and token figure should be read as a floor rather than an exact amount.
• Timings are wall-clock and include the time the model spends thinking, and the costs here are calculated API-equivalent amounts rather than dollars actually billed.

---

Verification appendix

• Verifier verdict: PASS after 2 round(s).
• A first draft was rejected and revised; the objections were resolved.