Highlights
Model Analytics Highlights
Distilled 2026-11-21 02:24 UTC from the published section write-ups alone, then checked by an independent reviewer. Each section’s own page carries the detail and the tables behind these figures.
What this shows
The Model Analytics area tracks what a unit of work costs when you run the same fixed set of prompts through coding assistants, priced at each provider’s published API rates. There is one section per tool. Each one reports what a single task costs now at each reasoning setting, and whether a move since the previous release can be told apart from the ordinary variation between runs. The two sections use their own separate prompt catalogs and are not a like for like comparison.
Claude Benchmarks
This section measures what one task costs per session on the Claude Code CLI, running a frozen prompt catalog repeatedly and pricing it at list rates, reported separately at high and at medium reasoning. At high reasoning the current per-session costs run from $0.0016 for haiku-5-5 to $0.0903 for fable-5-1; at medium they run from $0.0137 for sonnet-5-5 to $0.0669 for opus-5. This cycle has no change figures against the previous CLI version: at high reasoning and at medium reasoning alike, no family has runs on both sides of the comparison, so no percentage move could be measured and nothing could be checked against normal run-to-run variation. Across the last 10 cycles at high reasoning, fable-5-1 drifted up within a band of $0.0771 to $0.0903 while every other family showed no clear direction, and across the last 10 cycles at medium reasoning no family showed a clear direction; note also that token capture for the Haiku models is incomplete, so their figures are lower bounds.
Codex Benchmarks
This section measures what a unit of work costs on Codex and whether it is getting cheaper, running its own frozen prompt catalog and pricing it at OpenAI’s list rates, reported separately at high and at medium reasoning. At high reasoning one task now costs from $0.0013 for luna-6 to $0.1098 for astra; at medium, from $0.0011 for luna-6 to $0.1127 for astra. Across the last 10 cycles at high reasoning, luna drifted down inside a band of $0.0027 to $0.0034, sol and terra show no clear direction, and astra, luna-6 and sol-6-1 have too few cycles to say. Comparing the individual session costs of this cycle and the previous CLI version, three moves can be told apart from ordinary run-to-run variation, all at high reasoning:
- luna-6 rose 22.7%, p=0.006
- sol-6-1 rose 31.9%, p=0.014
- terra rose 19.0%, p=0.002
Every other move cannot be told apart from ordinary run-to-run variation, which does not mean nothing changed; that test is not controlled for cache state, because this study cannot tell which sessions started with a warm cache, so it is weaker evidence than the same test run on warm sessions alone, and a result at one reasoning setting does not carry over to the other.
Verification appendix
- Verifier verdict: PASS after 2 round(s).
- A first draft was rejected and revised; the objections were resolved.