Claude Benchmarks
Suite cost (USD)
- sonnet-5 (high)
- fable-5 (high)
- haiku-4-5
- opus-4-8 (high)
- opus-5 (high)
- fable-5-1 (high)
- opus-5-5 (high)
- sonnet-5-5 (high)
- haiku-5-5
- sonnet-4-6 (high)
- fable-5-2 (high)
Data table
| Claude Code CLI version | sonnet-5 (high) | fable-5 (high) | haiku-4-5 | opus-4-8 (high) | sonnet-5 (medium) | opus-5 (high) | opus-5 (low) | opus-5 (max) | opus-5 (medium) | opus-5 (xhigh) | fable-5 (medium) | opus-4-8 (medium) | fable-5 (max) | fable-5 (xhigh) | opus-4-8 (max) | opus-4-8 (xhigh) | sonnet-5 (max) | sonnet-5 (xhigh) | fable-5-1 (high) | fable-5-1 (medium) | fable-5-1 (max) | fable-5-1 (xhigh) | opus-5-5 (high) | opus-5-5 (medium) | sonnet-5-5 (high) | sonnet-5-5 (medium) | haiku-5-5 | sonnet-4-6 (high) | sonnet-4-6 (medium) | fable-5-2 (high) | fable-5-2 (medium) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2.1.196 | 0.39959 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.202 | 0.307441 | 1.585473 | 0.138659 | 0.787706 | 0.292861 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.204 | 0.43591 | 1.589345 | 0.136938 | 0.7473 | 0.423905 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.205 | 0.401936 | 1.412947 | 0.137007 | 0.68471 | 0.399699 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.206 | 0.415858 | 1.452328 | 0.134095 | 0.711829 | 0.411794 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.207 | 0.313161 | 1.169142 | 0.151735 | 0.494431 | 0.292131 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.208 | 0.301529 | 1.052804 | 0.137217 | 0.473321 | 0.271329 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.210 | 0.282338 | 1.052505 | 0.135879 | 0.478824 | 0.261952 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.211 | 0.304312 | 1.042501 | 0.135169 | 0.441384 | 0.257755 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.212 | 0.304967 | 1.037562 | 0.131874 | 0.449024 | 0.266073 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.214 | 0.298263 | 1.00327 | 0.140986 | 0.470604 | 0.270236 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.215 | 0.289674 | 0.98978 | 0.129231 | 0.403363 | 0.265379 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.216 | 0.269861 | 0.932445 | 0.143514 | 0.419973 | 0.250677 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.217 | 0.29384 | 0.970618 | 0.138043 | 0.405417 | 0.256946 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.218 | 0.284298 | 0.993526 | 0.128458 | 0.402994 | 0.265344 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.219 | 0.286448 | 1.040209 | 0.132845 | 0.408153 | 0.261634 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.220 | 0.276864 | 0.987417 | 0.125554 | 0.40121 | 0.264421 | 0.657846 | 0.432675 | 0.815055 | 0.566123 | 0.762884 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.221 | 0.286049 | 0.977372 | 0.131999 | 0.401626 | — | 0.65512 | — | 0.873209 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.222 | 0.29028 | 1.098188 | 0.131929 | 0.443637 | 0.268545 | 0.646637 | — | — | 0.56267 | — | 0.892297 | 0.380577 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.223 | 0.296099 | 0.997915 | 0.136063 | 0.419563 | 0.26388 | 0.654669 | — | — | 0.563948 | — | 1.019459 | 0.417344 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.224 | 0.292122 | 1.085281 | 0.131863 | 0.441856 | 0.264969 | 0.662316 | — | — | 0.574012 | — | 0.931341 | 0.397382 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.225 | 0.29478 | 1.084727 | 0.136727 | 0.424706 | 0.26217 | 0.68277 | — | — | 0.582747 | — | 0.945328 | 0.392409 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.226 | 0.298991 | 1.059551 | 0.143038 | 0.435342 | 0.262172 | 0.688851 | — | — | 0.593155 | — | 0.924563 | 0.423615 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.227 | 0.296432 | 1.074569 | 0.132082 | 0.474679 | 0.260069 | 0.72558 | — | — | 0.567964 | — | 0.992387 | 0.43579 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.228 | 0.309403 | 1.06973 | 0.133659 | 0.422149 | 0.253376 | 0.679035 | — | — | 0.573721 | — | 0.912215 | 0.413112 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.229 | 0.30224 | 1.069366 | 0.136153 | 0.452609 | 0.259779 | 0.683558 | — | — | 0.564781 | — | 0.909137 | 0.424274 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.231 | 0.301637 | 1.046028 | 0.134929 | 0.442898 | 0.261313 | 0.709022 | — | — | 0.574353 | — | 0.946331 | 0.407458 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.232 | 0.296741 | 1.046169 | 0.126988 | 0.448157 | 0.263367 | 0.636067 | — | — | 0.593776 | — | 0.934624 | 0.429926 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.233 | 0.299494 | 1.039348 | 0.131054 | 0.397076 | 0.263568 | 0.660987 | — | — | 0.608725 | — | 0.893037 | 0.423469 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.234 | 0.283871 | 1.049789 | 0.123664 | 0.415698 | 0.267799 | 0.627984 | — | — | 0.55549 | — | 0.884193 | 0.415903 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.235 | 0.28956 | 1.027151 | 0.128207 | 0.450877 | 0.251189 | 0.621802 | — | — | 0.533325 | — | 0.882008 | 0.436997 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.236 | 0.287515 | 0.957125 | 0.132953 | 0.454512 | 0.273248 | 0.631873 | — | — | 0.516582 | — | 0.923976 | 0.445696 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.237 | 0.295928 | 1.062191 | 0.131669 | 0.429356 | 0.254227 | 0.624628 | — | — | 0.512943 | — | 0.923061 | 0.42662 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.238 | 0.273972 | 0.959072 | 0.135126 | 0.418491 | 0.26626 | 0.620326 | — | — | 0.505544 | — | 0.902759 | 0.421965 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.239 | 0.277027 | 1.005888 | 0.132268 | 0.44007 | 0.256925 | 0.668432 | — | — | 0.521478 | — | 0.863074 | 0.412674 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.240 | 0.296605 | 1.050034 | 0.129466 | 0.464848 | — | 0.627648 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.241 | 0.284886 | 1.019615 | 0.134739 | 0.432911 | 0.264402 | 0.634396 | — | — | 0.495076 | — | 0.89114 | 0.440323 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.243 | 0.281347 | 1.065704 | 0.128777 | 0.439055 | 0.258768 | 0.639663 | — | — | 0.510795 | — | 0.895002 | 0.418189 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.245 | 0.277618 | 1.078632 | 0.127699 | 0.437461 | 0.275164 | 0.62811 | — | — | 0.506187 | — | 0.938824 | 0.4392 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.246 | 0.292935 | 1.026344 | 0.128329 | 0.430404 | 0.262555 | 0.598752 | — | — | 0.512927 | — | 0.90136 | 0.445008 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.247 | 0.274649 | 1.069361 | 0.12853 | 0.452896 | 0.262064 | 0.612011 | — | 0.881819 | 0.509641 | 0.759401 | 0.944405 | 0.431513 | 1.468829 | 2.045082 | 0.710167 | 0.549591 | 0.427245 | 0.319708 | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.248 | 0.241969 | 0.850789 | 0.116072 | 0.372271 | 0.221059 | 0.537779 | — | 0.807233 | 0.445608 | 0.674777 | 0.786564 | 0.379949 | 2.842755 | 1.903706 | 0.662301 | 0.457867 | 0.387419 | 0.280323 | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.250 | 0.243362 | 0.945936 | 0.119991 | 0.372947 | 0.22632 | 0.55282 | — | 0.828557 | 0.448003 | 0.696182 | 0.760813 | 0.354301 | 2.829469 | 1.871419 | 0.650926 | 0.45787 | 0.354036 | 0.281948 | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.251 | 0.250543 | 0.870722 | 0.118075 | 0.385571 | 0.235105 | 0.53371 | — | 0.787098 | 0.415421 | 0.72368 | 0.784472 | 0.359907 | 3.017697 | 1.857015 | 0.670861 | 0.433931 | 0.36842 | 0.282411 | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.252 | 0.25062 | 0.891468 | 0.12186 | 0.372919 | 0.236097 | 0.527568 | — | — | 0.405826 | — | 0.775454 | 0.361293 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.257 | 0.25659 | 0.87196 | 0.114944 | 0.380457 | 0.225249 | 0.525918 | — | — | 0.439004 | — | 0.738726 | 0.363476 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.258 | 0.247104 | 0.819051 | 0.11879 | 0.383242 | — | 0.535461 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.259 | 0.248202 | 0.839904 | 0.118003 | 0.386357 | 0.233196 | 0.526803 | — | — | 0.430426 | — | 0.747331 | 0.360691 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.260 | 0.248078 | 0.836241 | 0.117441 | 0.367197 | 0.223846 | 0.554445 | — | — | 0.415243 | — | 0.787331 | 0.365427 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.261 | 0.242446 | 0.867681 | 0.1213 | 0.37043 | 0.222379 | 0.554465 | — | — | 0.44107 | — | 0.74602 | 0.368507 | — | — | — | — | — | — | 0.56144 | 0.466341 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.263 | 0.246518 | 0.928523 | 0.12043 | 0.383042 | 0.229081 | 0.580929 | — | — | 0.412345 | — | 0.717638 | 0.366587 | — | — | — | — | — | — | 0.573391 | 0.454946 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.265 | 0.251823 | 0.880232 | 0.121911 | 0.386437 | 0.227746 | 0.601206 | — | — | 0.429159 | — | 0.806786 | 0.345241 | — | — | — | — | — | — | 0.542945 | 0.468057 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.266 | 0.251599 | 0.825064 | 0.118225 | 0.380074 | 0.226531 | 0.582992 | — | — | 0.439395 | — | 0.790282 | 0.358274 | — | — | — | — | — | — | 0.531119 | 0.47197 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.267 | 0.260124 | 0.913793 | 0.123804 | 0.380495 | 0.223534 | 0.584541 | — | — | 0.463294 | — | 0.756674 | 0.359009 | — | — | — | — | — | — | 0.536231 | 0.471379 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.268 | 0.247574 | 0.847196 | 0.120601 | 0.402512 | 0.233332 | 0.572696 | — | — | 0.467918 | — | 0.74891 | 0.371418 | — | — | — | — | — | — | 0.571852 | 0.479094 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.269 | 0.247532 | 0.883161 | 0.125496 | 0.37619 | 0.225884 | 0.595832 | — | — | 0.46909 | — | 0.792319 | 0.367078 | — | — | — | — | — | — | 0.549446 | 0.480273 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.270 | 0.248077 | 0.872274 | 0.120672 | 0.36345 | 0.217774 | 0.606586 | — | 0.788327 | 0.449836 | 0.69398 | 0.779055 | 0.383992 | 1.302987 | 0.970925 | 0.612335 | 0.461792 | 0.359038 | 0.280361 | 0.581184 | 0.476668 | 2.029729 | 1.025853 | — | — | — | — | — | — | — | — | — |
| 2.1.271 | 0.248705 | 0.875075 | 0.117043 | 0.379788 | 0.228741 | 0.607335 | — | — | 0.468559 | — | 0.78133 | 0.364044 | — | — | — | — | — | — | 0.602664 | 0.471878 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.272 | 0.254296 | 0.885335 | 0.12593 | 0.37832 | 0.228185 | 0.650927 | — | — | 0.472291 | — | 0.727833 | 0.367279 | — | — | — | — | — | — | 0.564856 | 0.485264 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.278 | 0.255792 | 0.876415 | 0.114951 | 0.416464 | 0.228845 | 0.63328 | — | — | 0.48095 | — | 0.735923 | 0.364167 | — | — | — | — | — | — | 0.592477 | 0.488103 | — | — | — | — | — | — | — | — | — | — | — |
| 2.1.280 | 0.252207 | 0.882213 | 0.116382 | 0.365332 | 0.230838 | 0.602693 | — | — | 0.439992 | — | 0.750666 | 0.373698 | — | — | — | — | — | — | 0.572017 | 0.47293 | — | — | 0.262347 | 0.215071 | — | — | — | — | — | — | — |
| 2.1.281 | 0.2483 | 0.866807 | 0.120914 | 0.375279 | 0.223112 | 0.622124 | — | — | 0.478544 | — | 0.7162 | 0.374442 | — | — | — | — | — | — | 0.583911 | 0.470826 | — | — | 0.270879 | 0.2318 | — | — | — | — | — | — | — |
| 2.1.282 | 0.24122 | 0.885678 | 0.11963 | 0.381661 | 0.22376 | 0.622266 | — | — | 0.483949 | — | 0.723319 | 0.361705 | — | — | — | — | — | — | 0.609478 | 0.481824 | — | — | 0.276317 | 0.225929 | — | — | — | — | — | — | — |
| 2.1.283 | 0.246044 | 0.85939 | 0.12412 | 0.392725 | 0.229795 | 0.6315 | — | — | 0.479399 | — | 0.740795 | 0.368393 | — | — | — | — | — | — | 0.583529 | 0.487228 | — | — | 0.265512 | 0.234032 | — | — | — | — | — | — | — |
| 2.1.284 | 0.247821 | 0.833854 | 0.119582 | 0.400604 | 0.2354 | 0.608328 | — | — | 0.469789 | — | 0.726728 | 0.368811 | — | — | — | — | — | — | 0.56529 | 0.486625 | — | — | 0.262267 | 0.225697 | 0.118675 | 0.101985 | — | — | — | — | — |
| 2.1.285 | 0.246052 | 0.8616 | 0.120087 | 0.381505 | 0.228927 | 0.671058 | — | — | 0.478074 | — | 0.728824 | 0.371018 | — | — | — | — | — | — | 0.572613 | 0.473703 | — | — | 0.259544 | 0.231185 | 0.11693 | 0.102549 | — | — | — | — | — |
| 2.1.286 | 0.245959 | 0.861661 | 0.124246 | 0.394839 | 0.23236 | 0.603531 | — | — | 0.474349 | — | 0.744979 | 0.367086 | — | — | — | — | — | — | 0.558759 | 0.488417 | — | — | 0.268258 | 0.231545 | 0.113905 | 0.10114 | — | — | — | — | — |
| 2.1.287 | 0.255694 | 0.860943 | 0.130512 | 0.398207 | 0.22398 | 0.580606 | — | — | 0.486412 | — | 0.732217 | 0.372231 | — | — | — | — | — | — | 0.570521 | 0.472712 | — | — | 0.260396 | 0.229905 | 0.112408 | 0.097659 | — | — | — | — | — |
| 2.1.288 | 0.252245 | 0.850935 | 0.114665 | 0.392857 | 0.228834 | 0.653173 | — | — | 0.48468 | — | 0.733234 | 0.380491 | — | — | — | — | — | — | 0.6001 | 0.482764 | — | — | 0.265522 | 0.226715 | 0.117025 | 0.101566 | — | — | — | — | — |
| 2.1.289 | 0.25334 | 0.909701 | 0.118378 | 0.397466 | 0.230586 | 0.613609 | — | — | 0.476427 | — | 0.758254 | 0.38755 | — | — | — | — | — | — | 0.58707 | 0.498698 | — | — | 0.270811 | 0.233209 | 0.116311 | 0.095377 | — | — | — | — | — |
| 2.1.290 | 0.264398 | 0.913671 | 0.127023 | 0.376629 | 0.240915 | 0.653922 | — | — | 0.528847 | — | 0.743275 | 0.371305 | — | — | — | — | — | — | 0.59559 | 0.493815 | — | — | 0.267004 | 0.248693 | 0.114914 | 0.106292 | — | — | — | — | — |
| 2.1.293 | 0.27512 | — | 0.128976 | 0.411107 | 0.249953 | 0.70118 | — | — | 0.539082 | — | — | 0.377414 | — | — | — | — | — | — | 0.548037 | 0.468933 | — | — | 0.265703 | 0.24093 | 0.114735 | 0.100852 | 0.012407 | 0.317412 | 0.286383 | — | — |
| 2.1.301 | 0.274533 | — | 0.129101 | 0.403438 | 0.240284 | 0.67537 | — | — | 0.517589 | — | — | 0.402355 | — | — | — | — | — | — | 0.612923 | 0.494517 | — | — | 0.280173 | 0.257248 | 0.117212 | 0.102852 | 0.013514 | — | — | — | — |
| 2.1.302 | 0.27476 | — | 0.12945 | 0.40878 | 0.245385 | 0.740242 | — | — | 0.530708 | — | — | 0.380468 | — | — | — | — | — | — | 0.574091 | 0.504586 | — | — | 0.273363 | 0.239183 | 0.115889 | 0.10331 | 0.013046 | — | — | — | — |
| 2.1.303 | 0.276517 | — | 0.137723 | 0.389024 | 0.246957 | 0.737249 | — | — | 0.523192 | — | — | 0.364898 | — | — | — | — | — | — | 0.596419 | 0.49159 | — | — | 0.287614 | 0.239036 | 0.126507 | 0.101658 | 0.012874 | — | — | — | — |
| 2.1.304 | 0.285516 | — | 0.12591 | 0.413152 | 0.241888 | 0.704236 | — | — | 0.581775 | — | — | 0.368391 | — | — | — | — | — | — | 0.588224 | 0.501371 | — | — | 0.272162 | 0.243686 | 0.118036 | 0.099702 | 0.013014 | — | — | — | — |
| 2.1.305 | 0.269313 | — | 0.128956 | 0.40663 | 0.239664 | 0.687509 | — | — | 0.540101 | — | — | 0.368287 | — | — | — | — | — | — | 0.552777 | 0.499523 | — | — | 0.275369 | 0.23456 | 0.115157 | 0.100287 | 0.01288 | — | — | — | — |
| 2.1.306 | 0.292285 | — | 0.128355 | 0.415916 | 0.240675 | 0.696166 | — | — | 0.536437 | — | — | 0.35114 | — | — | — | — | — | — | 0.594312 | 0.48451 | — | — | 0.26567 | — | 0.114957 | — | 0.013645 | — | — | — | — |
| 2.1.309 | 0.271531 | — | 0.131535 | 0.404778 | 0.246778 | 0.718432 | — | — | 0.544257 | — | — | 0.38016 | — | — | — | — | — | — | 0.61849 | 0.500838 | — | — | 0.272296 | 0.246157 | 0.118141 | 0.098034 | 0.012776 | — | — | — | — |
| 2.1.310 | 0.2726 | — | 0.126666 | 0.410776 | 0.243638 | 0.662596 | — | — | 0.534537 | — | — | 0.39158 | — | — | — | — | — | — | 0.612138 | 0.493487 | — | — | 0.283874 | 0.260408 | 0.118133 | 0.103902 | 0.013107 | — | — | — | — |
| 2.1.311 | 0.25525 | — | 0.121814 | 0.387315 | 0.241023 | 0.712077 | — | — | 0.568956 | — | — | 0.36072 | — | — | — | — | — | — | 0.599546 | 0.50011 | — | — | 0.288596 | 0.238603 | 0.122342 | 0.100448 | 0.012357 | — | — | — | — |
| 2.1.312 | 0.272963 | — | 0.129309 | 0.399972 | 0.239228 | 0.720583 | — | — | 0.54346 | — | — | 0.370936 | — | — | — | — | — | — | 0.648775 | — | — | — | 0.276535 | — | 0.118213 | — | 0.013254 | — | — | — | — |
| 2.1.318 | 0.285755 | — | 0.120486 | 0.394122 | 0.241499 | 0.709363 | — | — | 0.527107 | — | — | 0.384219 | — | — | — | — | — | — | 0.561625 | 0.486796 | — | — | 0.281136 | 0.227097 | 0.11595 | 0.097665 | 0.012456 | — | — | — | — |
| 2.1.321 | 0.272182 | — | 0.125245 | 0.438731 | 0.241937 | 0.651376 | — | — | 0.548398 | — | — | 0.366116 | — | — | — | — | — | — | 0.59697 | 0.504014 | — | — | 0.264678 | 0.238846 | 0.116889 | 0.097395 | 0.012379 | — | — | 0.527394 | 0.348641 |
| 2.1.322 | 0.259945 | — | 0.12616 | 0.439888 | 0.239271 | 0.672562 | — | — | 0.529529 | — | — | 0.369822 | — | — | — | — | — | — | 0.595599 | 0.481277 | — | — | 0.279134 | 0.243456 | 0.119007 | 0.099649 | 0.01294 | — | — | 0.472055 | 0.353539 |
| 2.1.323 | 0.278131 | — | 0.123447 | 0.40784 | 0.238585 | 0.699652 | — | — | 0.523971 | — | — | 0.376634 | — | — | — | — | — | — | 0.590615 | 0.492405 | — | — | 0.285388 | 0.234916 | 0.120937 | 0.101051 | 0.013744 | — | — | 0.584363 | 0.357154 |
| 2.1.324 | 0.264695 | — | 0.12685 | 0.420963 | 0.244583 | 0.713214 | — | — | 0.49357 | — | — | 0.372694 | — | — | — | — | — | — | 0.622137 | 0.514318 | — | — | 0.278203 | 0.249188 | 0.118546 | 0.100307 | 0.013532 | — | — | 0.442838 | 0.336239 |
| 2.1.325 | 0.274011 | — | 0.130489 | 0.436478 | 0.241792 | 0.719755 | — | — | 0.482108 | — | — | 0.374326 | — | — | — | — | — | — | 0.63244 | 0.504899 | — | — | 0.295232 | 0.233659 | 0.11875 | 0.100323 | 0.013501 | — | — | 0.431351 | 0.34793 |
| 2.1.326 | 0.267158 | — | 0.12479 | 0.414725 | 0.248715 | 0.694019 | — | — | 0.538314 | — | — | 0.384619 | — | — | — | — | — | — | 0.594971 | 0.49 | — | — | 0.291506 | 0.231234 | 0.11361 | 0.107711 | 0.013273 | — | — | 0.504757 | 0.355548 |
| 2.1.327 | 0.266702 | — | 0.131739 | 0.427732 | 0.242077 | 0.721012 | — | — | 0.531878 | — | — | 0.413209 | — | — | — | — | — | — | 0.608295 | 0.495829 | — | — | 0.27601 | 0.237103 | 0.118883 | 0.10206 | 0.013062 | — | — | 0.443485 | 0.362947 |
| 2.1.328 | 0.27141 | — | 0.125919 | 0.407449 | 0.256895 | 0.69162 | — | — | 0.531173 | — | — | 0.392702 | — | — | — | — | — | — | 0.614574 | 0.518693 | — | — | 0.279167 | 0.242042 | 0.113204 | 0.098269 | 0.013588 | — | — | 0.557931 | 0.395189 |
| 2.1.329 | 0.279635 | — | 0.127901 | 0.401872 | 0.238696 | 0.699724 | — | — | 0.551297 | — | — | 0.366651 | — | — | — | — | — | — | 0.592504 | 0.499057 | — | — | 0.287136 | 0.242259 | 0.126165 | 0.10162 | 0.014354 | — | — | 0.517004 | 0.385566 |
| 2.1.330 | 0.26751 | — | 0.130854 | 0.407189 | 0.240757 | 0.686058 | — | — | 0.555319 | — | — | 0.394668 | — | — | — | — | — | — | 0.640767 | 0.476908 | — | — | 0.272417 | 0.239644 | 0.11854 | 0.098903 | 0.013456 | — | — | 0.534376 | 0.356969 |
Data table
| Metric | fable-5-1 (high) | fable-5-2 (high) | haiku-4-5 | haiku-5-5 | opus-4-8 (high) | opus-5 (high) | opus-5-5 (high) | sonnet-5 (high) | sonnet-5-5 (high) |
|---|---|---|---|---|---|---|---|---|---|
| Cost per session (USD)mean | 0.090261 | 0.067033 | 0.016641 | 0.001599 | 0.056164 | 0.08685 | 0.035399 | 0.034214 | 0.015943 |
| Cost per session (USD)median | 0.095991 | 0.042908 | 0.016336 | 0.001523 | 0.059662 | 0.081807 | 0.030281 | 0.036727 | 0.014024 |
| Output tokens | 600 | 489 | 789 | 984.5 | 454 | 778 | 435 | 390 | 398 |
| Input tokens | 66 | 6 | 26 | 8 | 6 | 8 | 6 | 8 | 6 |
| Cache-read tokens | 57994 | 56719 | 67029 | 69971.5 | 48728 | 74605 | 52689 | 114695 | 52587 |
| Cache-write tokens | 1172 | 627 | 1289 | 1386 | 743 | 1573 | 633 | 1041 | 0 |
| Input + cache tokens | 59038 | 57684 | 67744 | 74835.5 | 52829 | 75979 | 53375 | 119418 | 52930 |
| API calls | 3 | 3 | 3 | 4 | 3 | 4 | 3 | 4 | 3 |
| Tool calls | 2 | 2.5 | 2 | 3.5 | 2 | 3 | 2 | 3 | 3 |
| Thinking blocks | 0 | 0 | 3 | 2 | 1 | 1 | 0 | 1 | 0 |
| Latency (s, wall-clock) | 15.811 | 5.8215 | 8.343 | 6.485 | 7.488 | 14.788 | 5.1225 | 8.646 | 6.877 |
| Suite cost (USD) | No published column carried Suite cost (USD) for this cycle. | ||||||||
Data table
| Metric | fable-5-1 (medium) | fable-5-2 (medium) | opus-4-8 (medium) | opus-5 (medium) | opus-5-5 (medium) | sonnet-5 (medium) | sonnet-5-5 (medium) |
|---|---|---|---|---|---|---|---|
| Cost per session (USD)mean | 0.064946 | 0.048559 | 0.050127 | 0.066884 | 0.030919 | 0.029556 | 0.013662 |
| Cost per session (USD)median | 0.064206 | 0.040979 | 0.051846 | 0.062671 | 0.026153 | 0.032392 | 0.010789 |
| Output tokens | 473 | 400 | 444 | 573 | 367 | 368 | 344 |
| Input tokens | 66 | 6 | 6 | 6 | 6 | 8 | 6 |
| Cache-read tokens | 54614 | 53235 | 52388 | 56221 | 49182 | 118725 | 52215 |
| Cache-write tokens | 630 | 0 | 171 | 1122 | 454 | 749 | 0 |
| Input + cache tokens | 58696 | 56885 | 52829 | 57345 | 53054 | 119343 | 52731 |
| API calls | 3 | 3 | 3 | 3 | 3 | 4 | 3 |
| Tool calls | 2 | 2 | 2 | 2 | 2 | 3 | 2 |
| Thinking blocks | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
| Latency (s, wall-clock) | 15.267 | 4.51 | 7.23 | 8.023 | 6.705 | 8.217 | 5.418 |
| Suite cost (USD) | No published column carried Suite cost (USD) for this cycle. | ||||||
Claude Code CLI 2.1.330 Analysis
Measured 2026-11-21 02:22 UTC from this cycle’s benchmark runs, written up from those measurements alone, then checked by an independent reviewer. Where the wording and the tables disagree, the tables are correct.
Verdict
In the high reasoning setting, one task costs this per session, cheapest to most expensive:
- haiku-5-5 $0.0016
- sonnet-5-5 $0.0159
- haiku-4-5 $0.0166
- sonnet-5 $0.0342
- opus-5-5 $0.0354
- opus-4-8 $0.0562
- fable-5-2 $0.0670
- opus-5 $0.0868
- fable-5-1 $0.0903
In the medium reasoning setting, the same task costs:
- sonnet-5-5 $0.0137
- sonnet-5 $0.0296
- opus-5-5 $0.0309
- fable-5-2 $0.0486
- opus-4-8 $0.0501
- fable-5-1 $0.0649
- opus-5 $0.0669
Across the last 10 cycles in the high setting, each family moved like this, with the dollar band it stayed inside:
- fable-5-1 drifted up, $0.0771 to $0.0903
- fable-5-2 no clear direction, $0.0593 to $0.0755
- haiku-4-5 no clear direction, $0.0147 to $0.0167
- haiku-5-5 no clear direction, $0.0015 to $0.0018
- opus-4-8 no clear direction, $0.0527 to $0.0602
- opus-5 no clear direction, $0.0827 to $0.0901
- opus-5-5 no clear direction, $0.0351 to $0.0402
- sonnet-5 no clear direction, $0.0312 to $0.0357
- sonnet-5-5 no clear direction, $0.0154 to $0.0180
Across the last 10 cycles in the medium setting:
- fable-5-1 no clear direction, $0.0649 to $0.0781
- fable-5-2 no clear direction, $0.0473 to $0.0529
- opus-4-8 no clear direction, $0.0460 to $0.0527
- opus-5 no clear direction, $0.0607 to $0.0686
- opus-5-5 no clear direction, $0.0305 to $0.0318
- sonnet-5 no clear direction, $0.0288 to $0.0308
- sonnet-5-5 no clear direction, $0.0134 to $0.0146
This cycle has no change figures against the previous CLI version. In the high setting and in the medium setting alike, no family has runs on both sides of the 2.1.329 to 2.1.330 comparison, so no percentage move could be measured and nothing could be checked against normal run-to-run variation. Costs now and the bands above are what this cycle supports.
One other thing: in the high setting the two ends of the list are far apart. Measured against fable-5-1, haiku-5-5 comes in 98.2% lower, and comparing the individual run costs of the two families gives p=0.000.
Which cost figure is which – setting high
No family has sessions on both sides, so there is nothing to reconcile.
Which cost figure is which – setting medium
No family has sessions on both sides, so there is nothing to reconcile.
What these terms mean
New to this report? Here is the vocabulary used below, in plain words. Skip ahead if you already know it.
- Session (also “task”): one complete run of one prompt by one model, start to finish. Every cost in this report is per session.
- Family: a model line, such as opus or sonnet. One family can cover more than one version, for example opus-4-8 and opus-5.
- Cache, warm and cold: the tool reuses recent context instead of sending it again, which is far cheaper. A session that gets to reuse it is “warm”; one that has to send everything fresh is “cold”. Whether a given session lands warm or cold is partly luck, and that luck moves the cost.
- Cache luck (or cache-hit luck): the cost swing caused by how many sessions happened to land warm rather than cold, rather than by any real change in the tool or the model.
- Blended cost: cost measured across every session, warm and cold alike. This is the real money figure, and it is what the cost feed shows. “Blended” says which sessions are counted, not how they are averaged; see “Which cost figure is which” above for the averaging.
- Warm-only cost: cost measured across warm sessions only. Removing the cold ones removes most of the luck, so this is the fair test of whether cost actually changed.
- Pooled: all sessions for a family added together into a single number, rather than split out per prompt.
- Mean cost per session: the ordinary average across every session. It multiplies out to real spend, so it is the budgeting figure and the one this report leads with.
- Median cost per session: the middle session once they are lined up by cost. It ignores an unusually cheap or expensive run, but it does not add up to real spend.
- Suite cost: the price of one full pass over every prompt in the catalog, found by taking each prompt’s median and adding those up. It is a total for the whole catalog, not an average per session, so it is a larger number than either figure above and is not comparable to them.
- Cell: one family and one prompt combination, for example haiku on the rename-refactor prompt. Cells hold few sessions, so a single cell is noisy.
- n: how many sessions sit behind a number. A larger n is more trustworthy.
- p-value, and “significant”: the same prompt run twice costs different amounts, so a p-value asks whether two sets of runs can be told apart at all. Line up every run from both cycles: if one cycle’s runs sit consistently higher, p is near 0 and the difference is called “significant”; if the runs are mixed together, p is near 1 and the two sets look like the same thing measured twice. Precisely, it is the chance of seeing a gap at least this lopsided if the two cycles were truly identical. A high p is not weak evidence of a change, it is no evidence either way.
- API-equivalent cost: dollars calculated at published API rates. It is not necessarily dollars billed, because a subscription plan may already cover the usage.
Pooled blended comparison
This is the headline lens: real per-task spend in dollars per session, counting every session of the cycle, which is what a user actually pays.
Setting high (nightly-2.1.329 to nightly-2.1.330):
No family has sessions on both sides – no comparison possible.
Setting medium (nightly-medium-2.1.329 to nightly-medium-2.1.330):
No family has sessions on both sides – no comparison possible.
So for both settings there is no pooled blended change to report this cycle. The current per-session levels for each setting are the ones given above.
Pooled warm comparison (significance control)
This lens answers the separate question of whether a change is real, by setting aside the luck of which runs started with a warm cache and comparing only the warm runs from each cycle.
Setting high:
No family has sessions on both sides – no comparison possible.
Setting medium:
No family has sessions on both sides – no comparison possible.
With no family measured on both sides at either setting, there is no significance test to run this cycle. That does not mean costs held still: it means this measurement cannot see a move either way.
What the reasoning setting costs
Blended mean $/session per model, medium measured against high. A negative percentage means medium is the cheaper setting. The p-value is Mann-Whitney over the two pooled session sets; this test is available here because both sides carry per-session rows.
| model | high mean $ | medium mean $ | change | p | n |
|---|---|---|---|---|---|
fable-5-1 |
0.0903 | 0.0649 | -28.0% | 0.010 | 61/53 |
fable-5-2 |
0.0670 | 0.0486 | -27.6% | 0.337 | 54/53 |
opus-4-8 |
0.0562 | 0.0501 | -10.7% | 0.416 | 61/53 |
opus-5 |
0.0868 | 0.0669 | -23.0% | 0.014 | 61/59 |
opus-5-5 |
0.0354 | 0.0309 | -12.7% | 0.414 | 54/53 |
sonnet-5 |
0.0342 | 0.0296 | -13.6% | 0.042 | 61/53 |
sonnet-5-5 |
0.0159 | 0.0137 | -14.3% | 0.193 | 54/53 |
Medium is the cheaper setting in every row of this table. Comparing the individual run costs at each setting, three of those differences clear the 0.05 line:
- fable-5-1 costs 28.0% less at medium, p=0.010
- opus-5 costs 23.0% less at medium, p=0.014
- sonnet-5 costs 13.6% less at medium, p=0.042
For the other four, the difference cannot be told apart from normal run-to-run variation:
- fable-5-2 costs 27.6% less at medium, p=0.337
- opus-4-8 costs 10.7% less at medium, p=0.416
- opus-5-5 costs 12.7% less at medium, p=0.414
- sonnet-5-5 costs 14.3% less at medium, p=0.193
Two models were measured at one setting only and so have no cross-setting figure: haiku-4-5 and haiku-5-5.
Notable cells (secondary)
This lens looks for individual prompts that moved more than the rest. There is nothing to report: in the high setting and in the medium setting, no cell moved beyond the noise threshold, and with no family measured on both sides there were zero cells available to compare in either setting. Single prompts are a watch-next-cycle signal anyway, never a result on their own: each cell holds only about five runs, and scanning many prompts at once means some will look like movers by chance, so any such cell is quoted with its run count and its p-value and read as something to confirm later.
Caveats
- Two cost lenses are reported. The headline is the real per-task spend, counting every session, including the luck of whether a run started with a warm cache; that is the figure that matches the dashboard. The warm-only lens strips that cache luck out and is the control for whether a change is real. Judge “did cost change” by the warm comparison, not by the headline move. A headline move with no matching warm result may just be cache luck rather than a real change.
- The compared cycles ran on different CLI versions (2.1.329 then 2.1.330). Any pooled difference therefore measures the effect of the CLI version, which is what this benchmark exists to track. Do not read it as a change in the model itself unless a run that holds the CLI version fixed says so.
- Token capture for the Haiku models is known to be incomplete, so their cost and token figures should be treated as lower bounds: the true numbers are at least that much.
- Timing is wall-clock, so it includes the time the model spends thinking. Cost is calculated as an API-equivalent figure, not read off a bill, so it is an estimate of spend rather than dollars actually charged.
Verification appendix
- Verifier verdict: PASS after 1 round(s).