How reasoning models changed what an answer costs.
The prompting trick that became a training technique.
Prompting the model to show its reasoning steps before the final answer · dramatically improves math, logic, and multi-step tasks.
“CoT was a hack in 2022 and a core capability by 2025. The next bench will measure CoT efficiency, not CoT presence.”
Read full chapterHow models that think before answering work.
A model that generates internal reasoning tokens before producing the answer, trading inference cost for accuracy on math and logic.
“Reasoning models changed the pricing ladder: the frontier is no longer one price.”
Read full chapterSpending more compute at answer time.
Test-time compute is the computation a model spends at answer time · spending more (longer reasoning, several attempts) can raise accuracy without retraining the model.
A hard science benchmark reasoning models are judged on.
GPQA diamond is a knowledge benchmark tracked on BenchGecko. Graduate-level science questions written by PhD experts. Diamond subset contains questions where experts disagree, testing deep understanding.
A puzzle benchmark built to resist memorization.
ARC-AGI is a reasoning benchmark tracked on BenchGecko. Abstraction and Reasoning Corpus. Tests fluid intelligence through novel visual pattern recognition puzzles. Core measure of general intelligence.
Why reasoning answers cost more.
Reasoning models (o-series, Claude extended thinking, DeepSeek R1) bill hidden thinking tokens at the output rate · total cost can be 2-10× a non-reasoning query.
“Reasoning token billing is the finops gotcha of 2026. Cap your reasoning_effort before deploying.”
Read full chapterBy the end you understand how reasoning models work, how they are evaluated, and why their answers cost more than chat.
Seven terms that decode whether AI is overpriced, fairly priced, or criminally underpriced. Read in order.