Reasoning Models: How 'Thinking' AI Actually Works

A reasoning model is a language model trained to work through a problem step by step before answering — spending extra compute at inference time (“thinking”) in exchange for a large jump in accuracy on tasks that require real deliberation: math, logic, planning, and code.

Ask an older language model a hard math question and it would blurt out an answer instantly — often a confidently wrong one. Ask a reasoning model the same question and it pauses, works through the problem, and then responds. That pause is the whole innovation.

What “thinking” really means

Under the hood, a reasoning model generates a hidden stretch of intermediate text — a chain of thought — before it commits to a final answer. It’s effectively working through the problem on scratch paper: breaking it into steps, trying an approach, catching its own mistakes, and revising. Only the polished result comes back to you.

This is often called test-time compute (or inference-time compute). The classic way to make models smarter was to make them bigger and train them longer. Reasoning models add a second lever: spend more compute when answering a specific hard question. For easy questions you barely need it; for a tricky proof or a multi-file code change, that extra deliberation is where the accuracy comes from.

From fixed budgets to adaptive thinking

Early reasoning models exposed a blunt control: a fixed “thinking budget” — a token allowance you set per request. It worked, but it was awkward. Set it too low and the model gave up early; too high and you paid for deliberation a simple question didn’t need.

The current generation has largely replaced that with adaptive thinking: the model itself decides how much to think based on the difficulty of the task. You no longer micromanage a token count. Instead you set a higher-level effort level — typically something like low, medium, high, or maximum — that tells the model how hard to work against how much you’re willing to spend.

• Low effort suits short, scoped tasks and latency-sensitive apps.
• High effort is the sweet spot for most coding and agentic work.
• Maximum is for the rare problem where correctness matters more than cost.

In agent workflows, this thinking also happens between tool calls — the model reasons about a result before deciding its next action, which is a big part of why agents have gotten so much more reliable.

Why you usually can’t see the raw reasoning

Most providers don’t return the unfiltered chain of thought. You’ll typically get either a readable summary of the reasoning or nothing at all, even though the model did the work (and you’re billed for it). There are a few reasons: the raw reasoning can be messy or misleading, exposing it raises safety and competitive concerns, and a summary is more useful to a human anyway. If your product needs to show users that the model is working, request the summarized view rather than expecting a verbatim transcript.

The tradeoffs

Reasoning isn’t free, and it isn’t always the right tool:

• Latency. A model that thinks takes longer to first token. On the hardest tasks, a single request can run for minutes. Plan your UX around that — stream output, show progress, let long jobs run asynchronously.
• Cost. Those hidden thinking tokens are real output tokens you pay for. Techniques like prompt caching help offset the input side, and lower effort levels keep spend in check.
• Overthinking. At high effort, a model can deliberate past the point of usefulness on a simple task. Match the effort to the problem.

When reasoning earns its keep

Reach for a reasoning model (or a high effort level) when the task has real depth: multi-step math, debugging, planning, complex code changes, or anything where a wrong answer is expensive. For lookups, formatting, classification, and chat, a fast non-reasoning pass is cheaper and feels snappier — and you can always escalate the hard cases.

The takeaway

Reasoning models are the reason AI got dramatically better at genuinely hard problems without simply ballooning in size. The mental model is simple: thinking is a dial, not a default. Spend it where depth pays off and save it where it doesn’t. The most capable models, like Claude Fable 5, lean hardest into this — which is exactly why they shine on long, agentic work and overkill for a quick question.