What Is an NPU? The AI Chip Inside Your Next Laptop

An NPU — neural processing unit — is a processor designed for exactly one kind of math: the massive grids of multiply-and-add operations that neural networks are made of. It does that one thing extremely fast and at very low power, and almost nothing else. Every current laptop platform now ships one — Qualcomm’s Hexagon, Intel’s AI Boost, AMD’s XDNA, Apple’s Neural Engine — because the industry decided that running AI models on the device, without melting the battery, was worth dedicating silicon to.

CPU, GPU, NPU: three answers to different questions

The easiest way to understand an NPU is by what it gives up.

A CPU is a few very fast, very flexible cores — brilliant at branching logic, terrible value for repeating the same arithmetic a billion times. A GPU is thousands of simpler cores — built for exactly that repetition, which is why AI training lives on GPUs, but power-hungry because it stays flexible enough for graphics and general parallel work. An NPU strips the flexibility out: fixed datapaths tuned for neural-network inference, low-precision arithmetic, and aggressive power gating.

	CPU	GPU	NPU
Cores	Few, complex	Thousands, simpler	Fixed matrix engines
Best at	Logic, branching, everything	Parallel math, training, graphics	Inference only
Power draw	Moderate	High	Very low
Flexibility	Total	High	Narrow

The narrowness is the feature. A model that would spin a laptop’s fans on the GPU can run on the NPU continuously — live captioning, background blur, transcription — while the machine stays cool and the battery survives the afternoon.

What do TOPS numbers mean?

NPUs are marketed in TOPS — trillions of operations per second. Today’s laptop chips cluster in a narrow band: Qualcomm’s first Snapdragon X Elite shipped 45 TOPS, Intel’s Lunar Lake about 48, AMD’s Ryzen AI 300 series 50, Apple’s M4 delivers 38, and Qualcomm’s Snapdragon X2 generation doubled its own number to 80. Microsoft’s Copilot+ PC program is why the band exists: it set 40 TOPS as the entry requirement for its on-device AI features, and every vendor promptly built to the test.

Two caveats make TOPS a starting point rather than a verdict. First, precision: TOPS figures are usually quoted at INT8 — 8-bit integer math — and a number quoted at a lower precision looks double the size for the same silicon, so compare like for like. Second, operations aren’t experiences: whether a model actually runs well depends on memory bandwidth, software support for the NPU, and how far the model was quantized to fit. A chip with modest TOPS and great software regularly beats the opposite combination.

What actually runs on it

The honest answer in 2026: useful small things, constantly — not frontier chatbots.

• Media pipelines — background blur, eye contact, noise suppression, upscaling — running all day at single-digit watts.
• Live language — transcription, captions, and translation happening locally, which is as much a privacy feature as a performance one.
• Small language models — summarization, rewriting, autocomplete, and search over your own files, where a few billion well-tuned parameters are plenty.
• Semantic search — computing embeddings over your documents and photos in the background, so search understands meaning rather than filenames.

The big models still live elsewhere. A 70-billion-parameter LLM wants more memory and bandwidth than laptop NPUs offer; running one locally today usually means the GPU and a lot of RAM, and the truly large ones stay in the cloud. The realistic split: latency-sensitive, private, always-on work on the NPU; heavyweight generation elsewhere.

Why this suddenly matters beyond laptops

The same trade — accept narrowness, win efficiency — is now being tested at data center scale. Qualcomm’s AI200 rack accelerators scale up the very Hexagon NPU line that ships in its laptops, betting that per-watt inference discipline learned on batteries translates to server economics. Whatever happens to that bet, it explains the direction of travel: inference is becoming its own hardware category, distinct from the GPUs that train the models — the same specialization story that produced the transformer-optimized accelerators in the first place.

The takeaway

An NPU is a specialist: fixed-function matrix hardware that runs neural networks fast at power levels a battery can tolerate, at the cost of doing nothing else. TOPS numbers tell you the class of chip — with 40 TOPS as the Copilot+ floor and current laptops between 38 and 80 — but precision, memory, and software support decide the experience. For always-on, private, small-model AI, the NPU is now the default home; for training and frontier-scale models, the GPU keeps the crown.