An NPU (neural processing unit) is a specialised processor, or a block inside a chip, designed to run the matrix and tensor operations at the heart of neural networks quickly and with low power use. Modern phones, laptops, and edge devices include one.
Why it matters
AI tasks such as background blur on video calls, speech recognition, photo enhancement, and small language models run on-device through the NPU, which leaves the CPU and GPU free and saves battery. NPUs usually work with low-precision numbers like 8-bit integers, which are enough for inference and far cheaper in energy than 32-bit floating point.
Performance is quoted in TOPS (trillions of operations per second). The figure is a peak value that depends on the numeric precision used, so TOPS from different vendors are not always comparable. Software support is important: an application must use the vendor's runtime to benefit.
Example
Some PC makers require a minimum NPU performance, for example 40 TOPS for certain on-device AI features, though requirements change by vendor and over time. A video app running background blur on the NPU may use a fraction of the power it would on the CPU.
Common mistakes
- Treating TOPS as the only measure of AI performance.
- Assuming every AI program automatically uses the NPU.
- Expecting an NPU to train large models; it is designed for inference.
Related
Get notified when this term updates
Plus occasional practical tips on uptime and monitoring, nothing else. Unsubscribe anytime.
This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.