NPUs: The New Frontier in PC Computing

For years, the CPU and GPU have reigned supreme in the world of personal computing. However, a new player has entered the arena: the NPU, or neural processing unit. This specialized processor is designed to accelerate AI tasks with remarkable power efficiency, potentially revolutionizing the way we interact with our computers and opening the door to a new generation of AI-powered applications.

While NPUs are poised to become a standard component in all PCs, they are currently found in a select few devices. Understanding what NPUs are and why they are generating so much buzz is crucial for anyone interested in the future of computing.

What is an NPU?

NPU stands for neural processing unit. It is a dedicated processor optimized for artificial intelligence (AI) and machine learning workloads. The name "neural" comes from the fact that AI models heavily rely on neural networks, which are complex structures of interconnected nodes that mimic the way the human brain processes information.

Unlike GPUs, which can be purchased as separate add-in cards, NPUs are typically integrated as part of a modern processor platform. Examples of such platforms include Intel’s Core Ultra, AMD’s Ryzen AI, and Qualcomm’s Snapdragon X Elite and Snapdragon X Plus. These platforms combine a CPU, an integrated GPU, and an NPU, offering a comprehensive processing solution.

The NPU Advantage: Power Efficiency and Performance

Traditionally, computers have relied on CPUs and GPUs to handle the majority of processing tasks, including AI-related operations. While GPUs excel at AI tasks due to their parallel processing capabilities, they are known for their high power consumption. This is where NPUs come in.

NPUs offer a unique balance between performance and power efficiency. They are faster than CPUs at handling AI workloads while consuming significantly less power than GPUs. This makes them ideal for laptops and other portable devices where battery life is a primary concern.

Moreover, by offloading AI tasks to the NPU, the CPU and GPU are freed up to handle other tasks, resulting in improved overall system performance. This synergistic approach ensures that each processor is utilized optimally, leading to a smoother and more responsive user experience.

NPUs in Action: Use Cases and Applications

NPUs are poised to revolutionize a wide range of applications, from AI-powered image generation to enhanced video streaming.

For example, if you are using AI image generation software like Stable Diffusion, a GPU would be the preferred choice for maximum performance. However, for less demanding AI tasks or when battery life is a priority, an NPU offers a compelling alternative.

NPUs also enable laptops to perform local AI tasks without generating excessive heat or consuming significant battery power. This allows users to take advantage of AI features without compromising portability or performance.

Beyond AI, NPUs can be used to accelerate other types of tasks. At CES 2024, HP demonstrated game-streaming software that utilizes the NPU for video-streaming, freeing up the GPU to focus on running the game. This approach significantly improves streaming performance and reduces the load on the GPU.

NPU Integration: Operating Systems and Applications

The true potential of NPUs lies in their seamless integration with operating systems and applications. With an AI PC, the NPU can be utilized by both the operating system and the applications running on the system.

For instance, laptops with Intel Meteor Lake processors can leverage the built-in NPU to run Windows Studio Effects, which are AI-powered webcam effects that provide features like background blur and forced eye contact.

Microsoft’s Copilot+ PCs, powered by Qualcomm Snapdragon X chips, have their own AI-powered features that utilize the NPUs built into the platform. For example, the Windows Recall feature requires an NPU to function.

In the future, app developers will be able to leverage the NPU in a variety of ways. Expect to see AI-powered audio and photo editing plugins for applications like Audacity and GIMP that harness the processing power of the NPU.

The Cloud vs. Local Processing: NPUs and AI

Currently, many applications with AI features rely on cloud servers to run their AI models. This allows users to access AI functionality on a wide range of devices, regardless of their processing capabilities. However, running AI models in the cloud is expensive for companies.

NPUs offer an alternative solution by enabling local processing of AI tasks. This not only reduces the cost for companies but also offers several benefits to users, including the ability to run AI features offline and keep their data private.

NPU Performance: TOPS and the Future

NPU performance is measured in TOPS (trillion operations per second). A low-end NPU might only handle 10 TOPS, while PCs that qualify for Microsoft’s Copilot+ PC branding must handle at least 40 TOPS.

While NPUs are primarily found in PCs, they are also present in other devices, such as Apple’s Neural Engine, Google’s Tensor platform for Pixel devices, and Samsung Galaxy phones.

The Current State of NPUs: Challenges and Opportunities

NPUs are still in their early stages of development, and there are challenges to overcome before their full potential can be realized.

For example, Microsoft initially announced that Meteor Lake NPUs were too slow for Copilot+ PC features, although Intel disagreed. This highlights the evolving nature of NPU technology and the need for continuous improvement.

Despite these challenges, NPUs represent a significant step forward in the evolution of computing. As NPUs become more powerful and more widely adopted, they will unlock new possibilities for AI-powered applications and transform the way we interact with our computers.

Making the NPU Decision: Is It Worth It?

If you are buying a new laptop, it is worth considering a model with an NPU. Modern processor platforms typically include NPUs, along with other features such as improved battery life.

However, if you find a heavily discounted previous-generation laptop, it may not be worth spending significantly more just for an NPU, especially if you don’t have specific AI tools in mind.

For desktop PCs, the NPU situation is different. Intel’s desktop CPUs do not currently have NPUs. While you can find AMD desktop chips with Ryzen AI NPUs, they may not support Copilot+ PC features. Therefore, if you are building a desktop PC, you can set aside the NPU for now.

NPU Recommendations: Where to Look

If you are looking for a PC with a high-performance NPU, consider Arm-based Windows laptops such as the Surface Laptop 7 and AMD Ryzen AI 300 series laptops like the Asus ProArt PX13. Also, keep an eye out for reviews of Lunar Lake-powered Intel laptops when they become available.

NPUs are the future of computing. Embracing this technology and understanding its capabilities will be crucial for navigating the evolving landscape of AI and personal computing.