Microsoft’s AI-Powered Future for Windows 11: A Deeper Dive

Microsoft is betting big on artificial intelligence, promising a future where computers respond to natural language commands, reminiscent of the seamless interactions envisioned in science fiction classics like Star Trek. This vision is starting to materialize, albeit in a limited scope, within the Windows 11 operating system.

The core concept involves using natural language to manipulate settings within Windows 11. Imagine simply telling your computer, "Increase the screen brightness," and the system automatically adjusts the display accordingly. This functionality is driven by "agents," which are small AI programs designed to execute tasks on the user’s behalf. However, this feature will initially be exclusive to Copilot+ PCs, which are equipped with a dedicated AI-accelerating Neural Processing Unit (NPU).

Microsoft is leveraging its Surface devices as the primary platform to showcase its latest software innovations. The new 13-inch Surface Laptop and 12-inch Surface Pro will be the initial launch vehicles for features like Windows Recall, semantic search, and Click-to-Do. Moreover, these devices will also debut groundbreaking functionalities, including agentic AI for managing settings, relighting features within the Photos app, as well as object editing and sticker generation capabilities within Paint.

The most notable addition to Microsoft’s AI arsenal is an AI agent designed to manage individual settings within Windows. This is a concept initially floated for Copilot, where Microsoft envisioned it helping with tasks like turning on dark mode. However, the technology remained rudimentary and eventually morphed into a feature providing how-to guides rather than executing tasks.

The agentic search capabilities Microsoft is now promising more closely align with user expectations for AI integration within Windows. Users will be able to use natural language to instruct Windows 11’s Settings menu. The system will either guide the user through the process or, with their permission, automatically perform the action. The potential applications are extensive, ranging from managing display settings to configuring network preferences.

Microsoft is promising that the agentic AI will be able to manage "thousands" of individual settings. Navjot Virk, Corporate Vice President of Windows Experiences, emphasizes that the goal is to create technology that works alongside the user, enabling greater productivity, creativity, and connection.

Beyond managing system settings, Microsoft is enhancing its creative applications with AI. The Relight AI feature in Photos allows users to modify the lighting of their images. This feature addresses the common challenge of poorly lit photographs, allowing users to simulate multiple AI-generated lighting sources and adjust their brightness and color to enhance the image’s overall appearance.

Microsoft is also adding "Object Select" to Paint. This feature allows users to add, erase, and fill specific objects in the scene, instead of trying to manually edit the whole scene.

The Snipping Tool is also receiving an AI-powered upgrade. The updated tool will be able to "interpret the intent of what you’re trying to capture," streamlining the process of grabbing the perfect screenshot.

Another new feature for Paint is the addition of a sticker generator. Similar to the memoji found in iOS, this allows users to create customized stickers that can be used in various chats and applications, including Teams.

Microsoft is also committed to upgrading the Start menu to include the Phone Link sidebar, which provides a dashboard for the user’s phone, displaying information such as unread messages, missed calls, and quick access to photos taken with the phone.

Future updates to the Microsoft Store, "AI actions within File Explorer," and the Microsoft Edge Game Assist are planned. The Game Assist feature is designed to provide helpful guides and assistance while gaming.

However, it’s crucial to approach these announcements with a degree of realism. Microsoft typically adopts a cautious approach to rolling out new features, often starting with limited trials on Copilot+ PCs running on Qualcomm’s Arm processor. This represents a small fraction of the overall PC market. The progression of features like Recall, improved search, and Click-to-Do has been gradual, moving from initial announcements to trials on specific hardware configurations before potential broader availability.

It is also important to note that Copilot Vision, which was announced a month ago, is not currently working in a useable form. Any new AI features will probably move forward at a similar pace.

Smaller features, such as Relight in Photos, typically see faster deployment. While Microsoft hasn’t announced specific timelines for the new features, it’s reasonable to expect that the smaller enhancements to Paint and Photos will debut before the more substantial updates to Settings.