Microsoft is making Copilot a core part of Windows 11, rolling out generative AI features like voice control, screen analysis, and local automation.
Users can now activate Copilot on Windows PCs by saying "Hey Copilot." The feature is optional and displays a microphone icon and audio signal when active. Sessions end automatically or with the "Goodbye" command.
The company aims to make Copilot the main way people interact with their PCs, much like how the mouse and keyboard once changed user habits, and claims voice users engage with Copilot twice as much as text users.
Moreover, "Copilot Vision" is now available globally on all devices that support Copilot. This feature analyzes what's on the screen and can offer context-sensitive help during photo editing, gaming, or Office work. Microsoft says Copilot Vision can process and analyze entire documents in Word, Excel, and PowerPoint.
Copilot is also being added to the Windows taskbar with a new button for voice and vision tools, and users can access Windows settings by voice command. A planned text-only option will let users interact without speaking.
Autonomous agents, with limits
"Copilot Actions" is a feature in testing that lets Copilot handle simple tasks on the local computer, like searching PDFs or sorting photos. The feature will first roll out as a preview in Copilot Labs for Insider Program users. Users can watch Copilot work or step in at any time.
These agent features are still limited and can make mistakes. They can not yet reliably control complex software. Studies show agent-based AI systems remain inaccurate and can introduce security risks. Microsoft says user testing should help improve reliability.
There's also integration with Manus, a startup originally from China. The Manus agent, which reportedly uses Anthropic models, is built into Windows Explorer and lets users create websites from local files with a right-click. Manus is also available as a standalone app, arriving in Windows before similar features from OpenAI. It uses the "Model Context Protocol," which is designed by Anthropic to enable system-wide access to local content.
Connected services and gaming
Copilot Connectors allow users to pull data from linked services like OneDrive, Outlook, Gmail, or Google Drive. You can search for appointments, contacts, or documents with prompts like "Find my dentist appointment details." Results can go straight into Word, Excel, or PowerPoint.
In gaming, Copilot is now available on the ROG Xbox Ally, a Windows-based handheld from ASUS. "Gaming Copilot" can be activated with a button and responds to questions or commands without pausing the game. A demo shows it working in "Minecraft," helping with navigation or explaining game mechanics.
Most features work on any Windows 11 device that supports Copilot. Microsoft previously marketed "AI PCs" alongside Copilot, but now simply recommends Copilot+ PCs for faster processing and local AI tasks. Only a "Click to Do" Zoom integration specifically requires a Copilot+ PC and participation in the Windows Insider program.
