Microsoft starts testing Copilot Vision update that can “see” your screen and apps – The Verge

🔍 Summary:

Microsoft is expanding the capabilities of its Copilot app on Windows with a new feature called Copilot Vision, which now extends beyond Microsoft’s Edge browser to encompass any application on a user’s PC. This enhancement allows Copilot Vision to assist users interactively, such as guiding them through Adobe Photoshop, analyzing images and webpages, or even optimizing settings in video editors like Microsoft’s Clipchamp. Initially showcased at Microsoft’s 50th anniversary event, this feature demonstrates how AI can guide users through tasks and settings in real-time, akin to a screen-sharing session in Microsoft Teams.

Additionally, Microsoft is testing a file search feature within Copilot that enables users to inquire about and locate contents within various file types like .docx, .xlsx, and .pdf, directly through the AI assistant. These new functionalities are currently available to U.S. testers in the Windows Insider beta program and are expected to roll out to all Windows 11 users soon. Notably, these features are accessible through the Copilot app on Windows without needing a specialized Copilot Plus PC and are also available on iOS and Android platforms.

📌 Source: https://www.theverge.com/news/645666/microsoft-copilot-vision-windows-beta-testing