On December 18, 2024, Microsoft announced it was piloting real-time translation in its live captions feature for AMD and Intel-powered Copilot+ PCs. According to a Windows blog post, the roll-out will gradually include the ability to translate live captions from more than 44 source languages into English, for use cases such as “real-time video calls, recordings, and streamed content.”
The release also delivers an improvement for real-time translation on Snapdragon-powered Copilot+ PCs, where users will be able to translate, in real time, from 26 languages to Simplified Chinese.
Despite the name “live captions,” the new ability to perform real-time translation effectively turns the feature into a subtitling tool: Live captions will “instantly” translate live or pre-recorded video “in any app or video platform.” Users can choose to display the captions “above screen,” “below screen,” or “floating on screen,” in an adjustable overlay window.
The use of the term “captions” seems to reflect Microsoft’s branding of the feature under the umbrella of accessibility, a concept often associated with deaf and hard of hearing users versus users accessing content in a language they do not understand. Microsoft has broadened the use of the term in a Microsoft 365 Support page, where the company encourages users to “use live captions to better understand audio.”
“To make more content accessible to more people, live captions now has the ability to provide translations and will turn any audio that passes through your PC into a single English-language caption experience,” the page explains.
Slator Translation as a Feature (TaaF) Report
The Slator Translation as a Feature (TaaF) Report is a vital and concise guide on how AI translation is becoming an integral feature in enterprise technology.
Live captions asks users for consent to process voice data on the device, and then prompts them to download language files, from one or more of the 44 supported source languages, for on-device speech recognition. Users can add more languages by downloading the corresponding files later on, and can use the mic on their PC to caption their own speech.
As one of the world’s largest technology companies, Microsoft publicized several of its developments in language tech over the course of 2024, from updating an automatic dubbing patent file to revealing a multilingual AI translation model and previewing an AI speech translation app.
Nonetheless, Microsoft gave users a heads up on “known issues to be fixed in future updates to Windows Insiders” (and suggested fixes), noting that the first launch, and switching languages while audio is playing or the microphone is enabled, may cause live captions to crash.
