Social Media

Meta’s New “Lip-Sync” AI Just Broke the Language Barrier Again

Published by

Meta has expanded its AI-powered translation capabilities for Reels, adding a broader set of languages to make short-form video content more accessible and easier to engage with for global users.

As they explained in a Facebook blog:

Imagine discovering a reel that makes you laugh, inspires you, or teaches you something new, no matter where it was created or what language it was filmed in. That’s the future we’re building at Meta.

AI-powered translations for Reels are starting to roll out in more languages, including Bengali, Tamil, Telugu, Marathi, and Kannada, on Instagram. These new additions build on our existing language support for English, Hindi, Portuguese, and Spanish.

Meta kicked off its AI translation feature for Reels back in August of last year, allowing users to translate clips in English and Spanish. What’s really cool about this is that the inclusion of additional languages will allow creators to reach audiences beyond their native speech communities, creating deeper engagement and stronger cross-cultural connections on the platform.

The updated translation feature uses on-device and cloud-based AI models to generate real­-time subtitles and text translations for Reels, helping viewers understand content in languages they do not speak.

According to the company, AI translations are automatically suggested when a viewer’s device language differs from the original audio or text language of a Reel. The system leverages neural machine translation models to produce captions, which users can toggle on or off, enhancing accessibility for people who are deaf or hard of hearing as well as for those viewing content in noisy environments or without audio.

We are seeing more and more short-form video platforms relying on AI to break down language barriers, aligning with broader industry trends. Services such as TikTok, YouTube Shorts, and Snapchat Spotlight have similarly integrated AI-assisted translation and subtitling to help creators expand their reach across borders, although their AI integrations are of varying levels of proficiency.

Meta’s AI Translations capture the unique sound and tone of each creator’s voice, perfectly syncing with their lip movements to deliver a more genuine representation of the original clip. Instagram’s head honcho, Adam Mosseri, mentions that creators are experiencing a boost in their reach thanks to translated content, with more Reels from across the globe popping up in people’s feeds.

Translating reels with Meta AI is free, and is accessible to Facebook creators with 1,000 followers or more and to all public Instagram accounts in countries where Meta AI is available.

 

Abdul Wasay

Abdul Wasay explores emerging trends across AI, cybersecurity, startups and social media platforms in a way anyone can easily follow.