Meta Announces Automatic AI Dubbing for Reels
Meta announced a bunch of new improvements to their AI products, and the big one is automatic translations for reels with AI. Auto dubbing over the video AND lip syncing. Cover image from Meta Newsroom.
These updates were all announced today during Meta Connect 2024. The event is on September 25th and 26th and is sure to bring news soon.
This is a massive new feature that will impact Instagram quite dramatically. Here’s the rundown.
– Automatic Dubbing and Lip Syncing to English and Spanish
– Currently testing on Instagram and Facebook with creators’ videos from Latin America
– Meta says “and we plan to expand this to more creators and languages”
In the announcement, Meta shared:
Testing Meta AI Translations for Creator Content
We’re testing a Meta AI translation tool that will automatically translate the audio of Reels, so more people can enjoy your content, even if you speak different languages. With automatic dubbing and lip syncing, Meta AI will simulate the speaker’s voice in another language and sync their lips to match. We’re starting with small tests on Instagram and Facebook, translating some creators’ videos from Latin America and the US in English and Spanish, and we plan to expand this to more creators and languages.
Meta announces automatic AI dubbing for Reels pic.twitter.com/MvoKsOzldW
— Jules Terpak (@julesterpak) September 25, 2024
Meta Also Plans to Expand Their AI Tools for Businesses
In the same announcement, Meta shared:
We’re expanding our business AIs to thousands of businesses using click-to-message ads on WhatsApp and Messenger in English, so they can quickly set up business AIs that can talk to their customers, offer support and facilitate commerce. From answering common customer questions to discussing products and finalizing a purchase, these business AIs can help businesses engage with more customers and increase sales.
Image: Meta Newsroom
My Thoughts About This
It will be interesting to see how Meta builds these tools out over the coming years. I can imagine that dubbing and lip syncing could have a lot of other use cases. This could also dramatically increase audiences for creators, so I am very excited about it. By making it easier to automatically add translations, it will really help creators reach new audiences.
Translating videos is something large creators like MrBeast talk about, but for smaller creators it’s hard to do because it may be costly or very time consuming. With advances like this from AI, I could imagine big shifts in the social media landscape. Especially when it rolls out to more languages.
