The couple bridged the language gap with an AI powered lip dubbing app

Mondo Technology Updated on 2024-02-25

In June, Leroy Romero made a four-hour drive from his hometown of Peoria, Arizona. He travels to the Apple Bee restaurant in Mexicali, Mexico, and meets Brenda Ochoa for the first time.

It was a blind date that was arranged by a mutual friend who sensed that the two might be paired. Romero, a mortgage originator, knew as soon as he entered that Ochoa, the 29-year-old casino dealer, was charismatic and much taller than him. He also foresaw a small problem: Romero did not speak Spanish, which was Ochoa's native language. And she doesn't speak English.

The two drank margaritas, and Romero suggested that they use translation apps like Google Translate and Deepl Translate to get to know each other. As their interlocutors, these apps inadvertently reduce small talk. The couple talked about their inspirations and their vision for themselves in the next one, three and five years. Romero, 45, said, "I think it's good to have no useless small talk. ”

But later that month, Romero realized that there might be an easier way to communicate. At work, Romero uses an app called Captions to make** and answer questions for clients. The app uses artificial intelligence to help users edit and record**, as well as write scripts and add subtitles. He recalls a friend urging him to test the app on Ochoa, saying, "Dude, you have a translation button." Translate it and send her a **.

Like Heygen, Verbalate, and Zeeebra (similar software launched last year, attracting millions in venture capital investment), Captions allows users to not only add subtitles in **, but also translate, lip-dub, and add subtitles, enabling users to sync audio and lip movements to target languages such as Spanish, Hindi, Italian, or Japanese. This lip dubbing feature is also a standalone app that mimics a person's natural lip movements and echoes his or her pitch and tone. This means that Mr. Romero's ** can look and sound like him – just in Spanish.

4:02 a.m. Whoosh. Message sending.

Technology has long been a medium of communication between people and their lovers and potential suitors. Sliding into DM, swiping right, and exchanging endless messages throughout the day has become the norm. While some people are wary of letting AI into their love lives, others are starting to embrace it, drafting love letters with chatbots and alleviating the tedium of online dating. Some experts say stories like Romero and Ochoa could become more common.

Related Pages