ElevenLabs brings video dubbing and lip sync together in 25 languages

ElevenLabs has added Dub & Sync to ElevenCreative, a video tool that translates spoken dialogue and adjusts the speaker’s mouth movements to match the new audio. In its November 12 launch video, the company said the tool dubs videos into 25 languages. Creators supply a source video and select target languages. The tool returns versions with dubbed speech and matching lip movements in the same run.

The change is in the handoff between audio and video. A creator adapting a presenter-led product clip for another language can get a revised video directly from the dubbing task, with the picture adjusted to the new delivery. ElevenLabs says the combined workflow handles both jobs without asking the user to write a generation prompt or edit the lip movements by hand.

A dubbed voice and a changed picture

ElevenLabs had already described a form of timing alignment in Dubbing v2. Its translation system accounts for when someone starts and stops speaking, helping the translated dialogue fit the same interval. Matching the overall timing of a sentence and matching the mouth shapes within it are different jobs. Two sentences can run for the same length while producing different visible syllables.

The company’s help page for its existing Dubbing product describes lip sync as a separate capability available in its video tools. Dub & Sync brings the sound and picture changes together in a dedicated ElevenCreative app. The input described in the announcement is a video, which gives the tool a picture to adjust. ElevenLabs continues to document audio-only dubbing as part of its general dubbing service.

The combined tool covers 25 languages

The 25-language figure comes from the description of ElevenLabs’ own Dub & Sync launch video. It is specific to the new workflow. The company’s documentation for existing Dubbing lists more than 90 languages, including Korean, Spanish, French and Japanese, and describes regional variants for some of them. The launch materials give the combined tool’s count without identifying all 25 languages or saying which regional variants it supports.

ElevenCreative shows users selecting target languages and receiving a localized version for each choice. The company gives short creator clips, product videos and course material as examples of footage that can be adapted this way. For a team distributing a filmed lesson in several markets, the intended output is a set of videos with different speech and matching mouth movements. The announcement gives no details about automatic subtitle generation or direct publishing to social channels.

ElevenLabs also has a Dubbing v2 API that can produce translated audio and, on Enterprise plans, regenerate selected segments after edits to the transcript or translation. The API release describes aligning the start and end of speech with the original timing. The new announcement covers the web app. It gives no details on whether Dub & Sync’s picture changes can be requested through the existing API or whether an editor can supply a custom translation to the new tool.

The app and its published limits

ElevenLabs has a dedicated Dub & Sync address in ElevenCreative. The public launch materials describe the video and language inputs, but give no maximum clip length, file size, plan requirements or credit rates. The company has separate billing guidance for general Dubbing, which charges by the length of the source and the number of target languages. It has yet to say whether the video app uses that same calculation.

The older Dubbing Studio has a project editor for changing translations, reassigning speakers and regenerating particular clips. The documentation ties that editor to projects made with the v1 model. It can be used when one line, such as a product name, needs to be revised. The new announcement describes the combined generation step and leaves clip-by-clip editing unspecified. ElevenLabs’ public link goes to a dedicated app inside ElevenCreative.