ElevenLabs adds speaker labels to streaming transcripts with Scribe v3 Live

ElevenLabs released two speech-to-text models on November 10. Scribe v3 handles recorded audio, and Scribe v3 Live handles streams. The Live model is the first realtime Scribe to label speakers. Meeting and captioning apps can opt in to receive speaker labels while a conversation is being transcribed.

Both models are available through ElevenAPI, and Live can also be selected in ElevenAgents. Scribe v3 is the default for file transcription on the company's standard website. ElevenLabs' help center lists the previous Scribe v2 Realtime as the default in ElevenAgents. Existing API integrations can choose the new models through their existing transcription endpoints.

Speaker labels can change as the call continues

Diarization, the process of assigning speech to a speaker, is optional in the realtime API. When enabled, a partial transcript can attach temporary labels to words before the segment ends. The API sends the finalized text first and follows it with a separate result containing the speaker assignments. An app can show the finalized words before those assignments arrive.

The labels can still change. If the model later determines that two labels refer to the same person, it tells the app to merge them, including on earlier segments. A word can remain unassigned if the model cannot identify the speaker. ElevenLabs' developer guide also says realtime diarization cannot be combined with its background-speech filter.

Using context to judge when someone has finished

Scribe v3 Live adds an optional way to decide when to finalize a transcript segment using the conversation's context. A caller who pauses halfway through a phone number, for example, can trigger an early handoff if an app treats a short silence as the end of a turn. With turn prediction enabled, the service commits a segment when the model's estimate crosses a threshold. A configured silence limit still applies. By default, the realtime API uses manual commits initiated by the application.

Live also adds a mode that follows language changes within a sentence and transcribes each language in its own script. ElevenLabs says short phrases can be less accurate in this mode and background speech in another language can be picked up. For recorded audio, Scribe v3 adds options to list expected secondary languages and select whether numbers appear as digits or words. The batch documentation lists Korean among the supported languages and allows up to 1,000 terms to guide recognition of names and jargon.

How the error and latency figures were measured

ElevenLabs' model documentation reports a 27% reduction in word error rate from v2 for recorded audio on FLEURS, a set of short read-speech clips in 102 languages. Scribe v3 Live reduced conventional word error rate by 31% and semantic word error rate by 43% from v2 Realtime. The semantic measure focuses on mistakes that change the meaning. The company scored languages separately before averaging them.

On the English portion of ElevenLabs' separate 16-language dataset of real-world audio, the Live model recorded a speaker assignment error rate of 3.1%. The company aligned each transcript with the reference before measuring whether words were assigned to the right speaker. The figure covers speaker assignments under that scoring method.

Cekura, which tests voice applications, published a separate evaluation using 1,000 public clips and eight conversation recordings. It measured word error rates of 1.81% and 2.05% on those datasets, respectively. The lab recorded timings for 205 of 206 streamed turns, with one failed request. The median wait for first text was 685 milliseconds from the start of streaming. The median wait for finalized text was 57 milliseconds after the speaker stopped.

ElevenLabs lists base API rates of $0.20 per audio hour for Scribe v3 and $0.22 for Live, matching the rates it lists for their respective predecessors. Live adds $0.08 an hour for a domain vocabulary library or $0.12 for transcript editing when those options are used. ElevenLabs says both models run in the US by default, with isolated environments in the EU, India and Singapore for enterprise customers.