Skip to content
AI Primer
release

ElevenLabs releases Scribe v3 and Scribe v3 Live speech models

ElevenLabs released Scribe v3 and Scribe v3 Live with reported reductions in word errors. The Live model adds speaker diarization and conversation-aware turn detection; both use existing API endpoints.

4 min read
ElevenLabs releases Scribe v3 and Scribe v3 Live speech models
ElevenLabs releases Scribe v3 and Scribe v3 Live speech models

TL;DR

Eneco's voice agent recognized Dutch postcodes 40% more often on the first try, and house numbers 15% more often. ElevenLabs also shipped a medical-conversation variant of Live inside ElevenAgents.

Accuracy and diarization

ElevenLabs reports four main results in its evaluation:

  • Batch word error rate: 27% lower than Scribe v2 across 102 languages.
  • Batch diarization errors: 12% lower overall than v2, rising to a 17% reduction on recordings with three or more speakers.
  • Live transcription: 31% lower word error rate than v2 Realtime. The blog reports a 43% reduction in semantic WER; ElevenLabsDevs' announcement gives 44% for that metric.
  • Live diarization: 3.1% error rate on the English portion of ElevenLabs' real-world dataset, versus 5.9% to 7.3% for other realtime models tested.

Semantic WER counts only mistakes that change meaning. Multilingual testing uses an internal version of FLEURS, containing short, clean read speech, with each of the 102 languages weighted equally rather than pooling clips into one score.

Speaker testing uses a separate, 16-language dataset of meetings, agent calls and voice notes that ElevenLabs says it will open-source. For Live diarization, transcripts are aligned to the reference before scoring to remove transcription mistakes, so the company explicitly says those figures cannot be compared directly with its batch diarization results.

Turn prediction

Live exposes the conversation-aware turn detector already used by ElevenAgents through commit_strategy=turn_prediction.

Conversation-aware commits are the prize for voice-agent builders. The commit-strategy documentation specifies the actual behavior:

  • Opt-in: The default strategy remains manual, including on v3 Live. Changing the model ID alone leaves that default intact.
  • Probability signal: Every Live partial_transcript includes turn_probability, regardless of commit strategy. With turn prediction enabled, the server commits when that probability reaches 0.9 or higher.
  • Silence fallback: VAD remains active and can also commit when vad_silence_threshold_secs is reached.

The launch example uses SDK syntax, but the documentation says the Python and JavaScript CommitStrategy enums do not yet include turn_prediction and describes connecting directly to the WebSocket API.

Keyterm library

Live supports two vocabulary mechanisms with different limits, according to the keyterm documentation:

  • keyterms: Up to 50 expected terms, each at most 20 characters, supplied as connection query parameters.
  • keyterm_library: Up to 10,000 terms, each at most 50 characters, with a 2 MB total limit. The model consults this broader vocabulary only when it hears a likely match.
  • Transport: The library belongs in the first input_audio_chunk message. Sending it later, or exceeding its limits, returns input_error and closes the connection.
  • SDK support: Official SDKs do not yet support keyterm_library; the documented path uses the WebSocket API directly.
  • Billing: Vocabulary prompting costs extra. Combining a library with keyterms incurs the keyterm prompting surcharge once.

Language and transcript controls

The new language and output parameters split by model:

  • Recorded audio, numbers_format: Returns numbers as digits, such as 42, or words, such as forty-two.
  • Recorded audio, secondary_language_codes: Supplies hints about other languages a recording might contain.
  • Live audio, multilang_mode=true: Supports language switching mid-sentence, as described by ElevenLabsDevs.

Batch keyterm prompting supports up to 1,000 terms, with a 50-character and five-word limit per term. Scribe v3 also accepts plain-English transcript formatting rules, such as “write every date as YYYY-MM-DD.”

API access and data residency

Existing integrations retain their endpoints and change model_id:

  • Files: scribe_v3
  • Streams: scribe_v3_live

Scribe v2 and Scribe v2 Realtime remain available.

The enterprise offering includes isolated EU, India and Singapore environments; the default deployment runs in the US. Zero Retention Mode deletes audio and text as soon as a request completes, and private deployment is also available.

Further reading

Discussion across the web

Where this story is being discussed, in original context.

On X· 3 threads
TL;DR3 posts
Accuracy and diarization1 post
API access and data residency1 post
Share on X