ElevenLabs TTS in Node.js, with Eleven v4 Turbo
@micdrop/elevenlabs streams ElevenLabs text to speech from a Node.js server, for @micdrop/server. It sends the agent’s answer to ElevenLabs over a WebSocket as the agent writes it, so the audio comes back before the answer is finished. The package works with Eleven v4 Turbo, Eleven v4, Eleven v3, Flash v2.5, Turbo v2.5 and Multilingual v2, and picks the WebSocket each model needs.
ElevenLabs and every other voice engine in Micdrop implement the same interface, so moving to an ElevenLabs alternative only means replacing new ElevenLabsTTS(...) with the other engine’s class.
Installation
npm install @micdrop/elevenlabsElevenLabs TTS (Text-to-Speech)
Usage with MicdropServer
import { ElevenLabsTTS } from '@micdrop/elevenlabs'import { MicdropServer } from '@micdrop/server'
const tts = new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY || '', voiceId: '21m00Tcm4TlvDq8ikWAM', // ElevenLabs voice ID modelId: 'eleven_v4_turbo', // Optional: model to use language: 'en', // Optional: language code voiceSettings: { stability: 0.5, },})
// Use with MicdropServernew MicdropServer(socket, { tts, // ... other options})Usage without MicdropServer
import { ElevenLabsTTS } from '@micdrop/elevenlabs'import { Readable } from 'stream'
const tts = new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY || '', voiceId: '21m00Tcm4TlvDq8ikWAM',})
// Audio is raw PCM, 16 bits, 16 kHz, monotts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))tts.on('Failed', (texts) => console.error('Failed:', texts))
tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))Events
| Event | Payload | Description |
|---|---|---|
Audio | Buffer | A chunk of audio, PCM 16 bits, 16 kHz, mono, ready to be played. |
Failed | string[] | The text left unspoken when ElevenLabsTTS gives up after its retries. |
See the TTS interface for the full contract.
Choosing a model
ElevenLabs streams speech over two WebSockets, each with its own protocol. Eleven v3 and v4 only stream over the Text to Dialogue WebSocket, while Flash v2.5, Turbo v2.5 and Multilingual v2 use the Text to Speech one. ElevenLabsTTS opens the right one for the modelId you pass, so moving from Flash to v4 Turbo only means changing the model ID.
| Model | WebSocket | Languages | Audio tags | Best for |
|---|---|---|---|---|
eleven_v4_turbo | Text to Dialogue | 90+ | Performed | Expressive voice agents, with a median inference latency of about 100 ms |
eleven_v4 | Text to Dialogue | 90+ | Performed | Narration and characters, where the quality of the voice comes first |
eleven_v3_conversational | Text to Dialogue | 70+ | Performed | Expressive real-time speech, the generation before v4 Turbo |
eleven_v3 | Text to Dialogue | 70+ | Performed | Expressive speech, the generation before v4 |
eleven_flash_v2_5 | Text to Speech | 32 | Read out | The lowest latency, about 75 ms |
eleven_multilingual_v2 | Text to Speech | 29 | Read out | Long narration, where the voice stays the most stable |
eleven_turbo_v2_5 | Text to Speech | 32 | Read out | Existing servers, since ElevenLabs deprecated it in favor of Flash v2.5 |
The latencies are the ones ElevenLabs gives, network excluded. A Micdrop test measured the first audio in a streamed voice agent, network included: about 90 ms on v4 Turbo and about 850 ms on v3. eleven_turbo_v2_5 stays the default of the package, so an existing server keeps its voice until you pick another model.
Eleven v4 and audio tags
Eleven v3 and v4 perform audio tags, short directions in square brackets. A tag like [whispers], [laughs] or [excited] goes right before the words it changes. Eleven v4 also plays sound effects like [door slams] and follows free directions like [strong French accent]. ElevenLabs lists the tags it tested with Eleven v4.
The LLM writes the tags, so tell it in the system prompt which ones it may use:
import { OpenaiAgent } from '@micdrop/openai'
const agent = new OpenaiAgent({ apiKey: process.env.OPENAI_API_KEY || '', systemPrompt: `You are a warm and playful assistant. Your answers are spoken out loud.
Your voice performs audio tags: short directions in square brackets, placed right before the words they change. Use one or two per answer, where they fit:- How you speak: [whispers], [excited], [curious], [sarcastic]- How you react: [laughs], [sighs], [clears throat]Use an ellipsis for a pause. The words you say always stay outside the brackets.`,})The tags reach the client with the rest of the answer, in the conversation and in the partial messages. Run the storyteller demo to hear what the tags change. Its page shows each tag apart from the words, and lets you switch the voice to Flash v2.5, where the same storyteller speaks without tags.
Only Eleven v3 and v4 perform the tags. Flash v2.5, Turbo v2.5 and Multilingual v2 read them out loud, so leave the tag instructions out of the system prompt when the voice runs on one of those models. When one of those models only serves as a backup voice, keep the tag instructions and strip the audio tags from the text it receives.
How Eleven v3 and v4 stream
The Text to Dialogue WebSocket holds text back until it has about 40 characters and 8 words. ElevenLabsTTS flushes it at the end of every sentence, so a short first sentence such as “Sure!” starts playing without waiting for the next one. The package counts an ellipsis as part of the sentence, since Eleven v4 performs it as a pause.
The Text to Dialogue WebSocket has no message to stop a synthesis halfway. When the user interrupts the assistant, ElevenLabsTTS closes the connection and opens a new one while the user is still speaking, so the interrupted answer stops where the user cut it. ElevenLabs closes a connection after 20 seconds without a message, so the package sends a keep-alive every 15 seconds.
ElevenLabs counts each connection to the Text to Dialogue WebSocket as one dialogue session for as long as it stays open, speaking or silent, so each call holds one session from start to end. ElevenLabs allows 14 sessions at once on its free plan and 105 on Scale, and publishes the session limit of each plan. Past the limit, it rejects new connections with too_many_concurrent_requests. ElevenLabs counts only the time spent generating audio on the Text to Speech WebSocket, so the same plan usually allows more calls at once with Flash.
Interruptions and reconnections
Flash v2.5, Turbo v2.5 and Multilingual v2 have no message to stop a synthesis either. When the user interrupts the assistant, ElevenLabsTTS drops their connection too and opens a new one while the user is still speaking, so no audio of the interrupted answer plays after the cut.
When a connection drops in the middle of an answer, ElevenLabsTTS reconnects and sends the answer again. Flash v2.5, Turbo v2.5 and Multilingual v2 pick it up where its audio stopped, while Eleven v3 and v4 speak it again from its start. ElevenLabs accepts a connection before it checks the key and the voice, so a connection that closes before carrying any audio counts as a failed attempt. After more than maxRetry failed attempts in a row, ElevenLabsTTS emits Failed with the text left unspoken, and the next call to speak() connects again with a full retry budget.
Options
| Option | Type | Default | Description |
|---|---|---|---|
apiKey | string | Required | Your ElevenLabs API key |
voiceId | string | Required | ElevenLabs voice ID |
modelId | ElevenLabsModelId | 'eleven_turbo_v2_5' | Model to use for speech synthesis, one of the supported models |
language | string | Optional | Language code (e.g., ‘en’, ‘fr’) |
outputFormat | TextToSpeechStreamRequestOutputFormat | 'pcm_16000' | Audio output format |
voiceSettings | VoiceSettings | Optional | Voice customization settings. Eleven v3 and v4 only read stability |
connectionTimeout | number | 5000 | Timeout in milliseconds for WebSocket connection |
retryDelay | number | 1000 | Delay in milliseconds between reconnection attempts |
maxRetry | number | 3 | Failed connection attempts in a row allowed before emitting Failed |
Voice Settings
The voiceSettings option takes the VoiceSettings type of the ElevenLabs SDK, with its keys in camel case:
const tts = new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY || '', voiceId: 'your-voice-id', modelId: 'eleven_flash_v2_5', voiceSettings: { stability: 0.5, // 0 to 1, lower is more expressive, higher is steadier similarityBoost: 0.75, // 0 to 1, how closely to match the original voice style: 0.5, // 0 to 1, exaggeration of the style useSpeakerBoost: true, // Boost the similarity to the speaker speed: 1, // 1 is the normal pace },})Eleven v3 and v4 only read stability. A lower value gives the voice a broader emotional range, so it responds more to the audio tags.
Supported Languages
The languages depend on the model: more than 90 for Eleven v4 and v4 Turbo, more than 70 for Eleven v3, 32 for Flash v2.5 and Turbo v2.5, and 29 for Multilingual v2. ElevenLabs keeps the list of languages of each model. Every model speaks these languages:
| Code | Language | Code | Language | Code | Language |
|---|---|---|---|---|---|
en | English | es | Spanish | fr | French |
de | German | it | Italian | pt | Portuguese |
pl | Polish | tr | Turkish | ru | Russian |
nl | Dutch | cs | Czech | ar | Arabic |
zh | Chinese | ja | Japanese | sv | Swedish |
ko | Korean | hi | Hindi | fi | Finnish |
Getting Started
- Sign up for an ElevenLabs account and create an API key
- Choose a voice from the ElevenLabs voice library or create a custom voice
- Install the package and configure with your credentials
import { ElevenLabsTTS } from '@micdrop/elevenlabs'
const tts = new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY || '', voiceId: 'your-voice-id', // Get this from ElevenLabs dashboard modelId: 'eleven_v4_turbo', // Choose based on your needs language: 'en',})
// Use with MicdropServernew MicdropServer(socket, { tts, // ... other options})Creating an ElevenLabs API key
You create API keys on the API keys page of your ElevenLabs settings. Name each key after the server that uses it, so you can revoke one server’s key and leave the other servers running.
Restrict the key to Text to Speech access. @micdrop/elevenlabs only opens the Text to Speech and Text to Dialogue WebSockets, so that one permission covers everything the package does. A leaked key can then only generate speech. Add Voices read access if your own code lists the voices of your account. You can also give each key a monthly credit limit and an IP allowlist, so the key stops once it has spent its credits for the month and works only from your servers’ addresses.
Set the key in the ELEVENLABS_API_KEY environment variable of your server, then pass process.env.ELEVENLABS_API_KEY as apiKey, as the code samples on this page do:
ELEVENLABS_API_KEY=your-elevenlabs-api-keyThe key stays on the server, since the browser talks to @micdrop/server and only the server connects to ElevenLabs. ElevenLabs checks the key only after it accepts the connection. With a wrong key or a missing permission, each connection closes before any audio, until ElevenLabsTTS runs out of retries and emits Failed.
Finding Voice IDs
You can find voice IDs in the ElevenLabs dashboard, by browsing the voices of your account, or through the public voice library API.
FAQ
Does ElevenLabs have an API?
Yes. ElevenLabs has a REST API for text to speech, speech to text, voices
and its other products, plus WebSockets that stream speech while the text
is still arriving. Both take an API key in the xi-api-key header. A
voice agent needs the WebSockets, which @micdrop/elevenlabs wraps for
Node.js. The free plan includes API access, within its own
character quota and concurrency limit.
How do I get an API key for ElevenLabs?
Sign in to ElevenLabs, open the API keys page of your settings and create
a key. Restrict it to Text to Speech access, which covers everything
@micdrop/elevenlabs does, then set it as ELEVENLABS_API_KEY on your
server. A free account gets a key too, with the
monthly character quota of the free plan.
You can also give the key a
credit limit and an IP allowlist.
Which ElevenLabs WebSocket supports Eleven v4?
Eleven v4 and v4 Turbo stream only over the Text to Dialogue WebSocket,
wss://api.elevenlabs.io/v1/text-to-dialogue/stream-input, like Eleven v3.
The Text to Speech WebSocket refuses them with unsupported_model.
ElevenLabsTTS opens the Text to Dialogue one as soon as modelId names a
v3 or v4 model, and the Eleven v4 Turbo test
details what changes in its protocol.
How do I stream ElevenLabs text to speech in Node.js?
Install @micdrop/elevenlabs, create an ElevenLabsTTS with your API key
and a voice ID, and pass it to MicdropServer as tts. The server sends
each chunk of the LLM answer to ElevenLabs as it arrives and forwards the
audio to the browser. Outside Micdrop, call tts.speak() with a readable
stream of text and listen to the Audio event, as in the
usage without MicdropServer.
Which ElevenLabs model has the lowest latency?
Flash v2.5, at about 75 ms according to ElevenLabs, network excluded. Eleven v4 Turbo follows at about 100 ms and performs audio tags, which Flash reads out loud. Network included, a Micdrop test measured the first audio of v4 Turbo at about 90 ms in a streamed voice agent. The table of models compares them all.
Does Flash v2.5 read audio tags out loud?
Yes. Flash v2.5, Turbo v2.5 and Multilingual v2 speak a tag like
[laughs] as words, and only Eleven v3 and v4 perform it. When Flash
serves as a backup voice behind v4 Turbo, keep the tags in the system
prompt and strip them from the text Flash receives.