---
title: "Write a Custom Text to Speech (TTS) | Micdrop"
description: "The TTS class is the core abstraction for text-to-speech functionality in Micdrop. It provides a standardized interface for integrating various text-to-speech…"
url: "https://micdrop.dev/docs/text-to-speech/custom"
---

# Write a Custom Text to Speech (TTS)

The `TTS` class is the core abstraction for **text-to-speech** functionality in Micdrop. It provides a standardized interface for integrating various text-to-speech providers into real-time voice conversations.

## Available Implementations

*   [ElevenLabsTTS](https://github.com/Godefroy/micdrop/blob/main/packages/elevenlabs/src/ElevenLabsTTS.ts) from [@micdrop/elevenlabs](/docs/text-to-speech/elevenlabs)
*   [CartesiaTTS](https://github.com/Godefroy/micdrop/blob/main/packages/cartesia/src/CartesiaTTS.ts) from [@micdrop/cartesia](/docs/text-to-speech/cartesia)
*   [MistralTTS](https://github.com/Godefroy/micdrop/blob/main/packages/mistral/src/MistralTTS.ts) from [@micdrop/mistral](/docs/text-to-speech/mistral), a `SentenceTTS` that reads one sentence per request
*   [MockTTS](https://github.com/Godefroy/micdrop/blob/main/packages/server/src/tts/MockTTS.ts) for testing

For automatic failover between multiple TTS providers, see [FallbackTTS](/docs/text-to-speech/fallback).

## Overview

The `TTS` class is an abstract base class that manages:

*   Real-time text stream processing
*   Audio stream generation and output
*   Cancellation and cleanup mechanisms
*   Integration with logging systems
*   Audio format conversion and optimization

```typescript
export abstract class TTS extends EventEmitter<TTSEvents> {
  public logger?: Logger

  // Convert text stream to audio (emits Audio event)
  abstract speak(textStream: Readable): void

  // Cancel current speech generation
  abstract cancel(): void

  // Protected logging method
  protected log(...message: any[]): void

  // Cleanup
  destroy(): void
}
```

## Events

The TTS class emits the following events:

### Audio

Emitted when an audio chunk is ready to be sent to the client.

```typescript
tts.on('Audio', (audioChunk: Buffer) => {
  console.log('Received audio chunk:', audioChunk.length, 'bytes')
  // Send to client
})
```

### Failed

Emitted when the TTS service fails after exhausting all retries. This event provides the buffered text chunks that were pending synthesis.

```typescript
tts.on('Failed', (textChunks: string[]) => {
  console.error('TTS failed with', textChunks.length, 'pending text chunks')
  // Handle failure (e.g., notify user, fallback to another TTS)
})
```

## Abstract Methods

### `speak(textStream: Readable): void`

Converts a streaming text input into audio, emitted as `Audio` events.

**Implementation Requirements:**

*   Accept a `Readable` stream containing text chunks
*   Emit each audio chunk with `this.emit('Audio', chunk)`, in PCM 16 bits at 16 kHz
*   A new `speak()` replaces the utterance still being spoken
*   Handle real-time streaming for optimal latency
*   Support cancellation via the `cancel()` method
*   Process text incrementally as it arrives
*   Handle text stream errors gracefully

### `cancel(): void`

Cancels the current speech generation process. Should:

*   Abort any ongoing API requests
*   Stop audio stream generation
*   Clean up resources (WebSocket connections, timers, etc.)
*   End audio output stream gracefully

## Public Methods

### `destroy()`

Cleans up the TTS instance:

*   Calls `cancel()` to stop ongoing operations
*   Logs destruction event
*   Performs final cleanup

## Debug Logging

Enable detailed logging for development:

```typescript
// Enable debug logging
tts.logger = new Logger('CustomTTS')
```

## Custom TTS Implementation

### Creating a WebSocket-based Streaming TTS

For real-time streaming text-to-speech services. `ReconnectingSocket`, described on the [STT page](/docs/speech-to-text/custom#connecting-over-a-websocket), holds the connection, so the class only deals with the protocol.

```typescript
import { ReconnectingSocket, TTS } from '@micdrop/server'
import { Readable } from 'stream'

interface CustomStreamingTTSOptions {
  apiKey: string
  voiceId: string
  language?: string
}

export class CustomStreamingTTS extends TTS {
  private socket: ReconnectingSocket
  // Numbers the utterances, so the audio of a cancelled one is dropped
  private sessionId = 0
  // Text of the utterance being spoken, sent again on a new connection
  private textSent = ''

  constructor(private readonly options: CustomStreamingTTSOptions) {
    super()
    this.socket = new ReconnectingSocket({
      url: `wss://api.example.com/v1/tts/stream?api_key=${options.apiKey}`,
      handshake: {
        start: (socket) =>
          socket.send(
            JSON.stringify({
              type: 'config',
              voice_id: options.voiceId,
              language: options.language || 'en',
              output_format: { encoding: 'pcm_s16le', sample_rate: 16000 },
            })
          ),
      },
      log: (...message) => this.log(...message),
    })
    this.socket.on('Open', this.onOpen)
    this.socket.on('Message', (data) =>
      this.handleMessage(JSON.parse(data.toString()))
    )
    this.socket.on('Failed', () => {
      const text = this.textSent
      this.cancel()
      // A FallbackTTS speaks the rest instead
      this.emit('Failed', text ? [text] : [])
    })
    this.socket.connect()
  }

  speak(textStream: Readable) {
    const sessionId = ++this.sessionId
    this.textSent = ''

    textStream.on('data', (chunk: Buffer) => {
      if (sessionId !== this.sessionId) return
      const text = chunk.toString('utf-8')
      this.textSent += text
      // Without a connection, the text goes out on `Open`
      this.send({ type: 'text', text, session_id: sessionId })
    })

    textStream.on('end', () => {
      if (sessionId !== this.sessionId) return
      this.send({ type: 'finalize', session_id: sessionId })
    })
  }

  cancel() {
    // The audio still on its way belongs to no utterance any more
    this.sessionId++
    this.textSent = ''
  }

  destroy() {
    super.destroy()
    this.socket.destroy()
  }

  // A new connection lost the utterance in progress, which starts over
  private onOpen = ({ reconnected }: { reconnected: boolean }) => {
    if (!reconnected || !this.textSent) return
    this.send({ type: 'text', text: this.textSent, session_id: this.sessionId })
  }

  private handleMessage(message: any) {
    if (message.session_id !== this.sessionId) return
    if (message.type === 'audio') {
      this.emit('Audio', Buffer.from(message.data, 'base64'))
    } else if (message.type === 'audio_end') {
      this.textSent = ''
    } else if (message.type === 'error') {
      this.log('TTS error:', message.error)
    }
  }

  private send(message: object) {
    this.socket.send(JSON.stringify(message))
  }
}
```

### Using CustomStreamingTTS with MicdropServer

```typescript
// Create custom TTS
const tts = new CustomStreamingTTS({
  apiKey: process.env.CUSTOM_TTS_API_KEY || '',
  voiceId: process.env.CUSTOM_VOICE_ID || '',
  language: 'en',
})

// Add logging
tts.logger = new Logger('CustomTTS')

// Create server with custom TTS
const server = new MicdropServer(socket, {
  tts,
  // ... other options
})
```

### Creating a Fetch-based TTS Implementation

For services that process complete text before generating audio:

```typescript
import { TTS } from '@micdrop/server'
import { Readable } from 'stream'
import { text } from 'stream/consumers'

interface CustomFetchTTSOptions {
  apiKey: string
  voiceId: string
  model?: string
  language?: string
}

export class CustomFetchTTS extends TTS {
  private currentRequest?: AbortController

  constructor(private readonly options: CustomFetchTTSOptions) {
    super()
  }

  speak(textStream: Readable) {
    // A new utterance replaces the one being spoken
    this.cancel()
    const request = new AbortController()
    this.currentRequest = request
    this.generateSpeech(textStream, request)
  }

  private async generateSpeech(textStream: Readable, request: AbortController) {
    try {
      // Collect all text first
      this.log('Collecting text content...')
      const textContent = await text(textStream)
      if (!textContent.trim() || request.signal.aborted) return

      this.log(`Generating speech for: "${textContent}"`)

      // Make API request
      const response = await fetch('https://api.example.com/v1/tts/generate', {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${this.options.apiKey}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          text: textContent,
          voice_id: this.options.voiceId,
          model: this.options.model || 'neural-v1',
          language: this.options.language || 'en',
          output_format: 'pcm_s16le', // 16-bit signed little-endian PCM at 16kHz
        }),
        signal: request.signal,
      })

      if (!response.ok) {
        throw new Error(
          `TTS API error: ${response.status} ${response.statusText}`
        )
      }

      if (!response.body) throw new Error('No audio data in response')
      const reader = response.body.getReader()
      while (true) {
        const { done, value } = await reader.read()
        if (done || request.signal.aborted) break
        this.emit('Audio', Buffer.from(value))
      }
    } catch (error) {
      // A cancelled request throws, which is no failure
      if (!request.signal.aborted) {
        this.log('TTS generation failed:', error)
      }
    } finally {
      if (this.currentRequest === request) this.currentRequest = undefined
    }
  }

  cancel(): void {
    this.currentRequest?.abort()
    this.currentRequest = undefined
  }
}
```

### Using CustomFetchTTS with MicdropServer

```typescript
// Create and configure custom TTS
const tts = new CustomFetchTTS({
  apiKey: process.env.CUSTOM_TTS_API_KEY || '',
  voiceId: process.env.CUSTOM_VOICE_ID || '',
  language: 'en',
})

// Add logging
tts.logger = new Logger('CustomTTS')

// Create server with custom TTS
const server = new MicdropServer(socket, {
  tts,
  // ... other options
})
```

### Simple Echo TTS Example

For testing and development:

```typescript
import { TTS } from '@micdrop/server'
import { readFileSync } from 'fs'
import { Readable } from 'stream'

export class EchoTTS extends TTS {
  private sampleAudio: Buffer

  constructor(sampleAudioPath: string) {
    super()
    // Raw PCM 16 bits at 16 kHz, played as the "speech" of any text
    this.sampleAudio = readFileSync(sampleAudioPath)
  }

  speak(textStream: Readable) {
    textStream.once('data', () => {
      this.log('Echoing sample audio for any text input')
      this.emit('Audio', this.sampleAudio)
    })
  }

  cancel(): void {
    // Nothing to cancel for this simple implementation
  }
}
```

### Using EchoTTS

```typescript
const echoTTS = new EchoTTS('/path/to/sample-audio.pcm')
echoTTS.logger = new Logger('EchoTTS')

echoTTS.on('Audio', (chunk) => {
  console.log(`Audio chunk received: ${chunk.length} bytes`)
})

const textStream = new PassThrough()
echoTTS.speak(textStream)
textStream.end('Hello, world!')
```

### Sharing a Local Model Between Calls

Each call creates its own TTS, while a local model takes from a second to a minute to load and holds its weights in memory. `SharedInstances` keeps one instance per configuration for the whole process, so every call after the first reuses it.

```typescript
import { SharedInstances } from '@micdrop/server'

const synthesizers = new SharedInstances<Synthesizer>()

function loadSynthesizer(model: string, device: string) {
  // Calls asking while it loads share the same load
  return synthesizers.load([model, device], () =>
    Synthesizer.load(model, { device })
  )
}
```

The key is any JSON value, usually the options the instance is built from. A failed load is forgotten, so the next call tries again. `get(key)` returns the instance loaded or loading without starting a load, `has(key)` tells whether there is one, `delete(key)` forgets one that broke, a crashed worker for instance, so the next call loads a new one, and `clear()` forgets them all. Releasing what a forgotten instance holds is left to the caller.

## Audio Format Considerations

### Output Format

TTS implementations should output audio in PCM format (specifically `pcm_s16le` - 16-bit signed little-endian PCM) for optimal compatibility and performance with Micdrop.

The recommended audio specifications are:

*   **Format**: PCM (pcm\_s16le)
*   **Sample Rate**: 16000 Hz
*   **Channels**: 1 (mono)
*   **Bit Depth**: 16-bit signed integers

### Why PCM?

PCM format is recommended because:

*   **Low latency**: No encoding/decoding overhead
*   **Universal compatibility**: Supported by all audio systems
*   **Real-time streaming**: Optimal for live conversation scenarios
*   **Simplicity**: Direct audio data without compression artifacts

### Streaming Considerations

For optimal real-time performance:

*   **Chunk Size**: Balance between latency and efficiency (typically 1-4KB chunks)
*   **Buffering**: Minimize buffering to reduce latency
*   **Error Handling**: Gracefully handle network interruptions
*   **Cancellation**: Support immediate cancellation for natural conversation flow

## Error Handling

Robust TTS implementations should handle:

```typescript
// A broken text stream ends the utterance with what was received
textStream.on('error', (error) => {
  this.log('Text stream error:', error)
})

// API errors
if (!response.ok) {
  throw new Error(`TTS API error: ${response.status}`)
}

// A cancelled request throws too, which is no failure
if (request.signal.aborted) return

// The service is out of reach for good: hand the unspoken text over, so a
// FallbackTTS speaks it instead
this.emit('Failed', [unspokenText])
```
