---
title: "Write a Custom Speech to Text (STT) | Micdrop"
description: "The STT class is the core abstraction for speech-to-text functionality in Micdrop. It provides a standardized interface for integrating various speech-to-text…"
url: "https://micdrop.dev/docs/speech-to-text/custom"
---

# Write a Custom Speech to Text (STT)

The `STT` class is the core abstraction for speech-to-text functionality in Micdrop. It provides a standardized interface for integrating various speech-to-text providers into real-time voice conversations.

## Available Implementations

*   [OpenaiSTT](https://github.com/Godefroy/micdrop/blob/main/packages/openai/src/OpenaiSTT.ts) from [@micdrop/openai](/docs/speech-to-text/openai)
*   [GladiaSTT](https://github.com/Godefroy/micdrop/blob/main/packages/gladia/src/GladiaSTT.ts) from [@micdrop/gladia](/docs/speech-to-text/gladia)
*   [MockSTT](https://github.com/Godefroy/micdrop/blob/main/packages/server/src/stt/MockSTT.ts) for testing

For automatic failover between multiple STT providers, see [FallbackSTT](/docs/speech-to-text/fallback).

`MicdropServer` accepts an STT on its own, yours included, for [dictation without an agent or a voice](/docs/server/dictation), where each sentence reaches the client as a message.

## Overview

The `STT` class is an abstract base class that extends `EventEmitter` and manages:

*   Real-time audio stream processing
*   Automatic audio format detection
*   Event emission for transcription results
*   Integration with logging systems
*   Resource cleanup and cancellation

```typescript
export abstract class STT extends EventEmitter<STTEvents> {
  public logger?: Logger

  // Transcribe audio stream to text (emits Transcript event)
  abstract transcribe(audioStream: Readable): void

  // Cleanup
  destroy(): void
}
```

## Events

The STT class emits the following events:

### Transcript

Emitted when a transcription is ready.

```typescript
stt.on('Transcript', (text: string) => {
  console.log('Transcript:', text)
})
```

### Failed

Emitted when the STT service fails after exhausting all retries. This event provides the buffered audio chunks that were pending transcription.

```typescript
stt.on('Failed', (audioChunks: Buffer[]) => {
  console.error('STT failed with', audioChunks.length, 'pending audio chunks')
  // Handle failure (e.g., notify user, fallback to another STT)
})
```

## Debug Logging

Enable detailed logging for development:

```typescript
// Enable debug logging
stt.logger = new Logger('CustomSTT')
```

## Custom STT Implementation

### Creating a Real-time STT Implementation

For services that support real-time streaming transcription. The server gives each utterance its own stream and expects exactly one `Transcript` per stream, in the order they were spoken, even when a new utterance starts before the transcript of the previous one came back.

```typescript
import { ReconnectingSocket, STT } from '@micdrop/server'
import { Readable } from 'stream'

interface Utterance {
  chunks: Buffer[]
  ended: boolean
  text: string
  timeout?: NodeJS.Timeout
}

export class CustomRealtimeSTT extends STT {
  private socket: ReconnectingSocket
  // Waiting for their transcript, in the order they were spoken
  private utterances: Utterance[] = []

  constructor(private options: { apiKey: string; language?: string }) {
    super()
    this.socket = new ReconnectingSocket({
      url: `wss://api.example.com/v1/stream?key=${options.apiKey}`,
      // The session is ready once the service accepts its configuration
      handshake: {
        start: (socket) =>
          socket.send(
            JSON.stringify({ type: 'config', language: options.language })
          ),
        read: (data) => JSON.parse(data.toString()).type === 'ready',
      },
      keepAlive: { interval: 30000, message: '{"type":"ping"}' },
      log: (...message) => this.log(...message),
    })
    this.socket.on('Open', this.onOpen)
    this.socket.on('Message', (data) =>
      this.handleMessage(JSON.parse(data.toString()))
    )
    this.socket.on('Failed', this.onFailed)
    this.socket.connect()
  }

  transcribe(audioStream: Readable) {
    const utterance: Utterance = { chunks: [], ended: false, text: '' }
    this.utterances.push(utterance)

    audioStream.on('data', (chunk: Buffer) => {
      utterance.chunks.push(chunk)
      // Without a connection, the chunk goes out with the others on `Open`
      this.socket.send(chunk)
    })

    audioStream.on('end', () => {
      utterance.ended = true
      this.socket.send(JSON.stringify({ type: 'end_stream' }))
      // A transcript that never comes still settles the utterance
      utterance.timeout = setTimeout(() => this.finish(utterance), 4000)
    })
  }

  destroy() {
    super.destroy()
    this.utterances.forEach((utterance) => clearTimeout(utterance.timeout))
    this.utterances = []
    this.socket.destroy()
  }

  // A new session knows nothing of the previous one: the audio waiting for a
  // transcript goes again
  private onOpen = ({ reconnected }: { reconnected: boolean }) => {
    if (!reconnected) return
    for (const utterance of this.utterances) {
      utterance.text = ''
      utterance.chunks.forEach((chunk) => this.socket.send(chunk))
      if (utterance.ended) {
        this.socket.send(JSON.stringify({ type: 'end_stream' }))
      }
    }
  }

  private handleMessage(message: any) {
    // The service answers in order, so its words belong to the oldest one
    const utterance = this.utterances[0]
    if (!utterance) return
    if (message.type === 'transcript' && message.is_final) {
      utterance.text += message.text
    } else if (message.type === 'stream_ended') {
      this.finish(utterance)
    }
  }

  private finish(utterance: Utterance) {
    if (!this.utterances.includes(utterance)) return
    // Those spoken before go first, with what they have
    while (this.utterances.length > 0) {
      const first = this.utterances.shift()!
      clearTimeout(first.timeout)
      this.emit('Transcript', first.text.trim())
      if (first === utterance) break
    }
  }

  // Every retry failed: the audio goes back to the caller, a FallbackSTT
  // for instance
  private onFailed = () => {
    const chunks = this.utterances.flatMap((utterance) => utterance.chunks)
    this.utterances.forEach((utterance) => clearTimeout(utterance.timeout))
    this.utterances = []
    this.emit('Failed', chunks)
  }
}
```

### Connecting over a WebSocket

`ReconnectingSocket` holds the connection of a provider and reconnects on its own, so an integration only deals with its protocol. It tries one connection at a time, gives up after `maxRetry` failed attempts in a row, reconnects after a close from the server whatever its code, and silences a socket it left behind so it never acts on the next one. A connection that closes before it proved usable, with a handshake, a first message or by staying open, counts as a failed attempt.

Option

Type

Default

Description

`url`

`string | () => string | Promise<string>`

Address, or a function giving it for each attempt

`clientOptions`

`ClientOptions`

Options of the `ws` client, headers included

`connectionTimeout`

`number`

`5000`

How long an attempt may take, handshake included, in ms

`retryDelay`

`number`

`1000`

Delay before a new attempt, in ms

`maxRetry`

`number`

`3`

Failed attempts in a row before giving up

`handshake`

`{ start?, read? }`

Sets the session up before the connection counts as open

`keepAlive`

`{ interval, message }`

Message sent on a regular basis while open

`log`

`(...message) => void`

Receives the log lines

`status` reads `idle`, `connecting`, `open`, `reconnecting`, `failed` or `destroyed`, and `isOpen` tells whether a message sent now goes out. `send()` returns false when the message did not go out, and `ready()` resolves to true once open, or to false if the connection failed or was destroyed meanwhile. `restart()` drops the connection for a new one right away, for a provider that can only stop a synthesis by closing. After `failed`, `connect()` starts over with a full budget. `destroy()` closes the connection for good, a retry still waiting included, and settles `ready()` with false.

Event

Payload

Description

`Open`

`{ reconnected: boolean }`

The moment to send again what a dropped connection lost

`Message`

`RawData, boolean`

A message, once the handshake is done

`Close`

`number, string`

The connection dropped, a new attempt follows

`Failed`

`Error`

Every retry failed

`StatusChange`

`ReconnectingSocketStatus`

The status changed

### Using CustomRealtimeSTT with MicdropServer

```typescript
// Create custom STT
const stt = new CustomRealtimeSTT({
  apiKey: process.env.CUSTOM_STT_API_KEY || '',
  language: 'en',
})

// Add logging
stt.logger = new Logger('CustomSTT')

// Create server with custom STT
const server = new MicdropServer(socket, {
  stt,
  // ... other options
})
```
