--- title: "Speech-to-Text (STT) | Micdrop" description: "The STT class is the core abstraction for speech-to-text functionality in Micdrop. It provides a standardized interface for integrating various speech-to-text…" url: "https://micdrop.dev/docs/ai-integration/custom-integrations/custom-stt" --- * [Getting Started](/docs/getting-started) * [Client (Browser)](/docs/client) * [Installation](/docs/client/installation) * [React Hooks](/docs/client/react-hooks) * [Start/Stop Call](/docs/client/start-stop-call) * [Pause/Resume Call](/docs/client/pause-resume-call) * [Mute/Unmute Call](/docs/client/mute-unmute-call) * [Call State](/docs/client/call-state) * [Display Conversation Messages](/docs/client/display-conversation-messages) * [Handling Tool Calls](/docs/client/handling-tool-calls) * [Device Management](/docs/client/devices-management) * [Voice Activity Detection (VAD)](/docs/client/vad) * [Error Handling](/docs/client/error-handling) * Utility Classes * [Mic](/docs/client/utility-classes/mic) * [MicdropClient](/docs/client/utility-classes/micdrop-client) * [MicRecorder](/docs/client/utility-classes/mic-recorder) * [Speaker](/docs/client/utility-classes/speaker) * [Server (Node.js)](/docs/server) * [Installation](/docs/server/installation) * [With Fastify](/docs/server/with-fastify) * [With NestJS](/docs/server/with-nestjs) * [Auth and Parameters](/docs/server/auth-and-parameters) * [First Message](/docs/server/first-message) * [Save Messages](/docs/server/save-messages) * [Resume a Conversation](/docs/server/resume-conversation) * [Recording Audio](/docs/server/recording-audio) * [Error Handling](/docs/server/error-handling) * [Tools](/docs/server/tools) * [Extract Value from Answer](/docs/server/extract) * [Auto End Call](/docs/server/auto-end-call) * [Semantic Turn Detection](/docs/server/semantic-turn-detection) * [Noise Filtering](/docs/server/noise-filtering) * [Micdrop Protocol](/docs/server/protocol) * [AI Integrations](/docs/ai-integration) * Provided Integrations * [AI SDK](/docs/ai-integration/provided-integrations/ai-sdk) * [Cartesia](/docs/ai-integration/provided-integrations/cartesia) * [ElevenLabs](/docs/ai-integration/provided-integrations/elevenlabs) * [Gladia](/docs/ai-integration/provided-integrations/gladia) * [Gradium](/docs/ai-integration/provided-integrations/gradium) * [Mistral](/docs/ai-integration/provided-integrations/mistral) * [OpenAI](/docs/ai-integration/provided-integrations/openai) * Custom Integrations * [Agent (LLM)](/docs/ai-integration/custom-integrations/custom-agent) * [Speech-to-Text (STT)](/docs/ai-integration/custom-integrations/custom-stt) * [Text-to-Speech (TTS)](/docs/ai-integration/custom-integrations/custom-tts) * Fallback Strategies * [FallbackAgent](/docs/ai-integration/fallback-strategies/agent-fallback) * [FallbackSTT](/docs/ai-integration/fallback-strategies/stt-fallback) * [FallbackTTS](/docs/ai-integration/fallback-strategies/tts-fallback) * [IA Vocale Souveraine 🇫🇷🇪🇺](/docs/ai-integration/sovereign-voice-ai) [Micdrop](/) › [Documentation](/docs/getting-started) # Speech-to-Text (STT) The `STT` class is the core abstraction for speech-to-text functionality in Micdrop. It provides a standardized interface for integrating various speech-to-text providers into real-time voice conversations. ## Available Implementations * [OpenaiSTT](https://github.com/Godefroy/micdrop/blob/main/packages/openai/src/OpenaiSTT.ts) from [@micdrop/openai](/docs/ai-integration/provided-integrations/openai) * [GladiaSTT](https://github.com/Godefroy/micdrop/blob/main/packages/gladia/src/GladiaSTT.ts) from [@micdrop/gladia](/docs/ai-integration/provided-integrations/gladia) * [MockSTT](https://github.com/Godefroy/micdrop/blob/main/packages/server/src/stt/MockSTT.ts) for testing For automatic failover between multiple STT providers, see [FallbackSTT](/docs/ai-integration/fallback-strategies/stt-fallback). ## Overview The `STT` class is an abstract base class that extends `EventEmitter` and manages: * Real-time audio stream processing * Automatic audio format detection * Event emission for transcription results * Integration with logging systems * Resource cleanup and cancellation ``` export abstract class STT extends EventEmitter { public logger?: Logger // Transcribe audio stream to text (emits Transcript event) abstract transcribe(audioStream: Readable): void // Cleanup destroy(): void} ``` ## Events The STT class emits the following events: ### Transcript Emitted when a transcription is ready. ``` stt.on('Transcript', (text: string) => { console.log('Transcript:', text)}) ``` ### Failed Emitted when the STT service fails after exhausting all retries. This event provides the buffered audio chunks that were pending transcription. ``` stt.on('Failed', (audioChunks: Buffer[]) => { console.error('STT failed with', audioChunks.length, 'pending audio chunks') // Handle failure (e.g., notify user, fallback to another STT)}) ``` ## Debug Logging Enable detailed logging for development: ``` // Enable debug loggingstt.logger = new Logger('CustomSTT') ``` ## Custom STT Implementation ### Creating a Real-time STT Implementation For services that support real-time streaming transcription: ``` import { STT } from '@micdrop/server'import { Readable } from 'stream'import WebSocket from 'ws' export class CustomRealtimeSTT extends STT { private socket?: WebSocket private reconnectTimeout?: NodeJS.Timeout private keepAliveInterval?: NodeJS.Timeout constructor( private options: { apiKey: string language?: string } ) { super() } async transcribe(audioStream: Readable) { // Initialize WebSocket connection await this.initConnection() // Process incoming audio chunks audioStream.on('data', (chunk: Buffer) => { this.processAudioChunk(chunk) }) audioStream.on('end', () => { this.finalizeStream() }) audioStream.on('error', (error) => { this.log('Audio stream error:', error) this.emit('error', error) }) } private async initConnection() { if (this.socket) return const wsUrl = `wss://api.example.com/v1/stream?key=${this.options.apiKey}` this.socket = new WebSocket(wsUrl) this.socket.addEventListener('open', () => { this.log('Connected to STT service') this.sendConfiguration() this.startKeepAlive() }) this.socket.addEventListener('message', (event) => { this.handleMessage(JSON.parse(event.data)) }) this.socket.addEventListener('error', (error) => { this.log('WebSocket error:', error) this.emit('error', error) }) this.socket.addEventListener('close', ({ code, reason }) => { this.log(`Connection closed: ${code} ${reason}`) if (code !== 1000) { this.reconnect() } }) } private sendConfiguration() { if (!this.socket) return const config = { type: 'config', language: this.options.language || 'en', encoding: 'pcm', interim_results: true, } this.socket.send(JSON.stringify(config)) } private processAudioChunk(chunk: Buffer) { if (this.socket?.readyState === WebSocket.OPEN) { this.socket.send(chunk) } } private handleMessage(message: any) { switch (message.type) { case 'transcript': if (message.is_final && message.text) { this.log(`Final transcript: "${message.text}"`) this.emit('Transcript', message.text) } break case 'error': this.log('Service error:', message.error) this.emit('error', new Error(message.error)) break case 'ping': this.socket?.send(JSON.stringify({ type: 'pong' })) break } } private finalizeStream() { if (this.socket?.readyState === WebSocket.OPEN) { this.socket.send(JSON.stringify({ type: 'end_stream' })) } } private startKeepAlive() { this.keepAliveInterval = setInterval(() => { if (this.socket?.readyState === WebSocket.OPEN) { this.socket.send(JSON.stringify({ type: 'ping' })) } }, 30000) } private reconnect() { this.log('Attempting reconnection...') this.reconnectTimeout = setTimeout(() => { this.initConnection().catch(() => this.reconnect()) }, 1000) } destroy() { super.destroy() if (this.reconnectTimeout) { clearTimeout(this.reconnectTimeout) } if (this.keepAliveInterval) { clearInterval(this.keepAliveInterval) } if (this.socket) { this.socket.close(1000, 'Client disconnect') } }} ``` ### Using CustomRealtimeSTT with MicdropServer ``` // Create custom STTconst stt = new CustomRealtimeSTT({ apiKey: process.env.CUSTOM_STT_API_KEY || '', language: 'en',}) // Add loggingstt.logger = new Logger('CustomSTT') // Create server with custom STTconst server = new MicdropServer(socket, { stt, // ... other options}) ``` [Previous← Agent (LLM)](/docs/ai-integration/custom-integrations/custom-agent)[NextText-to-Speech (TTS) →](/docs/ai-integration/custom-integrations/custom-tts) On this page * [Available Implementations](#available-implementations) * [Overview](#overview) * [Events](#events) * [Transcript](#transcript) * [Failed](#failed) * [Debug Logging](#debug-logging) * [Custom STT Implementation](#custom-stt-implementation) * [Creating a Real-time STT Implementation](#creating-a-real-time-stt-implementation) * [Using CustomRealtimeSTT with MicdropServer](#using-customrealtimestt-with-micdropserver)