--- title: "AI Integration | Micdrop" description: "Integrate speech-to-text, text-to-speech, and AI agents from multiple providers or build custom implementations." url: "https://micdrop.dev/docs/ai-integration" --- * [Getting Started](/docs/getting-started) * [Client (Browser)](/docs/client) * [Installation](/docs/client/installation) * [React Hooks](/docs/client/react-hooks) * [Start/Stop Call](/docs/client/start-stop-call) * [Pause/Resume Call](/docs/client/pause-resume-call) * [Mute/Unmute Call](/docs/client/mute-unmute-call) * [Call State](/docs/client/call-state) * [Display Conversation Messages](/docs/client/display-conversation-messages) * [Handling Tool Calls](/docs/client/handling-tool-calls) * [Device Management](/docs/client/devices-management) * [Voice Activity Detection (VAD)](/docs/client/vad) * [Turn Detection](/docs/client/turn-detection) * [Reducing Latency](/docs/client/latency) * [Error Handling](/docs/client/error-handling) * Utility Classes * [Mic](/docs/client/utility-classes/mic) * [MicdropClient](/docs/client/utility-classes/micdrop-client) * [MicRecorder](/docs/client/utility-classes/mic-recorder) * [Speaker](/docs/client/utility-classes/speaker) * [Client (React Native)](/docs/react-native) * [Installation](/docs/react-native/installation) * [Hooks and Call State](/docs/react-native/hooks) * [Audio Output and Devices](/docs/react-native/audio-output) * [Voice Activity Detection (VAD)](/docs/react-native/vad) * [Turn Detection](/docs/react-native/turn-detection) * [Using Another Audio Library](/docs/react-native/custom-audio) * [Server (Node.js)](/docs/server) * [Installation](/docs/server/installation) * [With Fastify](/docs/server/with-fastify) * [With NestJS](/docs/server/with-nestjs) * [Auth and Parameters](/docs/server/auth-and-parameters) * [First Message](/docs/server/first-message) * [Dictation and Text-Only Calls](/docs/server/dictation) * [Realtime Models](/docs/server/realtime) * [Partial Messages](/docs/server/partial-messages) * [Save Messages](/docs/server/save-messages) * [Resume a Conversation](/docs/server/resume-conversation) * [Recording Audio](/docs/server/recording-audio) * [Error Handling](/docs/server/error-handling) * [Tools](/docs/server/tools) * [Extract Value from Answer](/docs/server/extract) * [Auto End Call](/docs/server/auto-end-call) * [Semantic Turn Detection](/docs/server/semantic-turn-detection) * [Noise Filtering](/docs/server/noise-filtering) * [Micdrop Protocol](/docs/server/protocol) * [AI Integrations](/docs/ai-integration) * Provided Integrations * [AI SDK](/docs/ai-integration/provided-integrations/ai-sdk) * [Cartesia](/docs/ai-integration/provided-integrations/cartesia) * [ElevenLabs](/docs/ai-integration/provided-integrations/elevenlabs) * [Gemini](/docs/ai-integration/provided-integrations/gemini) * [Gladia](/docs/ai-integration/provided-integrations/gladia) * [Gradium](/docs/ai-integration/provided-integrations/gradium) * [Kokoro](/docs/ai-integration/provided-integrations/kokoro) * [Mistral](/docs/ai-integration/provided-integrations/mistral) * [OpenAI](/docs/ai-integration/provided-integrations/openai) * [Piper](/docs/ai-integration/provided-integrations/piper) * [Pocket TTS](/docs/ai-integration/provided-integrations/pocket-tts) * [Qwen3-TTS](/docs/ai-integration/provided-integrations/qwen-tts) * [Whisper](/docs/ai-integration/provided-integrations/whisper) * Custom Integrations * [Agent (LLM)](/docs/ai-integration/custom-integrations/custom-agent) * [Speech-to-Text (STT)](/docs/ai-integration/custom-integrations/custom-stt) * [Text-to-Speech (TTS)](/docs/ai-integration/custom-integrations/custom-tts) * Fallback Strategies * [FallbackAgent](/docs/ai-integration/fallback-strategies/agent-fallback) * [FallbackSTT](/docs/ai-integration/fallback-strategies/stt-fallback) * [FallbackTTS](/docs/ai-integration/fallback-strategies/tts-fallback) * [Local Models](/docs/ai-integration/local-models) * [Local LLM](/docs/ai-integration/local-models/agent) * [Local STT](/docs/ai-integration/local-models/speech-to-text) * [Local TTS](/docs/ai-integration/local-models/text-to-speech) * [Latency and Memory](/docs/ai-integration/local-models/performance) * [Explorations](/docs/ai-integration/local-models/explorations) * [MiniCPM5-2B](/docs/ai-integration/local-models/explorations/minicpm) * [Mistral 7B](/docs/ai-integration/local-models/explorations/mistral-7b) * [Voxtral Mini 3B](/docs/ai-integration/local-models/explorations/voxtral-stt) * [Voxtral TTS 4B](/docs/ai-integration/local-models/explorations/voxtral-tts) * [AuK and AuK-Flash](/docs/ai-integration/local-models/explorations/auk) * [IA Vocale Souveraine πŸ‡«πŸ‡·πŸ‡ͺπŸ‡Ί](/docs/ai-integration/sovereign-voice-ai) * [Migration](/docs/migration) * [Upgrade to v3](/docs/migration/v3) [Micdrop](/) β€Ί [Documentation](/docs/getting-started) # AI Integration Integrate speech-to-text, text-to-speech, and AI agents from multiple providers or build custom implementations. ## Overview Micdrop provides a modular AI architecture allowing you to: * **Mix and match** providers for optimal cost and quality * **Run everything locally**, with no API key and no data leaving your machine * **Build custom integrations** using abstract base classes Splitting transcription, the agent and the voice into three services is one of two architectures. The other sends audio straight to a [realtime model](/docs/server/realtime) such as Gemini Live or [the OpenAI Realtime API](/blog/openai-realtime-api-vs-pipeline), where one connection comes with a faster answer, the voice of that provider and a single bill. ## Provider Categories ### Provided Integrations Ready-to-use implementations for popular AI services: **Speech-to-Text (STT):** * **[Gladia](/docs/ai-integration/provided-integrations/gladia)** - Fast, accurate multilingual transcription * **[Gemini Transcribe](/docs/ai-integration/provided-integrations/gemini#gemini-stt-speech-to-text)** - Streaming transcription, 85+ languages * **[OpenAI Whisper](/docs/ai-integration/provided-integrations/openai)** - High-quality speech recognition * **[Mistral Voxtral](/docs/ai-integration/provided-integrations/mistral)** - Real-time transcription * **[Gradium](/docs/ai-integration/provided-integrations/gradium)** - Real-time transcription * **[Whisper](/docs/ai-integration/provided-integrations/whisper)** - Local transcription, in your Node process **Text-to-Speech (TTS):** * **[ElevenLabs](/docs/ai-integration/provided-integrations/elevenlabs)** - High-quality voice synthesis * **[Cartesia](/docs/ai-integration/provided-integrations/cartesia)** - Low-latency streaming TTS * **[Gemini TTS](/docs/ai-integration/provided-integrations/gemini#gemini-tts-text-to-speech)** - 30 voices, directed with plain instructions * **[Kokoro](/docs/ai-integration/provided-integrations/kokoro)** - Local voice, in your Node process, English only * **[Piper](/docs/ai-integration/provided-integrations/piper)** - Local voice, around forty languages * **[Pocket TTS](/docs/ai-integration/provided-integrations/pocket-tts)** - Local voice cloning, in your Node process, English only * **[Qwen3-TTS](/docs/ai-integration/provided-integrations/qwen-tts)** - Local voice, ten languages, on an mlx-audio server **AI Agents (LLM):** * **[AI SDK](/docs/ai-integration/provided-integrations/ai-sdk)** - Universal provider using Vercel AI SDK * **[OpenAI](/docs/ai-integration/provided-integrations/openai)** - GPT models for conversation * **[Mistral](/docs/ai-integration/provided-integrations/mistral)** - Open-source and commercial LLMs * **[Gemini](/docs/ai-integration/provided-integrations/gemini#gemini-agent)** - Gemini Flash models for conversation **Realtime models (speech to speech):** * **[Gemini Live](/docs/ai-integration/provided-integrations/gemini#gemini-live)** - Hears and speaks in one connection * **[OpenAI Realtime](/docs/ai-integration/provided-integrations/openai#openai-realtime)** - Hears and speaks in one connection ### Running Locally The three parts above all have a local counterpart, so a whole conversation can run on your machine. Read the [local models guide](/docs/ai-integration/local-models) to set it up. ### Custom Integrations Build your own integrations using abstract base classes: * **[Custom Agent](/docs/ai-integration/custom-integrations/custom-agent)** - Create custom AI agents * **[Custom STT](/docs/ai-integration/custom-integrations/custom-stt)** - Implement speech-to-text services * **[Custom TTS](/docs/ai-integration/custom-integrations/custom-tts)** - Build text-to-speech providers ## Quick Start ### Basic Setup ``` import { MicdropServer } from '@micdrop/server'import { OpenaiAgent } from '@micdrop/openai'import { ElevenLabsTTS } from '@micdrop/elevenlabs'import { GladiaSTT } from '@micdrop/gladia' new MicdropServer(socket, { // AI Agent for conversation agent: new OpenaiAgent({ apiKey: process.env.OPENAI_API_KEY, model: 'gpt-4-turbo-preview', }), // Speech-to-Text stt: new GladiaSTT({ apiKey: process.env.GLADIA_API_KEY, settings: { language_config: { languages: ['en'] } }, }), // Text-to-Speech tts: new ElevenLabsTTS({ apiKey: process.env.ELEVENLABS_API_KEY, voiceId: 'voice-id-here', }),}) ``` ### Cost-Optimized Setup ``` // Use different providers for optimal cost/quality balancenew MicdropServer(socket, { agent: new MistralAgent({ // Cost-effective LLM apiKey: process.env.MISTRAL_API_KEY, model: 'mistral-large-latest', }), stt: new GladiaSTT({ // Fast, affordable STT apiKey: process.env.GLADIA_API_KEY, }), tts: new CartesiaTTS({ // Low-latency TTS apiKey: process.env.CARTESIA_API_KEY, voiceId: 'cartesia-voice-id', }),}) ``` ## Models Comparison ### Speech-to-Text Comparison Provider Latency Languages Cost per hour Best For **Gladia πŸ‡«πŸ‡·** ~200ms 99+ $0.75 General use, multilingual **Gradium πŸ‡«πŸ‡·** ~300ms 5 $0.62 French, EU data residency **OpenAI Whisper** ~300ms 57 $0.36 High accuracy, multilingual **Whisper (local)** ~440ms 99 Free Privacy, no API key ### Text-to-Speech Comparison Provider Latency Quality Voices Cost per hour Best For **ElevenLabs** ~400ms Excellent 1000+ $9.00 High-quality voices **Cartesia** ~150ms Good 50+ $1.80 Low-latency streaming **Gradium πŸ‡«πŸ‡·** ~300ms Excellent 150+ $2.60 French, EU data residency **Kokoro (local)** ~1000ms Good 28 Free Privacy, English only **Piper (local)** ~390ms Fair 100+ Free Privacy, forty languages **Qwen3-TTS (local)** ~400ms Very good 9 Free Privacy, ten languages Costs are the published list price of the entry paid plan in August 2026, per hour of audio generated or transcribed. Every provider charges less as the volume grows, and Gladia goes furthest with real-time transcription at $0.25 an hour on a commitment. The plans priced per character are converted at the 45,000 characters an hour Gradium publishes for its own voices. ### AI Agent Comparison Provider Speed Quality Cost Best For **AI SDK** Varies Varies Varies Universal compatibility, any model **OpenAI GPT-4** Medium Excellent $$$ Complex reasoning **OpenAI GPT-3.5** Fast Good $ Simple conversations **Mistral Large πŸ‡«πŸ‡·** Fast Excellent $$ Cost-effective quality **Qwen3 4B Instruct (local)** Fast Fair Free Privacy, no API key [Previous← Micdrop Protocol](/docs/server/protocol)[NextAI SDK β†’](/docs/ai-integration/provided-integrations/ai-sdk) On this page * [Overview](#overview) * [Provider Categories](#provider-categories) * [Provided Integrations](#provided-integrations) * [Running Locally](#running-locally) * [Custom Integrations](#custom-integrations) * [Quick Start](#quick-start) * [Basic Setup](#basic-setup) * [Cost-Optimized Setup](#cost-optimized-setup) * [Models Comparison](#models-comparison) * [Speech-to-Text Comparison](#speech-to-text-comparison) * [Text-to-Speech Comparison](#text-to-speech-comparison) * [AI Agent Comparison](#ai-agent-comparison)