--- title: "Micdrop Protocol" description: "Micdrop uses a simple custom protocol over WebSocket for real-time communication between the client and server." url: "https://micdrop.dev/docs/server/protocol" --- * [Getting Started](/docs/getting-started) * [Client (Browser)](/docs/client) * [Installation](/docs/client/installation) * [React Hooks](/docs/client/react-hooks) * [Start/Stop Call](/docs/client/start-stop-call) * [Pause/Resume Call](/docs/client/pause-resume-call) * [Mute/Unmute Call](/docs/client/mute-unmute-call) * [Call State](/docs/client/call-state) * [Display Conversation Messages](/docs/client/display-conversation-messages) * [Handling Tool Calls](/docs/client/handling-tool-calls) * [Device Management](/docs/client/devices-management) * [Voice Activity Detection (VAD)](/docs/client/vad) * [Error Handling](/docs/client/error-handling) * Utility Classes * [Mic](/docs/client/utility-classes/mic) * [MicdropClient](/docs/client/utility-classes/micdrop-client) * [MicRecorder](/docs/client/utility-classes/mic-recorder) * [Speaker](/docs/client/utility-classes/speaker) * [Server (Node.js)](/docs/server) * [Installation](/docs/server/installation) * [With Fastify](/docs/server/with-fastify) * [With NestJS](/docs/server/with-nestjs) * [Auth and Parameters](/docs/server/auth-and-parameters) * [First Message](/docs/server/first-message) * [Save Messages](/docs/server/save-messages) * [Resume a Conversation](/docs/server/resume-conversation) * [Recording Audio](/docs/server/recording-audio) * [Error Handling](/docs/server/error-handling) * [Tools](/docs/server/tools) * [Extract Value from Answer](/docs/server/extract) * [Auto End Call](/docs/server/auto-end-call) * [Semantic Turn Detection](/docs/server/semantic-turn-detection) * [Noise Filtering](/docs/server/noise-filtering) * [Micdrop Protocol](/docs/server/protocol) * [AI Integrations](/docs/ai-integration) * Provided Integrations * [AI SDK](/docs/ai-integration/provided-integrations/ai-sdk) * [Cartesia](/docs/ai-integration/provided-integrations/cartesia) * [ElevenLabs](/docs/ai-integration/provided-integrations/elevenlabs) * [Gladia](/docs/ai-integration/provided-integrations/gladia) * [Gradium](/docs/ai-integration/provided-integrations/gradium) * [Mistral](/docs/ai-integration/provided-integrations/mistral) * [OpenAI](/docs/ai-integration/provided-integrations/openai) * Custom Integrations * [Agent (LLM)](/docs/ai-integration/custom-integrations/custom-agent) * [Speech-to-Text (STT)](/docs/ai-integration/custom-integrations/custom-stt) * [Text-to-Speech (TTS)](/docs/ai-integration/custom-integrations/custom-tts) * Fallback Strategies * [FallbackAgent](/docs/ai-integration/fallback-strategies/agent-fallback) * [FallbackSTT](/docs/ai-integration/fallback-strategies/stt-fallback) * [FallbackTTS](/docs/ai-integration/fallback-strategies/tts-fallback) * [IA Vocale Souveraine 🇫🇷🇪🇺](/docs/ai-integration/sovereign-voice-ai) [Micdrop](/) › [Documentation](/docs/getting-started) # Micdrop Protocol ## WebSocket Protocol Micdrop uses a simple custom protocol over WebSocket for real-time communication between the client and server. sequenceDiagram participant W as Client participant B as Server Note left of W: Audio setup W -->> B: Create call B -->> W: Message (First assistant message) Note right of B: Start TTS : Generate voice for first message B -->> W: Audio chunk (First assistant message) Note left of W: Play assistant speech B -->> W: Audio chunk (First assistant message) Note right of B: Stop TTS - Sent audio for first message loop Note left of W: Wait until user speaks W -->> B: StartSpeaking Note right of B: Start STT : Transcribe user speech W -->> B: Audio chunk (User speech) W -->> B: Audio chunk (User speech) Note left of W: Silence W -->> B: StopSpeaking - User stops speaking Note right of B: Stop STT : User speech transcribed B -->> W: Message (Transcribed user message) Note right of B: Start Agent : Generate answer Note right of B: Start TTS : Generate voice for answer B ->> W: Audio chunk (Assistant answer) B ->> W: Audio chunk (Assistant answer) B -->> W: Message (Assistant answer) Note right of B: Stop Agent : Finished answering B ->> W: Audio chunk (Assistant answer) B ->> W: Audio chunk (Assistant answer) Note right of B: Stop TTS : Sent audio for answer Note left of W: Play assistant speech end ## Why WebSocket? While WebRTC is a powerful protocol for real-time communication, Micdrop uses a simple custom protocol over WebSocket for several reasons: * 🎯 **Focused on our use case**: WebRTC is designed for peer-to-peer communication, with features we don’t need. Our client-server architecture is simpler. * 🔇 **Efficient audio transmission**: By using [Voice Activity Detection (VAD)](/docs/client/vad) on the client side, we only send audio when the user is actually speaking. This reduces bandwidth usage and processing load compared to continuous streaming. * 💡 **Simple implementation**: WebSocket provides a straightforward, reliable way to send both text and binary data. The protocol is easy to implement and debug on both client and server. * 🔄 **Bidirectional communication**: WebSocket allows for real-time bidirectional messaging, which is perfect for our text and audio exchange needs. * 🛠️ **Custom protocol control**: Our simple protocol lets us optimize exactly how and when audio/text messages are sent, without the overhead of WebRTC’s full feature set. This approach gives us the real-time capabilities we need while keeping the implementation lean and efficient. ## Client Commands The client can send the following commands to the server: * `MicdropClientCommands.StartSpeaking` - The user starts speaking * `MicdropClientCommands.StopSpeaking` - The user stops speaking * `MicdropClientCommands.Mute` - The user mutes the microphone ## Server Commands The server can send the following commands to the client: * `MicdropServerCommands.Message` - A message from the assistant. * `MicdropServerCommands.CancelLastUserMessage` - Cancel the last user message. * `MicdropServerCommands.SkipAnswer` - Notify that the last generated answer was ignored, it’s listening again. * `MicdropServerCommands.EndCall` - End the call. [Previous← Noise Filtering](/docs/server/noise-filtering)[NextAI Integrations →](/docs/ai-integration) On this page * [WebSocket Protocol](#websocket-protocol) * [Why WebSocket?](#why-websocket) * [Client Commands](#client-commands) * [Server Commands](#server-commands)