🎤Micdrop

Server (Node.js)

Micdrop server orchestrates voice conversations by integrating AI agents, speech-to-text, and text-to-speech services with WebSocket communication.

Installation

Terminal window
npm install @micdrop/server

See Installation for more details.

Quick Example

import { MicdropServer } from '@micdrop/server'
import { OpenaiAgent } from '@micdrop/openai'
import { GladiaSTT } from '@micdrop/gladia'
import { ElevenLabsTTS } from '@micdrop/elevenlabs'
import { WebSocketServer } from 'ws'


const wss = new WebSocketServer({ port: 8081 })


wss.on('connection', (socket) => {
  // Handle voice conversation
  new MicdropServer(socket, {
    firstMessage: 'How can I help you today?',


    agent: new OpenaiAgent({
      apiKey: process.env.OPENAI_API_KEY,
      systemPrompt: 'You are a helpful assistant',
    }),


    stt: new GladiaSTT({
      apiKey: process.env.GLADIA_API_KEY,
    }),


    tts: new ElevenLabsTTS({
      apiKey: process.env.ELEVENLABS_API_KEY,
      voiceId: process.env.ELEVENLABS_VOICE_ID,
    }),
  })
})

Calls that transcribe or write

agent and tts are optional. A server given a speech to text alone transcribes and stays quiet, which is all a dictation tool needs. One given an agent and no voice answers in writing.

new MicdropServer(socket, {
  stt: new GladiaSTT({ apiKey: process.env.GLADIA_API_KEY }),
})

See Dictation and text-only calls.

Demo

The basic example is the shortest server you can write, a ws server and the three OpenAI providers in forty lines.

For a wider tour, check out the advanced example, it shows:

  • Setting up a Fastify server with WebSocket support
  • Configuring the MicdropServer with custom handlers
  • Basic authentication flow
  • Example agent, speech-to-text and text-to-speech implementations
  • Error handling patterns

Core Components

  • MicdropServer - Main server class for conversation orchestration
  • Agent - AI agent base class and implementations, optional
  • STT - Speech-to-text base class and implementations
  • TTS - Text-to-speech base class and implementations, optional
  • Realtime - Speech-to-speech models, in place of the three above
  • Classifier, typed decisions about what the user says, optional

Framework Integrations