🎤Micdrop

OpenAI Realtime API in Node.js

A realtime model running on the OpenAI Realtime API. It hears the user and answers with its own voice, in place of a speech to text, an agent and a text to speech.

GPT-Live-1, OpenAI’s full-duplex voice model, needs its own endpoint and events, which OpenaiRealtime does not implement. GPT-Live-1 differs from GPT-Realtime-2.1 and Gemini 3.8 Live in price, turn-taking and tools.

Installation

Terminal window
npm install @micdrop/openai

The same package also provides an agent, a speech to text, a text to speech and a classifier. The providers page shows what every provider covers.

Usage with MicdropServer

import { OpenaiRealtime } from '@micdrop/openai'
import { MicdropServer } from '@micdrop/server'
const realtime = new OpenaiRealtime({
apiKey: process.env.OPENAI_API_KEY || '',
model: 'gpt-realtime-2.1', // Default model
voice: 'marin', // Default voice
systemPrompt: 'You are a helpful assistant',
language: 'en', // Optional, helps the transcription of the user
// Advanced features (optional)
autoEndCall: true, // Automatically end call when user requests
})
new MicdropServer(socket, {
realtime,
generateFirstMessage: true,
})

The Micdrop client detects when the user speaks, so the turn detection of the Realtime API is turned off. The audio of each turn is committed when the client closes it, and the answer is asked for right after. Tools are added with addTool(), as with OpenaiAgent.

When the user interrupts, the answer is cancelled and the conversation of the model is cut where the user most likely stopped hearing it.

Events

OpenaiRealtime emits the events of an agent, and two of its own.

EventPayloadDescription
AudioBufferA chunk of the voice of the model, PCM 16 bits, 16 kHz, mono.
PartialMessagestringThe transcript of the answer so far.
MessageMicdropConversationItemA message, a tool call or a tool result was added to the conversation.
ToolCallMicdropToolCallA tool declared with emitOutput ran, with its parameters and output.
CancelLastUserMessagenoneThe last user message was dropped because it carried no intent.
SkipAnswernoneThe model gave no spoken answer.
EndCallnoneThe model decided that the call is over.
FailednoneThe connection could not be restored after its retries.

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour OpenAI API key
systemPromptstringRequiredInstructions of the model
modelstring'gpt-realtime-2.1'Realtime model to use (gpt-realtime-2.1, gpt-realtime-mini, …)
voicestring'marin'Voice of the model
languagestringundefinedLanguage of the user, as an ISO 639-1 code, for the transcription
transcriptionModelstring'gpt-4o-transcribe'Model transcribing the user: gpt-live-transcribe or gpt-transcribe recommended, gpt-4o-transcribe shuts down on February 26, 2027
autoEndCallboolean | stringfalseEnds the call when the user asks for it
autoSemanticTurnboolean | stringfalseLets the model wait silently when the user did not finish their sentence
autoIgnoreUserNoiseboolean | stringfalseDrops the turn when it carries no meaning
connectionTimeoutnumber5000Time to open the connection, in milliseconds
retryDelaynumber1000Delay before reconnecting, in milliseconds
maxRetrynumber3Reconnection attempts before giving up

A Realtime session lasts sixty minutes at most. When the connection closes, OpenaiRealtime reconnects and sends the conversation written so far to the new session. The answer in progress ends with the old session, and a tool still running records its result in the conversation without answering on the new one. What the call sends while the connection opens, a greeting or a turn of the user, waits for it and goes out once it is back. After maxRetry failed attempts, Failed is emitted and every later answer ends right away.