OpenAI Realtime API in Node.js
A realtime model running on the OpenAI Realtime API. It hears the user and answers with its own voice, in place of a speech to text, an agent and a text to speech.
GPT-Live-1, OpenAI’s full-duplex voice model, needs its own endpoint and events, which OpenaiRealtime does not implement. GPT-Live-1 differs from GPT-Realtime-2.1 and Gemini 3.8 Live in price, turn-taking and tools.
Installation
npm install @micdrop/openaiThe same package also provides an agent, a speech to text, a text to speech and a classifier. The providers page shows what every provider covers.
Usage with MicdropServer
import { OpenaiRealtime } from '@micdrop/openai'import { MicdropServer } from '@micdrop/server'
const realtime = new OpenaiRealtime({ apiKey: process.env.OPENAI_API_KEY || '', model: 'gpt-realtime-2.1', // Default model voice: 'marin', // Default voice systemPrompt: 'You are a helpful assistant', language: 'en', // Optional, helps the transcription of the user
// Advanced features (optional) autoEndCall: true, // Automatically end call when user requests})
new MicdropServer(socket, { realtime, generateFirstMessage: true,})The Micdrop client detects when the user speaks, so the turn detection of the Realtime API is turned off. The audio of each turn is committed when the client closes it, and the answer is asked for right after. Tools are added with addTool(), as with OpenaiAgent.
When the user interrupts, the answer is cancelled and the conversation of the model is cut where the user most likely stopped hearing it.
Events
OpenaiRealtime emits the events of an agent, and two of its own.
| Event | Payload | Description |
|---|---|---|
Audio | Buffer | A chunk of the voice of the model, PCM 16 bits, 16 kHz, mono. |
PartialMessage | string | The transcript of the answer so far. |
Message | MicdropConversationItem | A message, a tool call or a tool result was added to the conversation. |
ToolCall | MicdropToolCall | A tool declared with emitOutput ran, with its parameters and output. |
CancelLastUserMessage | none | The last user message was dropped because it carried no intent. |
SkipAnswer | none | The model gave no spoken answer. |
EndCall | none | The model decided that the call is over. |
Failed | none | The connection could not be restored after its retries. |
Options
| Option | Type | Default | Description |
|---|---|---|---|
apiKey | string | Required | Your OpenAI API key |
systemPrompt | string | Required | Instructions of the model |
model | string | 'gpt-realtime-2.1' | Realtime model to use (gpt-realtime-2.1, gpt-realtime-mini, …) |
voice | string | 'marin' | Voice of the model |
language | string | undefined | Language of the user, as an ISO 639-1 code, for the transcription |
transcriptionModel | string | 'gpt-4o-transcribe' | Model transcribing the user: gpt-live-transcribe or gpt-transcribe recommended, gpt-4o-transcribe shuts down on February 26, 2027 |
autoEndCall | boolean | string | false | Ends the call when the user asks for it |
autoSemanticTurn | boolean | string | false | Lets the model wait silently when the user did not finish their sentence |
autoIgnoreUserNoise | boolean | string | false | Drops the turn when it carries no meaning |
connectionTimeout | number | 5000 | Time to open the connection, in milliseconds |
retryDelay | number | 1000 | Delay before reconnecting, in milliseconds |
maxRetry | number | 3 | Reconnection attempts before giving up |
A Realtime session lasts sixty minutes at most. When the connection closes, OpenaiRealtime reconnects and sends the conversation written so far to the new session. The answer in progress ends with the old session, and a tool still running records its result in the conversation without answering on the new one. What the call sends while the connection opens, a greeting or a turn of the user, waits for it and goes out once it is back. After maxRetry failed attempts, Failed is emitted and every later answer ends right away.