Gemini

Google Gemini implementation for @micdrop/server.

This package provides an agent, a speech-to-text and a text-to-speech running on the Gemini API. Each one can be combined with any other provider. GeminiLive goes further: it is a realtime model that hears the user and answers with its own voice, in place of the three.

Installation

npm install @micdrop/gemini

Gemini Agent

Answers with Gemini models through the Interactions API. The conversation is sent on every turn, nothing is stored on Google’s side.

Usage with MicdropServer

import { GeminiAgent } from '@micdrop/gemini'
import { MicdropServer } from '@micdrop/server'

const agent = new GeminiAgent({
  apiKey: process.env.GEMINI_API_KEY || '',
  model: 'gemini-3.8-flash', // Default model
  thinkingLevel: 'low', // Optional, the lower the sooner the first word
  systemPrompt: 'You are a helpful assistant',

  // Advanced features (optional)
  autoEndCall: true, // Automatically end call when user requests
  autoSemanticTurn: true, // Handle incomplete sentences
  autoIgnoreUserNoise: true, // Filter out meaningless sounds
})

new MicdropServer(socket, {
  agent,
  // ... other options
})

Usage without MicdropServer

import { GeminiAgent } from '@micdrop/gemini'

const agent = new GeminiAgent({
  apiKey: process.env.GEMINI_API_KEY || '',
  systemPrompt: 'You are a helpful assistant',
})

agent.on('Message', (message) => console.log('Message:', message))

agent.addUserMessage('Hello, what can you do?')

// The answer is a text stream, written as the model generates it
agent.answer().on('data', (chunk) => process.stdout.write(chunk))

Events

See the Agent interface for the full contract.

Options

For a voice call, the delay before the first word matters most. gemini-3.5-flash-lite answers in about a second, and gemini-3.8-flash, more capable, takes several seconds even with thinkingLevel: 'low'.

Gemini STT (Speech-to-Text)

Streams the voice of the user to Gemini Transcribe over the Live API, and emits the transcript of each utterance a fraction of a second after it ends.

Usage with MicdropServer

import { GeminiSTT } from '@micdrop/gemini'
import { MicdropServer } from '@micdrop/server'

const stt = new GeminiSTT({
  apiKey: process.env.GEMINI_API_KEY || '',
  language: 'en-US', // Optional, detected when left out
  vocabulary: ['Micdrop'], // Optional, words to recognize
})

new MicdropServer(socket, {
  stt,
  // ... other options
})

Usage without MicdropServer

import { GeminiSTT } from '@micdrop/gemini'
import { createReadStream } from 'fs'

const stt = new GeminiSTT({ apiKey: process.env.GEMINI_API_KEY || '' })

stt.on('Transcript', (transcript) => console.log('Transcript:', transcript))

// Audio is raw PCM, 16 bits, 16 kHz, mono
stt.transcribe(createReadStream('speech.pcm'))

Events

See the STT interface for the full contract.

Options

Gemini TTS (Text-to-Speech)

Gives the answers a Gemini voice. The model reads a whole text at once, so the answer is cut into sentences, and the audio of each one is sent as the model streams it.

Usage with MicdropServer

import { GeminiTTS } from '@micdrop/gemini'
import { MicdropServer } from '@micdrop/server'

const tts = new GeminiTTS({
  apiKey: process.env.GEMINI_API_KEY || '',
  model: 'gemini-2.5-flash-preview-tts', // Default model
  voice: 'Kore', // Default voice

  // Direction put before each sentence (optional)
  instructions: 'Say in a calm and friendly tone',
})

new MicdropServer(socket, {
  tts,
  // ... other options
})

Usage without MicdropServer

import { GeminiTTS } from '@micdrop/gemini'
import { Readable } from 'stream'

const tts = new GeminiTTS({ apiKey: process.env.GEMINI_API_KEY || '' })

// Audio is raw PCM, 16 bits, 16 kHz, mono
tts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))
tts.on('Failed', (texts) => console.error('Failed:', texts))

tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))

Events

See the TTS interface for the full contract.

Options

The voice follows the language of the text, in about ninety languages.

Gemini now and then blocks an ordinary sentence as if its content were forbidden, then accepts the very same sentence on the next request. GeminiTTS asks again, up to maxRetry times, as long as no audio of that sentence has been played.

gemini-3.1-flash-tts-preview starts speaking after about a second, then streams the rest slower than it is spoken, which leaves gaps in a long sentence. gemini-2.5-flash-preview-tts takes about three seconds and sends each sentence whole, so it plays without a gap.

Gemini Live

A realtime model running on the Gemini Live API. It hears the user and answers with its own voice, in place of a speech to text, an agent and a text to speech.

Usage with MicdropServer

import { GeminiLive } from '@micdrop/gemini'
import { MicdropServer } from '@micdrop/server'

const realtime = new GeminiLive({
  apiKey: process.env.GEMINI_API_KEY || '',
  model: 'gemini-3.8-live', // Default model
  systemPrompt: 'You are a helpful assistant',
  voice: 'Kore', // Optional, a prebuilt voice

  // Advanced features (optional)
  autoEndCall: true, // Automatically end call when user requests
})

new MicdropServer(socket, {
  realtime,
  generateFirstMessage: true,
})

The Micdrop client detects when the user speaks, so the automatic activity detection of Gemini is turned off. Each turn is sent between an activityStart and an activityEnd, and the model answers once the turn ends.

Tools

Tools are added as with any agent, and run on your server:

import { z } from 'zod'

realtime.addTool({
  name: 'get_weather',
  description: 'Get the current weather in a city',
  inputSchema: z.object({ city: z.string() }),
  execute: async ({ city }) => fetchWeather(city),
})

Gemini ends its turn when it calls a tool, then speaks about the result once it has it. The client keeps waiting for the answer in between. A tool declared with skipAnswer gives its result to the model silently.

Gemini reads the tools when the session opens, so add them right after creating the model, before the connection is established.

Events

GeminiLive emits the events of an agent, and two of its own.

Options

gemini-3.8-live-extended-thinking reasons before answering, for questions that take several steps. It refuses to start without a thinkingLevel, such as 'low', while gemini-3.8-live refuses any.

Long calls

A Gemini connection lasts a few minutes. GeminiLive keeps the handle Gemini sends to resume the session, and moves to a new connection at the first silence once Gemini announces the end of the current one. Context window compression is on, which lifts the fifteen minutes limit of an audio session.