🎤Micdrop

Mistral Agent, Voxtral STT and TTS in Node.js

Mistral AI implementation for @micdrop/server.

This package provides an AI agent on Mistral AI’s API, real-time speech to text with Voxtral Mini Transcribe Realtime 3, and text to speech with Voxtral Speech 3. The agent, the transcription and the voice can all run on one Mistral account.

Mistral guarantees inference in the EU only on its European endpoint, api.eu.mistral.ai, billed 10% more. MistralTTS reaches it through its baseUrl option. MistralAgent and MistralSTT call the global endpoint, api.mistral.ai, where Mistral makes no commitment on where requests run. To keep the agent in Europe, use @micdrop/ai-sdk with a Mistral model set to api.eu.mistral.ai, as in this sovereign voice AI setup.

Installation

Terminal window
npm install @micdrop/mistral

Mistral Agent

Usage with MicdropServer

import { MistralAgent } from '@micdrop/mistral'
import { MicdropServer } from '@micdrop/server'
const agent = new MistralAgent({
apiKey: process.env.MISTRAL_API_KEY || '',
model: 'mistral-large-latest',
systemPrompt: 'You are a helpful assistant',
})
// Use with MicdropServer
new MicdropServer(socket, {
agent,
// ... other options
})

Usage without MicdropServer

import { MistralAgent } from '@micdrop/mistral'
const agent = new MistralAgent({
apiKey: process.env.MISTRAL_API_KEY || '',
systemPrompt: 'You are a helpful assistant',
})
agent.on('Message', (message) => console.log('Message:', message))
agent.addUserMessage('Hello, what can you do?')
// The answer is a text stream, written as the model generates it
agent.answer().on('data', (chunk) => process.stdout.write(chunk))

Events

EventPayloadDescription
MessageMicdropConversationItemA message, a tool call or a tool result was added to the conversation.
ToolCallMicdropToolCallA tool declared with emitOutput ran, with its parameters and output.
CancelLastUserMessagenoneThe last user message was dropped because it carried no intent.
SkipAnswernoneThe agent stays silent and waits for the user to finish their sentence.
EndCallnoneThe agent decided that the call is over.
FailednoneThe agent gave up generating an answer after its retries.

See the Agent interface for the full contract.

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour Mistral AI API key
modelstring'mistral-large-latest'Mistral AI model to use
systemPromptstringRequiredSystem prompt for the agent
maxRetrynumber3Maximum number of retries on API failures
retryDelaynumber500Delay in milliseconds between retries
maxStepsnumber5Maximum number of steps (for tool calls)
autoEndCallboolean | stringfalseAuto-detect when user wants to end call
autoSemanticTurnboolean | stringfalseHandle incomplete user sentences
autoIgnoreUserNoiseboolean | stringfalseFilter meaningless user sounds
extractExtractJsonOptions | ExtractTagOptionsundefinedExtract structured data from responses
onBeforeAnswerfunctionundefinedHook called before answer generation, returns true to skip it or a text to answer instead
settingsobject{}Additional Mistral AI API parameters

Cancellation and Retries

When the user interrupts, the request to Mistral is aborted and the unfinished answer stays out of the conversation. A request that fails before writing anything is retried up to maxRetry times, retryDelay apart, unless the answer was cancelled meanwhile. A request cut off after part of the answer reached the stream ends there, and the part already spoken is the answer recorded, so no sentence is ever spoken twice.

Available Models

Mistral AI offers several models you can choose from:

  • ministral-8b-latest - Fast and efficient 8B parameter model (default)
  • mistral-large-latest - Most capable model for complex tasks
  • mistral-small-latest - Balanced performance and cost
  • codestral-latest - Specialized for code generation

Settings Object

The settings parameter accepts any additional options from the Mistral AI Chat Completions API:

const agent = new MistralAgent({
apiKey: process.env.MISTRAL_API_KEY || '',
systemPrompt: 'You are a helpful assistant',
settings: {
temperature: 0.7, // Controls randomness (0-1)
max_tokens: 1000, // Maximum tokens in response
top_p: 0.9, // Nucleus sampling parameter
random_seed: 42, // For reproducible outputs
safe_prompt: false, // Enable/disable safety filtering
},
})

Advanced Features

The Mistral Agent supports advanced features for improved conversation handling:

Mistral STT (Speech-to-Text)

Real-time transcription with Voxtral Mini Transcribe Realtime 3, in more than 40 languages, with a final transcript in under half a second. It costs $0.008 per minute. MistralSTT connects to Mistral’s global endpoint, so pick Gladia or Gradium when the audio has to stay in the EU.

Voxtral also runs on its own. With stt as the server’s only option, you get a dictation tool that transcribes each sentence and stays quiet.

Usage with MicdropServer

import { MistralSTT } from '@micdrop/mistral'
import { MicdropServer } from '@micdrop/server'
const stt = new MistralSTT({
apiKey: process.env.MISTRAL_API_KEY || '',
})
// Use with MicdropServer
new MicdropServer(socket, {
stt,
// ... other options
})

Usage without MicdropServer

import { MistralSTT } from '@micdrop/mistral'
import { createReadStream } from 'fs'
const stt = new MistralSTT({
apiKey: process.env.MISTRAL_API_KEY || '',
})
stt.on('Transcript', (text) => console.log('Transcript:', text))
stt.on('Failed', (chunks) => console.error('Failed:', chunks.length, 'chunks'))
// Audio is raw PCM, 16 bits, 16 kHz, mono
stt.transcribe(createReadStream('speech.pcm'))

Events

EventPayloadDescription
TranscriptstringTranscription of one utterance. The text is empty when nothing was recognized.
FailedBuffer[]Transcription gave up after its retries, with the audio chunks left pending.

See the STT interface for the full contract.

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour Mistral AI API key
modelstring'voxtral-mini-transcribe-realtime-3-0'Realtime transcription model to use
encodingMistralAudioEncoding'pcm_s16le'Audio encoding of the incoming stream
targetStreamingDelayMsnumberOptionalTarget streaming delay in milliseconds
connectionTimeoutnumber5000Timeout in milliseconds for WebSocket connection
transcriptionTimeoutnumber4000Timeout in milliseconds to wait for transcription
retryDelaynumber1000Delay in milliseconds between reconnection attempts
maxRetrynumber3Maximum number of reconnection attempts before failing

Each stream given to transcribe() is one utterance and gets exactly one Transcript event, sent when Mistral finalizes the utterance after its flush. With no answer within transcriptionTimeout, the utterance is emitted with the text received so far, and the late answer is discarded so it never lands on the next utterance.

A connection that drops, even a clean close by Mistral, is opened again within a budget of maxRetry attempts in a row, retryDelay apart, and the audio of every utterance still waiting for its transcript is sent again. Once the budget runs out, Failed hands that audio back, and the next utterance tries to connect again.

The Micdrop client streams 16kHz PCM16 mono audio, which matches Voxtral’s default input format, so no resampling is required.

The previous model, voxtral-mini-transcribe-realtime-2602, still works through the model option.

Mistral TTS (Text-to-Speech)

Voxtral Speech 3 reads the answers of your agent in more than 35 languages, with preset voices or a voice cloned from a few seconds of audio. The speech endpoint reads a whole text at once, so MistralTTS cuts the answer into sentences and sends the audio of each one as Mistral streams it. A sentence starts playing in about half a second and is generated several times faster than it is spoken, so the next one is ready before the current one ends.

Usage with MicdropServer

import { MistralTTS } from '@micdrop/mistral'
import { MicdropServer } from '@micdrop/server'
const tts = new MistralTTS({
apiKey: process.env.MISTRAL_API_KEY || '',
model: 'voxtral-mini-tts-3-0', // Default model
voiceId: 'voice_592', // Marie, French
})
new MicdropServer(socket, {
tts,
// ... other options
})

Usage without MicdropServer

import { MistralTTS } from '@micdrop/mistral'
import { Readable } from 'stream'
const tts = new MistralTTS({ apiKey: process.env.MISTRAL_API_KEY || '' })
// Audio is raw PCM, 16 bits, 16 kHz, mono
tts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))
tts.on('Failed', (texts) => console.error('Failed:', texts))
tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))

Events

EventPayloadDescription
AudioBufferA chunk of audio, PCM 16 bits, 16 kHz, mono, ready to be played.
Failedstring[]Synthesis gave up after an error, with the text that stayed unspoken.

See the TTS interface for the full contract.

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour Mistral AI API key
modelstring'voxtral-mini-tts-3-0'Speech model to use
voiceIdstring'voice_3150'Preset voice, or the id of a voice saved in your Mistral account
refAudiostringundefinedA few seconds of speech to clone, base64 encoded, used over voiceId
settingsRecord<string, unknown>undefinedExtra fields of the request body, such as phoneme_hints
baseUrlstring'https://api.mistral.ai'API address, such as https://api.eu.mistral.ai to stay in Europe

Voxtral Speech 3 costs $13 per million characters.

Voices

Each preset voice has a native language and reads the others too. The full list comes from GET /v1/audio/voices on the Mistral API.

VoiceNameLanguage
voice_3150Jamesen-US
voice_478Sheilaen-US
voice_3149Hugoen-GB
voice_590Harrieten-GB
voice_592MarieFrench
voice_3141CamilleFrench
voice_3137ManonFrench
voice_589LeaFrench
voice_481HeidiGerman
voice_483ElenaSpanish
voice_482LorenzoItalian
voice_480GabrielaPortuguese

Mistral also offers voices in Arabic, Catalan, Hindi, Japanese, Korean, Dutch, Polish, Russian, Chinese and more. The Voxtral 2 voices, such as gb_jane_neutral or fr_marie_happy, still work.

The text can carry emotion and pace tags such as [emotion=...], which Voxtral Speech 3 performs rather than reads out. Ask your agent to write them in its system prompt.