🎤Micdrop

Gemini Decisions API Classifier in Node.js

GeminiClassifier is a classifier running the Decisions API of Gemini. It answers a set of typed questions in a single forward pass, without writing a token: the probability that a condition is true, a choice out of several values, or a score on ordered levels. Adding questions barely changes the latency, up to 1,000 per request.

Gemini Decision 1 is its only model, on the v1beta Gemini Developer API. It asks the same three kinds of questions as Jev, with the same helpers and the same shape of answers, so code reading answers.wantsHuman.noul works with both.

Installation

Terminal window
npm install @micdrop/gemini

The same package also provides an agent, a speech to text, a text to speech and Gemini Live. The providers page shows what every provider covers.

Usage with MicdropServer

import { choice, GeminiClassifier, noul, score } from '@micdrop/gemini'
import { MicdropServer } from '@micdrop/server'
const classifier = new GeminiClassifier({
apiKey: process.env.GEMINI_API_KEY || '',
questions: {
intent: choice(
'What does the user in `turn` want? Read `history` for context.',
{
billing: 'A charge, an invoice, a refund',
outage: 'The service is down or slow',
cancel: 'Cancel the subscription or switch provider',
other: 'Anything else',
}
),
frustration: score('How frustrated is the user in `turn`?', [
'Calm',
'Annoyed',
'Angry',
]),
wantsHuman: noul('Does the user in `turn` ask for a human?'),
},
})
new MicdropServer(socket, {
stt,
agent,
tts,
classifier,
classifierOptions: { sendToClient: true, waitBeforeAnswer: true },
})

The server classifies each turn of the user when it ends, and sends it the MicdropTurnInput as JSON, with the turn and its history. The model never sees the keys of the questions, so each instruction holds the whole question and points at these fields by name, between backticks. See Classifier for the server options and routing the answer in onBeforeAnswer.

Usage without MicdropServer

import { GeminiClassifier, noul } from '@micdrop/gemini'
const classifier = new GeminiClassifier({
apiKey: process.env.GEMINI_API_KEY || '',
questions: { urgent: noul('Does this message sound urgent?') },
})
const classification = await classifier.classify('My internet is down!')
console.log(classification?.result.answers.urgent.noul)

Questions

Three helpers build the questions. The key of each question names its answer.

HelperAnswersExample
noul(instructions, criteria?)The probability that the condition is trueAsks for a human, wants to cancel
choice(instructions, criteria)One value out of several, without orderThe intent, the topic, the language
score(instructions, levels)A position on ordered levels, from the lowestFrustration, urgency, severity

noul() can take { true, false }, a description of each outcome. choice() takes an object of 2 to 255 values, each with a short description of when it applies. Its probabilities always add up to 1, even when no value fits, so add a fallback such as other. score() takes the description of each level from the lowest up. Keep the levels evenly spaced, since the score averages their indices.

Every question goes in the same request. To pick them per input, pass a function of the input as questions.

Reading the answers

The answers are typed from your questions:

const { answers, usage } = getTurnClassification(this.conversation)!.result
answers.intent.choice // 'billing', typed as one of your values
answers.intent.confidence // 0.93
answers.intent.probabilities // { billing: 0.93, outage: 0.02, cancel: 0.03, other: 0.02 }
answers.frustration.score // 1.1, the weighted average of the level indices
answers.frustration.probabilities // { 0: 0.1, 1: 0.7, 2: 0.2 }
answers.wantsHuman.noul // 0.07
answers.wantsHuman.confidence // 0.93
usage?.total_input_tokens // 412

A noul probability stays between 0.01 and 0.99, so a threshold such as > 0.99 never matches. Set your thresholds on labeled turns of your own application, and send the turns with a low confidence to a fallback.

When the safety filters of Gemini block an input, the request fails and the classification resolves to undefined, as on a timeout.

Images, audio and video

The model reads text, images, audio, video and PDF. The input option builds what it reads from the input of the classifier, to add a picture the user sent for instance:

const classifier = new GeminiClassifier({
apiKey: process.env.GEMINI_API_KEY || '',
questions: { damaged: noul('Does the photo show a damaged item?') },
input: (input) => [
{ type: 'text', text: JSON.stringify(input) },
{ type: 'image', uri: lastPhotoUrl, mime_type: 'image/jpeg' },
],
})

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour Gemini API key, or pass a genai client instead
modelstring'gemini-decision-1'Model answering the questions
questionsQuestions | (input) => QuestionsRequiredQuestions built with noul(), choice() and score(), or a function of them
input(input) => string | Content[]The input as JSONBuilds what the model reads, a text or blocks of text, image, audio, video, PDF
timeoutnumber3000Timeout of each request in ms

Price

Input costs $0.10 per million tokens, and output is free. A request of a thousand tokens costs about $0.0001, so classifying every turn of a call stays well under a cent.