🎤Micdrop

OpenAI Decisions API Classifier in Node.js

OpenaiClassifier is a classifier running the Decisions API of OpenAI. Where the Responses API writes an answer token by token, Decisions answers a set of typed questions at once: the probability that a condition is true, a choice out of several values, or a score on ordered levels. OpenAI announces it about ten times faster than the Responses API, which leaves time to route the answer before the LLM starts.

The Decisions API is in public beta, and gpt-6-luna is its only model. For a complete example, route a support call with the Decisions API, refusals and late requests included.

Installation

Terminal window
npm install @micdrop/openai

The same package also provides an agent, a speech to text, a text to speech and the Realtime API. The providers page shows what every provider covers.

Usage with MicdropServer

import { choice, OpenaiClassifier, predicate, score } from '@micdrop/openai'
import { MicdropServer } from '@micdrop/server'
const classifier = new OpenaiClassifier({
apiKey: process.env.OPENAI_API_KEY || '',
questions: {
intent: choice(
'What does the user in `turn` want? Read `history` for context.',
{
billing: 'A charge, an invoice, a refund',
outage: 'The service is down or slow',
cancel: 'Cancel the subscription or switch provider',
other: 'Anything else',
}
),
frustration: score('How frustrated is the user in `turn`?', [
'Calm',
'Annoyed',
'Angry',
]),
wantsHuman: predicate('Does the user in `turn` ask for a human?'),
},
})
new MicdropServer(socket, {
stt,
agent,
tts,
classifier,
classifierOptions: { sendToClient: true, waitBeforeAnswer: true },
})

The server classifies each turn of the user when it ends, and sends it the MicdropTurnInput as JSON, with the turn and its history. The questions point at these fields by name, between backticks. See Classifier for the server options and routing the answer in onBeforeAnswer.

Usage without MicdropServer

import { OpenaiClassifier, predicate } from '@micdrop/openai'
const classifier = new OpenaiClassifier({
apiKey: process.env.OPENAI_API_KEY || '',
questions: { urgent: predicate('Does this message sound urgent?') },
})
const classification = await classifier.classify('My internet is down!')
console.log(classification?.result.answers.urgent?.probability)

Questions

Three helpers build the questions. The key of each question names its answer.

HelperAnswersExample
predicate(instructions)The probability that the condition is trueAsks for a human, wants to cancel
choice(instructions, choices)One value out of several, without orderThe intent, the topic, the language
score(instructions, levels)A position on ordered levels, from the lowestFrustration, urgency, severity

choice() takes an object of values, each with a description of when it applies, or null. Add a fallback such as other when the values do not cover every turn. score() takes the levels from the lowest up, each a label or a { label, description }.

Every question goes in the same request. To pick them per input, pass a function of the input as questions.

Reading the answers

The answers are typed from your questions:

const { answers, refusals, usage } = getTurnClassification(
this.conversation
)!.result
answers.intent?.choice // 'billing', typed as one of your values
answers.intent?.confidence // 0.93
answers.intent?.probabilities // { billing: 0.95, outage: 0.01, cancel: 0.02, other: 0.02 }
answers.frustration?.score // 1.1, the weighted average of the level indices
answers.frustration?.probabilities // { 0: 0.1, 1: 0.7, 2: 0.2 }
answers.wantsHuman?.probability // 0.07
refusals // ['wantsHuman'] when the model declined that question
usage.input_tokens // 412

The model can refuse a question. Its answer is then missing from answers and its name is listed in refusals, hence the ?. above. Level indices start at 0, and a score can fall between two levels. Set your thresholds on labeled turns of your own application.

Options

OptionTypeDefaultDescription
apiKeystringRequiredYour OpenAI API key, or pass an openai client instead
modelstring'gpt-6-luna'Model answering the questions
questionsQuestions | (input) => QuestionsRequiredQuestions built with predicate(), choice() and score(), or a function of them
input(input) => string | DecisionInputMessage[]The input as JSONBuilds what the model reads, a text or user messages with text and images
safetyIdentifierstringundefinedIdentifies the end user to OpenAI, for abuse detection
timeoutnumber3000Timeout of each request in ms

Latency and price

Measured on the real API with a turn, its history and three questions, each request took about 200 to 530 ms. The first request of a process takes a few seconds more, while it opens the connection.

Input costs $0.10 per million tokens, and output is free. A request of a thousand tokens costs about $0.0001, so classifying every turn of a call stays well under a cent.