Speaker

Speaker is the speaker, as a single object shared by the whole app. It plays the 16 kHz PCM the server sends, as it arrives, and manages the output.

Playback is done by a driver: the Web Audio API in a browser, the native audio engine on a phone. Importing @micdrop/web or @micdrop/react-native installs the right one.

Usage Example

import { Speaker } from '@micdrop/web'

// Play the audio a call sends, PCM16 at 16 kHz
const response = await fetch('speech.pcm')
Speaker.playAudio(await response.arrayBuffer())

// Move the sound to another output
await Speaker.changeDevice(deviceId)

// The level of what is being heard, in decibels
Speaker.on('Volume', (volume: number) => {
  console.log('Current level:', volume)
})

Speaker.on('StartPlaying', () => console.log('The assistant has the floor'))
Speaker.on('StopPlaying', () => console.log('It is quiet again'))

// Drop everything queued and stop at once
Speaker.stopAudio()

Chunks arrive faster or slower than they are heard, so each one is scheduled right after the previous one rather than played on arrival. That is what keeps a sentence from stuttering when the network hiccups.

API Reference

Methods

playAudio(pcm: Int16Array | ArrayBuffer, sampleRate?: number): void

Queues samples for playback, right after the ones already queued. Mono PCM16, at 16 kHz unless another rate is given.

stopAudio(): void

Drops everything queued and stops playing immediately. This is what an interruption calls.

changeDevice(deviceId: string): Promise<void>

Plays through another output, and remembers it for the next call. On a phone the outputs are the loudspeaker and the earpiece, see Audio output and devices.

getDevices(): Promise<MicdropDevice[]>

Lists the outputs the system offers.

setDriver(driver: SpeakerDriver): void

Plays with another library. See Using another audio library.

Properties

isPlaying: boolean

Whether the assistant is being heard right now. This is what state.isAssistantSpeaking reads.

deviceId: string | undefined

The output in use, when the system reports one.

volume: number

The last level measured, in decibels between -Infinity and 0.

Events

StartPlaying and StopPlaying

The assistant took or gave back the floor.

Volume: [number]

The level of what is being heard, about ten times a second. It follows playback rather than arrival, so a burst of chunks does not light a meter up all at once.