Client (React Native)

@micdrop/react-native runs a Micdrop call on a phone. It records the microphone, plays the assistant voice, detects who is speaking and streams both ways over a WebSocket, against the same server a browser talks to.

The server holds the transcription, the agent and the voice, so no speech model ships in the app. Speech to text in React Native compares server-side transcription with the iOS and Android recognisers and with running Whisper on the device.

Installation

npm install @micdrop/react-native @micdrop/react react-native-audio-api

Audio goes through native code, so the app runs from a development build rather than from Expo Go. See Installation for the permissions and the config plugin.

Quick start

import { useMicdropState } from '@micdrop/react'
import { Micdrop } from '@micdrop/react-native'
import { Button, Text, View } from 'react-native'

export default function Call() {
  const state = useMicdropState()

  const handlePress = () =>
    state.isStarted
      ? Micdrop.stop()
      : Micdrop.start({ url: 'wss://example.com/call' })

  return (
    <View>
      <Text>{state.isUserSpeaking ? 'Listening' : 'Your turn'}</Text>
      <Button
        title={state.isStarted ? 'Hang up' : 'Start the call'}
        onPress={handlePress}
      />
    </View>
  )
}

Micdrop.start() asks for the microphone permission, configures the audio session for a call, opens the connection and starts listening. The rest of the app only reads the state.

What you get

• Microphone capture at 16 kHz, and playback of the answer as it streams in
• Voice activity detection, on volume or with the Silero model, so the user just talks
• Interruption: the assistant stops the moment the user starts speaking
• Mute, pause, and a choice between the loudspeaker and the earpiece
• Reconnection when the network drops
• React hooks for the whole call state

Same API as the browser

Running a call is the very same code as in a browser, @micdrop/client, which this package and @micdrop/web both build on. The pages of the browser section apply here as they are written, only the import changes:

• Start and stop a call, mute and pause
• Call state and the React hooks that read it
• The conversation, tool calls and error handling
• Voice activity detection and turn detection, options and events included
• Reducing latency

What the hardware underneath changes:

• Output is a route rather than a device, see Audio output and devices
• The audio session is configured as a phone call, which is what turns on echo cancellation
• Silero and the turn detector run on the native ONNX runtime, which takes a patch, see Installation
• Settings are remembered only once the app installs a store, see Voice activity detection

Example

The React Native example is a complete screen: conversation, level meters, call controls, and the Node server it talks to.