Installation

Packages

npm install @micdrop/react-native react-native-audio-api

Audio is captured and played by react-native-audio-api, an implementation of the Web Audio API for React Native. It ships a player UI that pulls in three more packages, so install those as well:

npx expo install react-native-worklets react-native-gesture-handler react-native-reanimated

For the React hooks:

npm install @micdrop/react

The Silero voice detection and turn detection both run their model on onnxruntime-react-native, which takes a patch before it links. The volume detection needs nothing.

Native code is involved, so the app runs from a development build. Expo Go does not include these libraries.

The ONNX runtime

Skip this section unless you want Silero or turn detection. Version 1.24.3 does not link on Expo SDK 57 as it ships, so the install takes a patch. Four steps with pnpm.

1. Install the runtime.

npx expo install onnxruntime-react-native

2. Take the patch, next to your root package.json.

mkdir -p patches
curl -o "patches/onnxruntime-react-native@1.24.3.patch" \
  "https://raw.githubusercontent.com/Godefroy/micdrop/main/patches/onnxruntime-react-native@1.24.3.patch"

3. Name it in the root package.json of the workspace.

{
  "pnpm": {
    "patchedDependencies": {
      "onnxruntime-react-native@1.24.3": "patches/onnxruntime-react-native@1.24.3.patch"
    }
  }
}

4. Install and rebuild. Metro alone does not pick up a change to a native module.

pnpm install
npx expo run:android
# or
npx expo run:ios

The React Native example runs on exactly this setup.

With npm or Yarn

patch-package writes the paths inside a patch differently, so the file above will not serve. Make the two edits below in node_modules/onnxruntime-react-native, then record them:

npm install --save-dev patch-package
npx patch-package onnxruntime-react-native

Add "postinstall": "patch-package" to your scripts to replay it. Yarn Berry has the flow built in, as yarn patch and yarn patch-commit -s.

What the patch changes

Two edits, both inside the package, which is why they need a patch rather than a setting of your own. Redo them by hand for a newer version of the runtime.

Delete unimodule.json at the root of the package. Expo autolinking reads that file as the mark of an Expo module and leaves the package out of the React Native side, so OnnxruntimePackage is never registered and the first import throws Cannot read property 'install' of null. Without the file, autolinking treats it as the plain React Native module it is.

Rewrite line 250 of android/build.gradle, since VersionNumber is a class Gradle 9 no longer exposes. The minor version it needs is already computed a few lines above.

// Before
if (VersionNumber.parse(REACT_NATIVE_VERSION) < VersionNumber.parse("0.71")) {

// After
if (REACT_NATIVE_MINOR_VERSION < 71) {

Adding the package to plugins in app.json looks like the answer and is not. Its config plugin adds the Gradle project and the pod, neither of which registers the native module. Bare React Native never meets the first edit, since the community autolinking ignores unimodule.json.

Expo

Add the config plugin to app.json. It writes the microphone permission and the audio service into the native projects:

{
  "expo": {
    "plugins": [
      [
        "react-native-audio-api",
        {
          "iosMicrophonePermission": "We need the microphone so you can talk with the assistant.",
          "androidPermissions": [
            "android.permission.RECORD_AUDIO",
            "android.permission.MODIFY_AUDIO_SETTINGS",
            "android.permission.FOREGROUND_SERVICE",
            "android.permission.FOREGROUND_SERVICE_MICROPHONE",
            "android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK"
          ],
          "androidFSTypes": ["microphone", "mediaPlayback"]
        }
      ]
    ]
  }
}

Then build once:

npx expo run:ios
# or
npx expo run:android

Later runs only need npx expo start --dev-client.

Bare React Native

Add the microphone permission by hand.

ios/<App>/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>We need the microphone so you can talk with the assistant.</string>
<key>UIBackgroundModes</key>
<array>
  <string>audio</string>
</array>

android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

Then run pod install in ios/.

The permission itself is requested by Micdrop.start(), there is nothing to call beforehand.

Targeting phones only

Expo offers a web target unless you say otherwise, and bundling for it needs react-native-web. An app meant for phones says so in app.json, which keeps the dev server from failing on Unable to resolve react-native-web:

{
  "expo": {
    "platforms": ["ios", "android"]
  }
}

Reaching the server

A phone does not share localhost with your machine, so a development server has to be reached by its address on the network. With Expo, the app already knows where it was loaded from:

import Constants from 'expo-constants'
import { Platform } from 'react-native'

let host = Constants.expoConfig?.hostUri?.split(':')[0] ?? 'localhost'

// An Android emulator has its own loopback, so `localhost` there is the
// emulator rather than the machine running the server
if (Platform.OS === 'android' && ['localhost', '127.0.0.1'].includes(host)) {
  host = '10.0.2.2'
}

export const serverUrl = `ws://${host}:8087`

Have the server listen on every interface (host: '0.0.0.0') so it accepts that connection.

An iOS simulator shares the machine’s network, and a phone on the same Wi-Fi gets the address Expo advertised, so both work without the special case. A phone plugged in over USB reaches the machine through adb reverse tcp:8087 tcp:8087.

In production, use wss:// and let the app talk to your deployed server.

Monorepo

Metro has to see the packages outside the app folder. In metro.config.js:

const path = require('path')
const { getDefaultConfig } = require('expo/metro-config')

const projectRoot = __dirname
const monorepoRoot = path.resolve(projectRoot, '../..')

const config = getDefaultConfig(projectRoot)
config.watchFolders = [monorepoRoot]
config.resolver.nodeModulesPaths = [
  path.resolve(projectRoot, 'node_modules'),
  path.resolve(monorepoRoot, 'node_modules'),
]

module.exports = config

With pnpm, leave hierarchical lookup on: each package finds its own dependencies through the store next to it.