Kyutai Pocket TTS in Node.js, with Voice Cloning
Pocket TTS is a 100 million parameter text to speech model from
Kyutai, a nonprofit AI lab in Paris. It runs
on the CPU and clones a voice from a few seconds of reference audio. Kyutai
publishes it as a Python package in its
GitHub repository, under the MIT
license. @micdrop/pocket-tts runs the same model in the Node process of
@micdrop/server, through the
sherpa-onnx addon and
ONNX Runtime, with no Python to install.
The model generates about five times faster than real time and emits its audio while it is still writing the rest. On a MacBook Pro M2, its first sound arrives 250 to 430 ms after it receives a sentence. On the same machine, Piper takes about 390 ms and Kokoro 900 to 1300 ms, all three measured side by side from Node. Phonon, which Gradium built on Pocket TTS, starts within 100 ms and speaks five languages, and Pocket TTS keeps the cloning of a voice from a wav file.
Installation
Install the package:
npm install @micdrop/pocket-ttsDownload the English model, which the sherpa-onnx project converted to ONNX from Kyutai’s PyTorch weights:
curl -LO https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-pocket-tts-int8-2026-01-26.tar.bz2tar xf sherpa-onnx-pocket-tts-int8-2026-01-26.tar.bz2rm sherpa-onnx-pocket-tts-int8-2026-01-26.tar.bz2The archive weighs 98 MB and takes 190 MB on disk. A full precision archive,
sherpa-onnx-pocket-tts-2026-01-26.tar.bz2, sits next to it and works the same
way, since Micdrop looks the weights up by name in the folder and takes the
quantized ones first when both files are there.
The addon ships prebuilt binaries for macOS, Linux and Windows on both Intel
and ARM, so npm install is all it takes.
Voice cloning
A few seconds of someone speaking is enough for the model to answer in that
voice, with no transcript of the sample and no training step. The archive ships
three such recordings in its test_wavs folder, reachable by name through
BUNDLED_VOICES.
bria is forty seconds of a woman reading calmly. loona is a one second clip,
about the shortest a reference can be. frenchAccent is a French speaker, so
the recording gives the timbre and the accent while the words come from the text
and stay English.
To clone another voice, pass the path of a wav file instead:
import { BUNDLED_VOICES, PocketTTS } from '@micdrop/pocket-tts'
const tts = new PocketTTS({ modelDir: './sherpa-onnx-pocket-tts-int8-2026-01-26', voice: './voices/my-voice.wav',})Pocket TTS reads only the first referenceLength seconds, so a long recording
costs nothing extra. The model reproduces the quality of the sample along with
the voice, which makes a clean recording worth more than a long one.
Clone a voice only with the consent of the person it belongs to. The Kyutai license makes it a condition of use, and impersonation is the main misuse of voice cloning.
Usage with MicdropServer
import { PocketTTS } from '@micdrop/pocket-tts'import { MicdropServer } from '@micdrop/server'
const tts = new PocketTTS({ modelDir: './sherpa-onnx-pocket-tts-int8-2026-01-26', voice: 'bria',})
// Use with MicdropServernew MicdropServer(socket, { tts, // ... other options})Usage without MicdropServer
import { PocketTTS } from '@micdrop/pocket-tts'import { Readable } from 'stream'
const tts = new PocketTTS({ modelDir: './sherpa-onnx-pocket-tts-int8-2026-01-26', voice: 'bria',})
// Audio is raw PCM, 16 bits, 16 kHz, monotts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))tts.on('Failed', (texts) => console.error('Failed:', texts))
tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))Events
| Event | Payload | Description |
|---|---|---|
Audio | Buffer | A chunk of audio, PCM 16 bits, 16 kHz, mono, ready to be played. |
Failed | string[] | Synthesis gave up after its retries, with the text that stayed unspoken. |
See the TTS interface for the full contract.
Options
| Option | Type | Default | Description |
|---|---|---|---|
modelDir | string | Required | Folder you extracted the archive into |
voice | string | 'bria' | Bundled voice name, or path to a wav to clone |
speed | number | 1 | Speech rate, above 1 to speak faster |
numSteps | number | 2 | Sampling steps per frame |
referenceLength | number | 12 | Seconds of the reference the model builds from |
seed | number | Optional | Sampling seed, to make a sentence reproducible |
numThreads | number | 2 | Threads given to ONNX Runtime |
provider | string | 'cpu' | Execution provider passed to ONNX Runtime |
warmup | boolean | true | Speaks one word at startup |
Languages
Pocket TTS speaks English in this integration. Kyutai’s Python package also covers French, German, Spanish, Italian, Portuguese and Dutch, with one checkpoint per language. The sherpa-onnx project publishes only the English checkpoint in ONNX, so that is the one the addon loads. Hand it French and you get fluent nonsense rather than an accent, since it reads every word with English phonemes. For French and the other languages in Node, use Phonon, built on Pocket TTS, or Piper.
Latency
These times come from a MacBook Pro M2, on CPU. The generation timings assume the model is already loaded.
| Step | Time |
|---|---|
| Loading the model | ~470 ms |
| First audio of a sentence | 250 to 430 ms |
| Generating four seconds of speech | ~850 ms |
Micdrop cuts the answer into sentences and hands each sentence over in chunks as the model writes it, so the user hears its opening words while the model is still generating its end. The generation stops between two chunks rather than at the end of the sentence, which makes an interruption immediate.
The model runs inside your Node server
Kyutai’s package starts its own HTTP server with pocket-tts serve, a Python
process with PyTorch that listens on port 8000 and keeps the model in memory
between requests. With Micdrop, the model loads inside the Node server that
already handles your calls, so each sentence goes from the agent to the model
without a network hop.
Memory
The model takes about 350 MB once loaded and around 600 MB while generating. That is more than Kokoro at 80 MB, and it stays small next to the language model running beside it. Micdrop loads one instance per configuration and shares it across every call, so a second conversation costs nothing more.
Documentation
Read the guide on running Micdrop with local models to see how Pocket TTS fits with a local LLM and local speech to text.
License
Pocket TTS is released under the MIT license, and the sherpa-onnx addon under Apache 2.0. The reference recordings shipped with the archive carry their own licenses, listed on the Kyutai voice repository.