Qwen3-TTS in Node.js and TypeScript

@micdrop/qwen-tts runs Qwen3-TTS locally from Node: a TypeScript class that sends each sentence to an mlx-audio server holding the Qwen3-TTS weights and streams the speech back. It needs no API key, and the audio stays on your machine. Use it as the voice of a @micdrop/server call, or on its own from any Node script.

Qwen3-TTS is an open source text to speech model from Alibaba. It is the only local voice in Micdrop that speaks ten languages with one checkpoint: Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian and Spanish. It accepts an instruction such as "Very happy" or "speak slowly" that changes how a sentence is read, can design a voice from a description, and can clone a voice from three seconds of reference audio.

It also needs more computation than the other local voices, because it generates audio with a language model. It needs a GPU and a server running next to your app. On a machine without a GPU, use Piper or Kokoro.

Installation

Install the package:

npm install @micdrop/qwen-tts

Start an mlx-audio server, on Python 3.10 or above:

uv run --with "mlx-audio[server]" python -m mlx_audio.server --port 8000

pip install "mlx-audio[server]" in a virtual environment does the same. The server downloads the checkpoint on its first request, 2.6 GB for the default one, and holds it in memory afterwards.

Usage with MicdropServer

import { MicdropServer } from '@micdrop/server'
import { Qwen3TTS } from '@micdrop/qwen-tts'

const tts = new Qwen3TTS({
  url: 'http://localhost:8000',
  voice: 'Ryan',
  language: 'en-US',
})

// Use with MicdropServer
new MicdropServer(socket, {
  tts,
  // ... other options
})

Usage without MicdropServer

import { Qwen3TTS } from '@micdrop/qwen-tts'
import { Readable } from 'stream'

const tts = new Qwen3TTS({ voice: 'Ryan' })

// Audio is raw PCM, 16 bits, 16 kHz, mono
tts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))
tts.on('Failed', (texts) => console.error('Failed:', texts))

tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))

Events

See the TTS interface for the full contract.

Options

Voices

The CustomVoice checkpoints include nine speakers, so changing the voice costs nothing at runtime. Each speaker was recorded in one language and can read all ten. language sets the language of the call, whatever the speaker.

language accepts a locale such as fr-FR, a code such as fr, or the checkpoint name french. resolveLanguage() returns the value sent to the model. For a language outside the ten, it returns auto, and the model detects the language from the text.

QWEN_SPEAKERS exports this list with the recording language of each voice. The demo uses it to fill its voice select.

A voice recorded in Chinese reads French as accurately as an English one. In a test of nine French sentences across four speakers, Whisper transcribed eight word for word, and one had "quatorze" mispronounced. Expect an occasional mispronounced word with any speaker.

Reading with an intention

instruct describes how to read each sentence:

new Qwen3TTS({ voice: 'Serena', instruct: 'Warm and reassuring, speak slowly' })

The instruction applies to every sentence of the call, so use it for the overall style of the assistant. It changes how the speaker talks and keeps the same voice.

Checkpoints

model can be any Qwen3-TTS export the server can load. The server keeps each checkpoint it has loaded in memory.

The 0.6B checkpoint is smaller and returns its first audio about 100 ms sooner. But it starts each sentence with half a second of silence, against a tenth of a second for the 1.7B, and it more often produces audio much longer than the sentence. The 8 bit export of the 1.7B is slower than real time on a MacBook Pro M2, so prefer the 6 bit one.

Inventing a voice

A VoiceDesign checkpoint has no preset speakers. It generates the voice from the description in instruct.

new Qwen3TTS({
  model: 'mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-6bit',
  instruct: 'A calm British man in his fifties, low pitch, unhurried',
  language: 'en-US',
})

Cloning a voice

A Base checkpoint has no preset speakers either. It reads a sentence in the voice of a recording, given the recording and its transcript. The mlx-audio server opens the file, so the path must exist on the machine that runs the server.

new Qwen3TTS({
  model: 'mlx-community/Qwen3-TTS-12Hz-1.7B-Base-6bit',
  refAudio: '/srv/voices/marie.wav',
  refText: 'Bonjour, je suis Marie et je vous accompagne aujourd’hui.',
  language: 'fr-FR',
})

Clone a voice only with the consent of the person it belongs to.

How the audio arrives

The server decodes audio tokens as they are generated, so a sentence starts playing before it is fully generated. streamingInterval sets the length of each chunk, half a second by default. A smaller value gives an earlier first word, but adds a decoder pass per chunk.

Sentences are generated one after the other, so they stay in order and the GPU handles one sentence at a time. When the user interrupts, the package closes the connection and the server stops generating.

Latency and memory

The first audio of a sentence arrives after about 400 ms, measured on a MacBook Pro M2 with 24 GB of memory and the default checkpoint loaded. The rest of the sentence takes about 0.75 seconds of computation per second of speech, so playback never waits for generation. In the demo, with the assistant speaking first, the first audio reached the browser 1.1 to 1.3 seconds after the call opened.

The mlx-audio server holds 2.5 GB with the default checkpoint, outside the Node process. A full local call with Qwen3 4B as the agent and Whisper french for transcription holds about 6.5 GB and answers in about 1.6 seconds. It fits on a 16 GB Mac, with little memory left. You can compare its latency and memory with the other local stacks.

The 0.6B checkpoint answers about 100 ms sooner. With warmup on, the server loads the checkpoint at startup, so the first call starts without waiting for the load.

Another server with the same API

The package sends POST /v1/audio/speech with the OpenAI fields (model, input, voice) plus the Qwen ones (instruct, lang_code, ref_audio, ref_text), sets stream, and reads raw PCM at 24 kHz with response_format: "pcm", then resamples it to the 16 kHz the browser client plays. Any server that implements this route can replace mlx-audio, for instance one running the weights on an NVIDIA card.

FAQ

Documentation

You can run a whole voice call on local models, with Qwen3-TTS next to a local LLM and local speech to text, and compare Qwen3-TTS with the other local voices by language, setup and latency.

License

Qwen3-TTS is released under the Apache 2.0 license, including the weights, so it can be used commercially. Cloning a person’s voice still requires their consent.