Qwen3-TTS

Local text to speech for @micdrop/server, generated by an mlx-audio server holding the Qwen3-TTS weights.

Qwen3-TTS is an open source text to speech model from Alibaba. It is the only local voice in Micdrop that speaks ten languages with one checkpoint: Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian and Spanish. It accepts an instruction such as "Very happy" or "speak slowly" that changes how a sentence is read, can design a voice from a description, and can clone a voice from three seconds of reference audio.

It also needs more computation than the other local voices, because it generates audio with a language model. It needs a GPU and a server running next to your app. On a machine without a GPU, use Piper or Kokoro.

Installation

Install the package:

npm install @micdrop/qwen-tts

Start an mlx-audio server, on Python 3.10 or above:

uv run --with "mlx-audio[server]" python -m mlx_audio.server --port 8000

pip install "mlx-audio[server]" in a virtual environment does the same. The server downloads the checkpoint on its first request, 2.6 GB for the default one, and holds it in memory afterwards.

Usage with MicdropServer

import { MicdropServer } from '@micdrop/server'
import { Qwen3TTS } from '@micdrop/qwen-tts'

const tts = new Qwen3TTS({
  url: 'http://localhost:8000',
  voice: 'Ryan',
  language: 'en-US',
})

// Use with MicdropServer
new MicdropServer(socket, {
  tts,
  // ... other options
})

Usage without MicdropServer

import { Qwen3TTS } from '@micdrop/qwen-tts'
import { Readable } from 'stream'

const tts = new Qwen3TTS({ voice: 'Ryan' })

// Audio is raw PCM, 16 bits, 16 kHz, mono
tts.on('Audio', (chunk) => console.log('Audio:', chunk.length, 'bytes'))
tts.on('Failed', (texts) => console.error('Failed:', texts))

tts.speak(Readable.from(['Hello! ', 'What can I do for you?']))

Events

See the TTS interface for the full contract.

Options

Voices

The CustomVoice checkpoints include nine speakers, so changing the voice costs nothing at runtime. Each speaker was recorded in one language and can read all ten. The language of the call is set by language, not by the speaker.

language accepts a locale such as fr-FR, a code such as fr, or the checkpoint name french. resolveLanguage() returns the value sent to the model. For a language outside the ten, it returns auto, and the model detects the language from the text.

QWEN_SPEAKERS exports this list with the recording language of each voice. The demo uses it to fill its voice select.

A voice recorded in Chinese reads French as accurately as an English one. In a test of nine French sentences across four speakers, Whisper transcribed eight word for word, and one had "quatorze" mispronounced. Expect an occasional mispronounced word with any speaker.

Reading with an intention

instruct describes how to read each sentence:

new Qwen3TTS({ voice: 'Serena', instruct: 'Warm and reassuring, speak slowly' })

The instruction applies to every sentence of the call, so it describes the overall style of the assistant, not one answer. It changes how the speaker talks, not which speaker it is.

Checkpoints

model can be any Qwen3-TTS export the server can load. The server keeps each checkpoint it has loaded in memory.

The 0.6B checkpoint is smaller and returns its first audio about 100 ms sooner. But it starts each sentence with half a second of silence, against a tenth of a second for the 1.7B, and it more often produces audio much longer than the sentence. The 8 bit export of the 1.7B is slower than real time on a MacBook Pro M2, so prefer the 6 bit one.

Inventing a voice

A VoiceDesign checkpoint has no preset speakers. It generates the voice from the description in instruct.

new Qwen3TTS({
  model: 'mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-6bit',
  instruct: 'A calm British man in his fifties, low pitch, unhurried',
  language: 'en-US',
})

Cloning a voice

A Base checkpoint has no preset speakers either. It reads a sentence in the voice of a recording, given the recording and its transcript. The mlx-audio server opens the file, so the path must exist on the machine that runs the server.

new Qwen3TTS({
  model: 'mlx-community/Qwen3-TTS-12Hz-1.7B-Base-6bit',
  refAudio: '/srv/voices/marie.wav',
  refText: 'Bonjour, je suis Marie et je vous accompagne aujourd’hui.',
  language: 'fr-FR',
})

Clone a voice only with the consent of the person it belongs to.

How the audio arrives

The server decodes audio tokens as they are generated, so a sentence starts playing before it is fully generated. streamingInterval sets the length of each chunk, half a second by default. A smaller value gives an earlier first word, but adds a decoder pass per chunk.

Sentences are generated one after the other, so they stay in order and the GPU handles one sentence at a time. When the user interrupts, the package closes the connection and the server stops generating.

Another server

The package sends POST /v1/audio/speech with the OpenAI fields (model, input, voice) plus the Qwen ones (instruct, lang_code, ref_audio, ref_text), sets stream, and reads raw PCM at 24 kHz with response_format: "pcm". Any server that implements this route can replace mlx-audio.

Documentation

Read the guide on running Micdrop with local models to combine Qwen3-TTS with a local LLM and local speech to text, and the local TTS page to compare it with the other three.

License

Qwen3-TTS is released under the Apache 2.0 license, including the weights, so it can be used commercially. Cloning a person’s voice still requires their consent.