voice_audio_multi/docs/models/qwen3_tts.md

Qwen3 TTS

Qwen3 TTS includes Base voice cloning, VoiceDesign, and CustomVoice variants. For shared conversion guidance, see Qwen3 GGUF checkpoints.

Qwen3 TTS Base

Qwen3 TTS Base is the voice-clone TTS path. It requires reference audio and, by default, its transcript. To use reference audio without a transcript, pass --request-option x_vector_only_mode=true.

Field	Value
Family	qwen3_tts
Model directory	models/Qwen3-TTS-12Hz-1.7B-Base
Tasks	tts, clon
Modes	offline, streaming
Voice input	Reference WAV through --voice-ref
Transcript	--reference-text required unless x_vector_only_mode=true

audiocpp_cli --task tts --family qwen3_tts --model models/Qwen3-TTS-12Hz-1.7B-Base --backend cuda --text "Hello from Qwen3 TTS." --voice-ref assets/resources/b.wav --reference-text "Some call me nature. Others call me Mother Nature. I've been here for over 4.5 billion years. 22,500 times longer than you." --out out.wav

tts and clon enter the same Base synthesis implementation.

Common Options (use directly)

Option	Values	Default	Meaning
--text	text	required	Text to synthesize.
--voice-ref	WAV path	required	Voice clone reference audio.
--reference-text	text	required by default	Transcript for reference audio; optional with x_vector_only_mode=true.
--language	language code	empty string	Text language hint.
--text-chunk-size	integer chars	8192	Long-form text chunk size. Each chunk is voiced separately; with the session option qwen3_tts.stitch_decode=true the talker still runs per chunk but the codes of all chunks are voiced in one decoder pass.
--max-tokens	positive integer	checkpoint generation config (2048 if absent)	Maximum generated speech tokens per chunk.

Qwen3 TTS CustomVoice

Qwen3 CustomVoice uses speaker ids packaged with the model. Use --request-option voice_id=Vivian to select one; legacy --speaker Vivian remains accepted. Examples use Vivian and Ryan.

Field	Value
Family	qwen3_tts
Model directory	models/Qwen3-TTS-12Hz-1.7B-CustomVoice
Task	tts
Modes	offline, streaming
Voice input	Built-in speaker id through --speaker
Style control	Optional instruction through --instruct
External voice WAV	Not used by this path

audiocpp_cli --task tts --family qwen3_tts --model models/Qwen3-TTS-12Hz-1.7B-CustomVoice --backend cuda --text "Hello from a custom voice." --speaker Vivian --instruct "Very happy." --out out.wav

Common Options (use directly)

Option	Values	Default	Meaning
--text	text	required	Text to synthesize.
--speaker	packaged speaker name	required	Built-in speaker id, such as Vivian or Ryan.
--instruct	text	empty string	Style or emotion instruction.
--language	language code	empty string	Text language hint.
--text-chunk-size	integer chars	8192	Long-form text chunk size. Each chunk is voiced separately; with the session option qwen3_tts.stitch_decode=true the talker still runs per chunk but the codes of all chunks are voiced in one decoder pass.
--max-tokens	positive integer	checkpoint generation config (2048 if absent)	Maximum generated speech tokens per chunk.

Qwen3 TTS VoiceDesign

Qwen3 VoiceDesign creates a voice from an instruction. It does not require a speaker reference WAV.

Field	Value
Family	qwen3_tts
Model directory	models/Qwen3-TTS-12Hz-1.7B-VoiceDesign
Tasks	tts, vdes
Modes	offline, streaming
Voice input	Instruction text through --instruct
Reference audio	Not required

audiocpp_cli --task vdes --family qwen3_tts --model models/Qwen3-TTS-12Hz-1.7B-VoiceDesign --backend cuda --text "Hello from a designed voice." --instruct "A warm adult narrator" --out out.wav

tts and vdes enter the same VoiceDesign synthesis implementation.

Common Options (use directly)

Option	Values	Default	Meaning
--text	text	required	Text to synthesize.
--instruct	text	empty string	Optional voice design instruction.
--language	language code	empty string	Text language hint.
--text-chunk-size	integer chars	8192	Long-form text chunk size. Each chunk is voiced separately; with the session option qwen3_tts.stitch_decode=true the talker still runs per chunk but the codes of all chunks are voiced in one decoder pass.
--max-tokens	positive integer	checkpoint generation config (2048 if absent)	Maximum generated speech tokens per chunk.

Qwen3 TTS Streaming

All three TTS variants accept --mode streaming. Audio is decoded and emitted while speech tokens are still being generated, rather than after the complete utterance. The first audio chunk contains one codec frame (80 milliseconds at 24 kHz). Subsequent chunks double up to the configured stream_frames_per_event maximum: with the default 12, the sequence is 1, 2, 4, 8, 12, 12, and so on. These are audio durations, not latency guarantees. Set stream_chunk_policy=fixed to use stream_frames_per_event from the first event instead; the last event may contain fewer frames. A fixed size of 1 emits one frame at a time. The default policy is grow.

Streaming retains 25 codec frames of left context and reuses a fixed-capacity decoder graph, with a separate cached graph for the first frame. The final partial chunk is flushed at end of generation. The final audio is exactly the concatenation of the emitted chunks. Streaming uses smaller codec windows than offline decoding, so waveform differences are expected; the offline decoding path is unchanged.

For the server, configure the model with "mode": "streaming" and request "stream_format": "sse" from /v1/audio/speech (streaming defaults to PCM). Audio arrives through the existing speech.audio.delta events. Base models require reference audio; the transcript can be omitted only with x_vector_only_mode=true. VoiceDesign accepts an empty instruction; CustomVoice requires a packaged speaker.

Qwen3 TTS Sampling

These sampling controls are shared by the Qwen3 TTS Base, VoiceDesign, and CustomVoice paths.

Common Options (use directly)

Option	Values	Default	Meaning
--do-sample	true, false	true	Enable sampling.
--temperature	float	0.9	Main talker temperature; must be > 0 when --do-sample=true, ignored when false.
--top-k	integer	50	Main talker top-k.
--top-p	float	1.0	Main talker top-p.
--repetition-penalty	float > 0	1.05	Main talker repetition penalty.
--seed	integer	random if omitted	Sampling seed.

Request Options (use with --request-option)

Option	Values	Default	Meaning
subtalker_do_sample	bool	true	Enable subtalker sampling for the remaining audio codebooks.
voice_id	packaged speaker name	unset	CustomVoice preset selection. Legacy speaker remains accepted. An explicit request option takes precedence over the structured voice's cached id.
subtalker_temperature	float	0.9	Must be > 0 when subtalker_do_sample=true; ignored when false.
subtalker_top_k	integer	50	Subtalker top-k.
subtalker_top_p	float	1.0	Subtalker top-p.
x_vector_only_mode	bool	false	Base voice cloning using only the reference speaker embedding, without ICL reference codec prompting.
stream_frames_per_event	positive integer	12	Codec frames per fixed event, or the maximum for growing events. Smaller values reduce chunk latency but increase decoding overhead. Ignored offline.
stream_chunk_policy	grow or fixed	grow	Grow from one frame to the configured maximum, or emit fixed-size chunks. Ignored offline.

Session Options (use with --session-option)

These options apply to Base, VoiceDesign, and CustomVoice sessions. Arena and constant-context sizes are in MiB.

cuda_perf is a preset. Explicit session options override its settings:

--session-option qwen3_tts.perf_mode=cuda_perf \
--session-option qwen3_tts.attention=eager \
--session-option qwen3_tts.packed_projections=false

In server configs, put the same keys and values in the model's session_options.

Option	Values	Default	Meaning
qwen3_tts.mem_saver	bool	false	Release the talker cached-step graph after each request. Voice-prompt, prefill, code-predictor, and speech-decoder caches remain reusable.
qwen3_tts.lora	adapter directory or checkpoint file	not set	Load a PEFT Talker/Subtalker adapter at session creation. Keep adapter_config.json beside its weights. A trained custom voice may also provide speaker_embedding and tts_config.json sidecars; select its name with voice_id.
qwen3_tts.lora_scale	finite float	1	Strength multiplier applied after the adapter's alpha/r scaling. Speaker embedding replacements are not scaled.
qwen3_tts.perf_mode	off, flash_attention, cuda_perf	off	Preset. off keeps normal defaults; flash_attention selects flash attention. cuda_perf also selects packing, optimized cache writes, F16 KV and convolution weights, stateful streaming decoding, and KV alignment 256. Explicit options override the preset.
qwen3_tts.attention	mixed, auto, eager, flash	mixed	mixed uses manual prefill and contiguous-K/V flash cached decoding for the talker and code predictor, with eager speech tokenization. auto uses the shared backend policy; eager disables flash; flash uses flash with K/V views. Explicit values override the preset; cuda_perf selects flash.
qwen3_tts.packed_projections	bool	false; true with cuda_perf	Pack QKV and gate/up projections in the talker and code predictor without changing weight dtype.
qwen3_tts.kv_cache_write	standard, optimized	standard; optimized with cuda_perf	Optimized cache views and I64 indices allow backend fusion where supported; no CUDA-version requirement.
qwen3_tts.kv_cache_type	f32, f16	f32; f16 with cuda_perf	Talker and code-predictor KV storage. F16 reduces cache memory and may change numerical results. Explicit values override the preset.
qwen3_tts.talker_kv_cache_alignment	int ≥ 1	1; 256 with cuda_perf	Round talker KV capacity to this many rows. Alignment 1 disables padding. Explicit values override the preset. Larger alignments may use more memory and change numerical results.
qwen3_tts.stream_decoder	windowed, stateful	windowed; stateful with cuda_perf	GGML streaming decoder. stateful carries convolution and attention history and caches graphs for exact chunk lengths. Uses more graph memory and can change audio.
qwen3_tts.stitch_decode	bool	false	Voice the concatenated speech codes of all text chunks in one decoder pass instead of voicing each chunk separately. Per-chunk voicing voices the voice-clone reference once per chunk and zeroes the stateful vocoder cache at every chunk boundary (measured on RK3588, 300 characters, 22.5 s reference: 7 vocoder runs for 159.9 s against 1 run for 57.5 s); the stitched pass costs device memory proportional to the utterance, since one decode call builds a vocoder graph sized by its frame count.
qwen3_tts.voice_prompt_cache_slots	nonnegative integer	1	Cached voice-clone prompts; 0 disables prompt caching.
qwen3_tts.weight_type	native, f32, f16, bf16, q8_0	native	Talker weight storage fallback.
qwen3_tts.talker_weight_type	native, f32, f16, bf16, q8_0	inherits weight_type	Explicit talker weight storage override.
qwen3_tts.speech_encoder_weight_type	native, f32, f16, bf16, q8_0	native	Reference speech encoder matmul weight storage.
qwen3_tts.speech_decoder_weight_type	native, f32, f16, bf16, q8_0	native	Speech decoder matmul weight storage.
qwen3_tts.conv_weight_type	native, f32, f16	f32; f16 with cuda_perf	Speech tokenizer and speaker encoder convolution weight storage.
qwen3_tts.talker_graph_arena_mb	positive integer	32	Talker graph arena.
qwen3_tts.speech_encoder_graph_arena_mb	positive integer	32	Reference speech encoder graph arena.
qwen3_tts.speech_decoder_graph_arena_mb	positive integer	32	Speech decoder graph arena.
qwen3_tts.speaker_encoder_graph_arena_mb	positive integer	32	Speaker encoder graph arena.
qwen3_tts.talker_constant_context_mb	positive integer	4	Talker constant tensor context.
qwen3_tts.code_predictor_constant_context_mb	positive integer	4	Code predictor constant tensor context.
qwen3_tts.speech_decoder_constant_context_mb	positive integer	4	Speech decoder constant tensor context.
qwen3_tts.exec_plan	JSON object of component plans, encoded as a string	{}	{"speech_decoder":{"impl":"rknn","artifacts":{"main":"/models/decoder.rknn","first":"/models/decoder-first.rknn","window":"/models/decoder-window.rknn"}}} selects the NPU decoder and skips GGML decoder weights. main is required; first optionally uses a smaller engine for the first streaming group; window optionally streams with window semantics. No decoder-specific params. See component execution plans.

Rockchip NPU decoder

The speech decoder stage can run on a Rockchip NPU through librknnrt instead of ggml. The path is opt-in at build time:

cmake -S . -B build -DENGINE_ENABLE_RKNN=ON \
  -DENGINE_RKNN_INCLUDE_DIR=/usr/local/include \
  -DENGINE_RKNN_LIBRARY=/usr/lib/librknnrt.so

The .rknn must be the stateful fp16 vocoder export for the same checkpoint and target SoC: inputs audio_codes, freqs_in, cache_valid, cache_in; outputs audio_values and cache_out; 1920 samples per token at 24 kHz, with the convolution and KV cache carried across steps in the graph I/O. The optional first-group model runs the first token group at a smaller compiled batch; the cache layout is batch-independent, so its cache feeds the main model directly. The optional window model (artifacts.window) streams with the ggml path's window semantics instead: every event runs a full decode of its whole [left context + new] window from a zeroed cache and drops the context frames' audio, so each event reproduces the CPU streaming function exactly (25 frames re-voiced per event, at one NPU step of 25 + frames_per_event tokens). Naming the same file as main aliases it: the resident engine covers the default 25 + 12 window; a dedicated export compiled at --codes-length 25 + stream_frames_per_event is cheaper per event. Offline decoding keeps using main. A .rknn is compiled per SoC and is not portable between chips. Build the file with tools/rknn/prepare_qwen3_tts_rknn_vocoder.py and see the Rockchip NPU build guide for the toolkit venv, the measured numbers, and the limits of this path. This path is opt-in and validated only on RK3588.