voice_audio_multi/docs/models/moss_tts_v15.md

MOSS-TTS-v1.5

Zero-shot voice cloning from a short reference recording, on the 8B delay-pattern member of the MOSS-TTS family.

• Family: moss_tts_v15
• Tasks: tts, clon (cloning), offline
• Languages: English, Chinese
• Weights: OpenMOSS-Team/MOSS-TTS-v1.5, Apache-2.0
• Codec weights: OpenMOSS-Team/MOSS-Audio-Tokenizer, Apache-2.0
• Architecture: moss_tts_delay — a Qwen3-8B backbone with 32 audio codebook embeddings and 1 + 32 output heads, decoded on a delay pattern
• Codec: MOSS-Audio-Tokenizer v1, 24 kHz mono, hop 1920 (12.5 frames a second)

It shares its architecture with moss_voicegen, which is the same design at Qwen3-1.7B and 16 codebooks. The backbone, heads, delay decoder and config parser are shared framework code (engine/framework/decoders/moss_tts_delay/); what is specific to this checkpoint is the prompt and the session.

Usage

Cloning, which is what this checkpoint is for:

audiocpp_cli --family moss_tts_v15 --model <model-dir> --task clon \
  --voice-ref reference.wav \
  --text "This sentence should be spoken in the voice from the reference recording." \
  --out out.wav

Without a reference, it is plain TTS and the voice is whatever the model picks:

audiocpp_cli --family moss_tts_v15 --model <model-dir> --task tts \
  --text "The quick brown fox jumps over the lazy dog." --out out.wav

The reference is resampled and downmixed to the codec's 24 kHz mono and trimmed to whole frames, so any sample rate and channel count will do. It is encoded once per request, not once per text chunk.

Options

option	meaning
--voice-ref <wav>	Reference recording to clone.
--request-option duration_frames=<int>	Duration budget in codec frames at 12.5 a second — the model's own - Tokens: field. 40 frames is about 3.2 s. Legacy tokens remains accepted.
--request-option instruct=<text>	Voice description. Legacy instruction remains accepted. See the limitation below.
--request-option text_temperature=<float>	Text-stream sampling temperature; default 1.5. Legacy moss_tts_v15.text_temperature remains accepted.
--language <name>	Full language name; the model does not understand codes like en.
--seed, --temperature, --top-p, --top-k, --repetition-penalty	Sampling.
--request-option min_frames=<int>	Minimum audio frames per text chunk; default 0 derives a text-length-based floor. Nonpositive values select this automatic behavior. Separate from duration_frames conditioning.
--request-option max_frames=<int>	Audio-frame ceiling per text chunk; default 0 derives a text-length-based ceiling. Nonpositive values select this automatic behavior. Runtime raises the ceiling to at least the effective minimum plus the number of codebooks.
moss_tts_v15.weight_type	native (default), f32, bf16 or q8_0; native preserves packaged quantization.

f16 is rejected for the backbone: its attention-sink activations run far past f16's range and produce NaN from the first position. This is inherited from the family and is the same on moss_voicegen.

⚠ Limitation: voice-attribute instructions are not reliably followed

instruct is accepted, but on this checkpoint a description of the speaker is followed only loosely. Measured on the reference implementation, four prompts differing only in the requested speaker, median F0 of the result:

"A high-pitched young woman's voice, clearly female."   -> 216.3 Hz   plausible
"A woman speaking softly and warmly."                   -> 184.5 Hz   borderline
"A man speaking in a low register."                     -> 187.8 Hz   miss
"A very deep, low-pitched man's voice."                 -> 180.8 Hz   clear miss

Five samples with median F0 as a proxy is not a rigorous evaluation, and one of the four was plausible — so this supports "unreliable", not "never works". But a "very deep, low-pitched man's voice" at 181 Hz is wrong by any measure.

Use a reference recording when the voice matters. Cloning does work: a clone of a 189.4 Hz reference came back at 184.0 Hz, where the same model without a reference produced 117.3 Hz. If you want a voice from a written description rather than a recording, moss_voicegen is the model built for that.

Performance and VRAM

Measured on an RTX 3090 (24 GiB) and a 16-thread CPU. The times are session.wall_ms — the work itself, excluding model load, which for a 9 GB package is tens of seconds of disk I/O and swamps everything else if you time the whole process.

package	backend	RTF	peak VRAM (cloning)
q4_k	CUDA	0.69–0.72	14.1 GiB
q8_0	CUDA	0.80–0.84	17.9 GiB
bf16	CUDA	—	does not fit in 24 GiB when cloning
q4_k	CPU, 16 threads	~16	—

The CUDA figures are the range over three runs on an otherwise-in-use desktop GPU; a single run of q8_0 also came back at 1.08, so treat these as a band rather than a number.

CUDA is roughly 25-30x faster than the CPU path here, and comfortably faster than real time; the CPU path is not.

weight_type defaults to native, and you want to keep it there. A GGUF package carries its own type; forcing bf16 dequantises it on load, so a quantised package costs exactly as much memory as bf16 and the quantisation buys nothing. Measured: q4_k peaks at 21.2 GiB forced to bf16 against 10.6 GiB left native. For safetensors it makes no difference — those are bf16 upstream anyway.

bf16 cannot clone on a 24 GiB card. Cloning holds both halves of the codec resident, where plain TTS needs only the decoder, and the encoder's weights are the allocation that does not fit. Plain TTS works at every size. Use q8_0 on a 24 GiB card, or bf16 on something larger.

What works

Measured on the reference implementation and reproduced through audio.cpp:

voice cloning	tracks the reference closely
Chinese	works, from an English instruction
long form	20.4 s from four sentences; RTF improves with length
tokens budget	honoured — 40 tokens produced 41 frames

Packaging

For safetensors, the model and the audio tokenizer are separate Hugging Face repositories. Place the tokenizer snapshot under audio_tokenizer/ inside the model root — a real directory, not a symlink, which the GGUF converter does not follow:

MOSS-TTS-v1.5/
  config.json
  tokenizer.json
  tokenizer_config.json
  merges.txt
  model.safetensors.index.json
  model-0000{1..4}-of-00004.safetensors
  audio_tokenizer/
    config.json
    model.safetensors.index.json
    model-0000{1,2}-of-00002.safetensors

Building the GGUF package:

audiocpp_gguf \
  --input model_weights=<root>/model.safetensors.index.json \
  --input audio_tokenizer_weights=<root>/audio_tokenizer/model.safetensors.index.json \
  --output moss_tts_v15_bf16_codec_f16.gguf \
  --type bf16 --keep-type "audio_tokenizer_weights*=f16" \
  --family moss_tts_v15 --root <root>

That produces about 20.5 GB with 2063 tensors, the model spec and 22 sidecars embedded. Unlike moss_voicegen, the codec encoder is included as well as the decoder, because cloning needs it.