voice_audio_multi/docs/models/vieneu_v3_turbo.md

VieNeu-TTS v3 Turbo

VieNeu-TTS v3 Turbo is an on-device bilingual (Vietnamese / English) TTS model with instant voice cloning by Phạm Nguyễn Ngọc Bảo (pnnbao97): a 12-layer Qwen3-style backbone, a small acoustic decoder that emits the 16 residual-VQ codes of each 80 ms frame, and the MOSS-Audio-Tokenizer-Nano codec (48 kHz). audio.cpp runs it torch-free on CPU and CUDA. This family is maintained in audio.cpp by the model author.

Field	Value
Family	vieneu_v3_turbo (alias: vietneu_tts, the name of the first community port)
Model directory	models/VieNeu-TTS-v3-Turbo-GGUF
Task	tts (including reference voice cloning)
Modes	offline
Languages	vi, en
Text input	SEA-G2P phonemes (see below), not raw text
Voice input	Reference WAV, or pre-encoded reference codes + speaker embedding
Output	stereo 48 kHz WAV

Model package

GGUF packages published by the model author in the model's own repo, pnnbao-ump/VieNeu-TTS-v3-Turbo under gguf/, built from its update/ weights with audiocpp_gguf (the model spec, config.json and tokenizer sidecars are embedded, so one file is enough):

File	Precision	Size
vieneu-v3-turbo-q8_0.gguf	Q8_0 matmuls, bf16 norms / embeddings, f16 codec	188 MB
vieneu-v3-turbo-bf16.gguf	bf16	292 MB
voices/<id>/{ref_codes.txt,speaker.emb.txt}	—	Preset voice resources listed in voices/manifest.json; the model-manager packages install only the default minh_quan_pro pair. Download other voice pairs separately.

python tools/model_manager_v2.py install vieneu_v3_turbo          # q8_0 + the default voice (minh_quan_pro)
python tools/model_manager_v2.py install vieneu_v3_turbo_bf16

The original HF layout also loads directly (--model <dir> with model.safetensors, config.json, tokenizer.json and speech_tokenizer/{config.json,model.safetensors} = OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano).

The codec stays at f16 in both packages, and it is the only tensor group that does. Quantising it to Q8_0 costs 4 dB of SNR against the reference decoder on identical codes (32.2 vs 36.0 dB) — audible as a faint haze — for 16 MB and no speed, because the codec runs once per chunk rather than once per frame. bf16 is the wrong shape for it in the other direction: the same 2 bytes, but 7 bits of mantissa against f16's 10, and the encoder is sensitive enough to show it. On a 4.6 s clip, reference codes from the f16 codec agree with the fp32 reference encoder on 96.3% of the 912 codes and the bf16 codec on 82.4%, so a bf16 package keeps a bf16 backbone and an f16 codec. f32 measures the same as f16.

To repack from a safetensors directory:

audiocpp_gguf --input model_weights=VieNeu-TTS-v3-Turbo/model.safetensors \
  --input speech_tokenizer_weights=VieNeu-TTS-v3-Turbo/speech_tokenizer/model.safetensors \
  --root VieNeu-TTS-v3-Turbo --family vieneu_v3_turbo --type q8_0 \
  --keep-type "speech_tokenizer_weights*=f16" \
  --output vieneu-v3-turbo-q8_0.gguf

The bf16 package is the same command with --type bf16, and it keeps the same --keep-type.

Text input

The model reads SEA-G2P phonemes, and the front end that produces them — sea-g2p: Vietnamese normalisation of numbers, dates, units and abbreviations, English code-switching, emotion cues — is a Rust library with a C ABI. Point the session at it and --text takes ordinary text:

# once: build the library (Rust toolchain, nothing added to the audio.cpp build)
git clone https://github.com/pnnbao97/sea-g2p && cd sea-g2p
cargo build --release --no-default-features --features capi
# target/release/{libsea_g2p_rs.so | sea_g2p_rs.dll | libsea_g2p_rs.dylib}

audiocpp_cli --task tts --family vieneu_v3_turbo --model .../vieneu-v3-turbo-q8_0.gguf   --backend cpu --text "Tỉ lệ giải ngân đầu tư công đạt 68,5% kế hoạch năm."   --session-option vieneu_v3_turbo.g2p_dict_file=/path/to/sea_g2p.bin   --request-option reference_codes_file=.../voices/minh_quan_pro/ref_codes.txt   --request-option speaker_embedding_file=.../voices/minh_quan_pro/speaker.emb.txt   --out out.wav

The library is found by name (next to the binary, or on the search path) unless vieneu_v3_turbo.g2p_library names it. sea_g2p.bin is the dictionary, ~50 MB, published with the Python package.

Legacy vieneu_v3_turbo.g2p_dict and bare g2p_dict remain accepted for the dictionary file.

Without those options nothing changes: --text is then the phoneme string, which is what every earlier version expected.

from vieneu_utils.phonemize_text import phonemize_text_with_emotions   # pip install vieneu
print(phonemize_text_with_emotions("Xin chào thế giới. Đây là bản thử nghiệm."))
# sˈin tʃˈaː2w tˈeɜ zˈəːɜj. ɗˈəɪ lˌaː2 bˈaː4n tˈy4 ŋˈiɛ6m.

Both routes run the same code, so they agree character for character — checked on dates, times, money, percentages and English words.

Voice

Every request needs a voice. Two ways:

1. Reference WAV (--voice-ref): the codec encodes it into reference codes (mono, resampled to 48 kHz, first 8 s). A 192-d speaker embedding is also required: the CAM++ speaker encoder is not ported yet, so pass one with speaker_embedding_file= (192 comma-separated floats, produced by the Python engine's extract_speaker_emb). Without it the embedding is all zeros and the voice will not match.
2. Packaged voice (no audio): reference_codes_file= (one frame per line, 16 integers — numpy.savetxt(codes, fmt="%d") of the Python ref_codes) plus speaker_embedding_file=. This is what the Python preset voices ship, and it skips the encoder pass.
3. Speaker embedding only: speaker_embedding_file= with no reference audio and no codes (x_vector_only_mode is then implied). The voice is weaker than with reference codes, because the model only gets the speaker anchor, not the in-context reference frames.

audiocpp_cli --task tts --family vieneu_v3_turbo \
  --model models/VieNeu-TTS-v3-Turbo-GGUF/vieneu-v3-turbo-q8_0.gguf --backend cpu \
  --text "sˈin tʃˈaː2w tˈeɜ zˈəːɜj. ɗˈəɪ lˌaː2 bˈaː4n tˈy4 ŋˈiɛ6m." \
  --request-option reference_codes_file=models/VieNeu-TTS-v3-Turbo-GGUF/voices/minh_quan_pro/ref_codes.txt \
  --request-option speaker_embedding_file=models/VieNeu-TTS-v3-Turbo-GGUF/voices/minh_quan_pro/speaker.emb.txt \
  --out out.wav

Reference audio instead of packaged codes:

audiocpp_cli --task tts --family vieneu_v3_turbo \
  --model models/VieNeu-TTS-v3-Turbo-GGUF/vieneu-v3-turbo-q8_0.gguf --backend cpu \
  --text "<phonemes>" --voice-ref voice/ref.wav \
  --request-option speaker_embedding_file=voice/speaker.emb.txt --out out.wav

Enrolling a voice

Adding a voice to an application is not a synthesis: what it needs is the reference codes, kept on disk, so that every later request is a packaged voice (option 2 above) and the clip and the encoder are never touched again.

encode_reference_only=true runs the encoder pass and stops. The codes come back as an acoustic_tokens artifact whose payload is the text reference_codes_file reads — one frame per line, code_groups integers each — so the file the CLI writes goes straight back in:

# once per voice: the clip becomes codes
audiocpp_cli --task tts --family vieneu_v3_turbo \
  --model models/VieNeu-TTS-v3-Turbo-GGUF/vieneu-v3-turbo-q8_0.gguf --backend cpu \
  --voice-ref voice/ref.wav --request-option encode_reference_only=true \
  --out-dir voice/
# voice/vieneu_v3_turbo_reference_codes.txt

# from then on: no clip, and no encoder pass
audiocpp_cli --task tts --family vieneu_v3_turbo \
  --model models/VieNeu-TTS-v3-Turbo-GGUF/vieneu-v3-turbo-q8_0.gguf --backend cpu \
  --text "<phonemes>" \
  --request-option reference_codes_file=voice/vieneu_v3_turbo_reference_codes.txt \
  --request-option speaker_embedding_file=voice/speaker.emb.txt --out out.wav

frames, code_groups and sample_rate are in the artifact's metadata, for a caller reading it through the API rather than off disk.

A normal request that was given --voice-ref returns the same artifact alongside its audio, so a caller can keep the voice it just paid to derive. Codes passed in through reference_codes are not echoed back.

The speaker embedding still has to come from elsewhere: the CAM++ encoder is not ported (see Not yet ported).

Options

Defaults follow Vieneu.infer() in the Python package. Use --request-option name=value; --temperature, --top-k, --top-p, --repetition-penalty and --max-tokens map to the same names.

Option	Default	Meaning
temperature / top_k / top_p	0.8 / 25 / 0.95	One sampler for all 16 codebooks of the acoustic decoder.
repetition_penalty	1.2	Penalty on codes seen in the recent window of each codebook.
repetition_window	64	Frames each codebook remembers (~5 s); 0 = unbounded.
do_sample	true	false = greedy (argmax) decoding.
max_tokens	300	Frame budget per chunk (80 ms each).
auto_frame_cap	true	Also cap the budget by the phoneme count of the chunk (Python max_expected_frames), which stops runaway generation when EOS is missed. Legacy frame_cap remains accepted.
seed	random	Sampling seed.
reference_codes_file / speaker_embedding_file / speaker_embedding	—	Voice inputs, see above.
x_vector_only_mode	input-dependent	Defaults to true for embedding-only input; otherwise false. Ignore reference codes and clone from the speaker embedding alone.
encode_reference_only	false	Encode --voice-ref into reference codes, return them as an artifact and generate nothing.
text_chunk_size	200	Character budget per chunk of the phoneme string. The cut follows paragraphs, then sentences, then minor punctuation, then whitespace.
text_chunk_min_size	20	Chunks shorter than this join a neighbour — alone they read as a stutter. Legacy text_chunk_min remains accepted.
babble_retries	2	Re-generations for a short chunk that kept talking past its text (0 disables the guard; negative values are clamped to 0).
subtalker_temperature / subtalker_top_k / subtalker_top_p	= main	Acoustic decoder overrides.
codes_dump_file	—	Parity debugging: appends prompt ids, reference codes and generated codes as text.

Session options: vieneu_v3_turbo.weight_type (native|f32|f16|bf16|q8_0), vieneu_v3_turbo.mem_saver, vieneu_v3_turbo.voice_prompt_cache_slots. Options written with the old vietneu_tts. prefix are still accepted (mapped to the current prefix, with a deprecation note in the log); an unknown option is rejected rather than ignored.

Parity and performance

Checked against fp32 references with identical prompt inputs (CPU, weight_type=f32): backbone prefill hidden state max |diff| 6e-7; acoustic-decoder logits identical to a numpy fp32 implementation of the safetensors weights; codec decoder 83 dB SNR on the same codes. With the GGUF packages the acoustic logits deviate by 0.007 (bf16) / 0.02–0.03 (q8_0) on average — smaller than the int8 acoustic graph the Python CPU path ships.

Speed on an Intel Core i5-12400F (6 P-cores, --threads 6, q8_0): a 28 s utterance in 8.0 s wall including process start and model load, RTF ≈ 0.29 (0.34 with --threads 4); the Python CPU path on the same machine is RTF 0.55–0.62 (fp32 ONNX) / 0.35 (int8 ONNX, needs VNNI).

Chunking, pauses and the babble guard

A long text is generated chunk by chunk, and the seams are what make the pieces sound like one utterance:

• the cut follows paragraphs, then sentences (. ! ? …), then minor punctuation, then whitespace, packing up to text_chunk_size characters of the phoneme string; a chunk under text_chunk_min_size joins a neighbour;
• the pause between two chunks is a minimum, not an insertion: the silence the model already left (tail of one, lead of the next) is measured and only the shortfall is padded with zeros — 0.70 s after a paragraph, 0.50 s after a sentence, 0.30 s otherwise. A longer natural pause is kept;
• a chunk that ran to its frame ceiling never emitted EOS, so its tail is missing; with sampling on it is generated again (up to twice) and the shorter result kept;
• the babble guard catches a short chunk that kept talking past its text ("Được." coming out as "Được không?"): it compares the energy bursts in the decoded audio with the chunk's syllable count, and re-generates a suspect chunk, keeping the best attempt. Measured on the Python engine: ~5% of one-syllable chunks, 8 cases in 480 chunks down to 0.

The text-level rules of the Python engine that need the normaliser — cutting at connectives, never between two number words — are not here: they belong with a text front end rather than as a second copy.

Not yet ported

• CAM++ speaker encoder — pass speaker_embedding_file.
• Reference denoiser used at enrollment by the Python engine.
• Streaming; the enrollment denoiser.

Credits

• Model, training data and reference implementation: Phạm Nguyễn Ngọc Bảo (pnnbao97, pnnbao-ump on Hugging Face) — also the maintainer of this audio.cpp family.
• First audio.cpp port (vietneu_tts, PR #80): Phuoc Nguyen (phuocnguyen90).
• Audio codec: MOSS-Audio-Tokenizer-Nano (OpenMOSS-Team).