voice_audio_multi/docs/models/higgs_audio_tts.md

Higgs Audio v3 TTS

Higgs Audio v3 TTS is a voice-clone TTS model. The current integration uses the framework chunker for long text and keeps the reference prompt state in the model session. Use clon for voice-cloning requests; the existing tts route remains an alias to the same model path for compatibility.

For chunked generation without reference audio, the first generated chunk's audio codes and transcript condition the remaining chunks to preserve its voice. This reference is local to the request; supplied reference audio takes precedence.

Install

The model manager installs the Q8_0 standalone GGUF package by default:

python3 tools/model_manager_v2.py install --models-root models higgs_audio_tts_4b_q8_0

Quick Start

audiocpp_cli --task clon --family higgs_audio_tts --model models/Higgs-Audio-v3-TTS-4B-GGUF/higgs-audio-v3-tts-4b-q8_0.gguf --backend cuda --text "Hello from Higgs Audio." --voice-ref assets/resources/b.wav --reference-text "Some call me nature. Others call me Mother Nature. I've been here for over 4.5 billion years. 22,500 times longer than you." --out out.wav

Model

Field	Value
Family	higgs_audio_tts
Model path	models/Higgs-Audio-v3-TTS-4B-GGUF/higgs-audio-v3-tts-4b-q8_0.gguf when installed through the model manager
Tasks	tts, clon
Modes	offline, streaming
Languages	Model auto-handles supported languages
Voice input	Reference WAV through --voice-ref; transcript through --reference-text when known
Built-in voices	Not exposed

Common Options (use directly)

Option	Values	Default	Meaning
--voice-ref	WAV path	optional	Reference speaker audio for cloning; generation without a reference is also supported.
--reference-text	text	empty string	Transcript for reference audio.
--text-chunk-size	integer chars	1024	Long-form chunk size.
--max-tokens	integer	2048	Maximum generated AR tokens; zero uses the default. Must not exceed the configured context length.
--temperature	float	0.8	AR sampling temperature. Must be positive.
--top-k	integer	30	AR top-k sampling limit. The narrower default is less prone to premature EOC than the Python client's 50.
--top-p	float	0.8	AR nucleus sampling probability. Must be positive.
--repetition-penalty	float	1.1	Accepted for Python API compatibility; audio sampling does not consume this value. Compatibility-only, but must be finite and positive.

Request Options (use with --request-option)

Option	Values	Default	Meaning
max_tokens	integer >= 0	2048	Maximum generated AR tokens; zero uses the default. Must not exceed the configured context length.
allow_incomplete_generation	boolean	false	Return generated audio with a warning instead of failing when max_tokens is reached before EOC. Audio may be truncated or contain silence/noise.
temperature	float	0.8	AR sampling temperature. Must be positive.
top_p	float	0.8	AR nucleus sampling probability. Must be positive.
top_k	integer	30	AR top-k sampling limit.
filter_nan_logits	boolean	false	Filter NaN and infinity during greedy and stochastic top-k selection. All-invalid logits still fail.
repetition_penalty	float	1.1	Accepted for Python API compatibility; audio sampling does not consume this value. Compatibility-only, but must be finite and positive.
seed	integer >= 0	random	Torch sampling seed; omitted chooses a random seed.
reference_text	text	empty string	Transcript accompanying reference audio.
text_chunk_size	integer	1024	Framework long-form text chunk size.
text_chunk_mode	default, tag_aware, japanese, endline	default	Framework text chunking mode.
stream_frames_per_event	integer >= 1	32	Streaming audio frame cap; each codec frame contains 40 ms of audio.
stream_chunk_policy	grow, fixed	grow	Start with one frame and grow to the cap, or emit fixed-size chunks. Streaming only.

If mixed finite and non-finite logits disrupt greedy or stochastic top-k sampling, try --request-option filter_nan_logits=true, or "filter_nan_logits": true in the server request's options. Greedy sampling (top_k=1) then selects the highest finite logit. It does not repair invalid model output; all-invalid logits still fail.

For best-effort output when the model does not finish within the token budget, use --request-option allow_incomplete_generation=true. On the server, set "allow_incomplete_generation": true in the request's options object. This applies to offline and streaming generation. Normal EOC stopping is unchanged; without the option, reaching the limit before EOC still fails.

Streaming

Set the session mode to streaming. Audio is decoded and emitted during AR generation, once all eight delayed codebooks for each frame are available. The default chunk schedule is 1, 2, 4, 8, 16, then 32 frames per event; the final event may be shorter. The non-causal codec needs ten frames of future context, so a one-frame first event still waits for that context. This is not a one-AR-step latency guarantee.

Streaming keeps bounded codec windows and reuses their graphs within the session. Text chunking and reference conditioning remain the same as offline. The final WAV contains exactly the concatenated audio deltas. Floating-point differences from offline decoding are possible because codec graph shapes differ.

For the server, configure "mode": "streaming" on the model entry and request "stream_format": "sse" from /v1/audio/speech. Streaming controls belong in the request's options object. They are not applied to offline generation.

Session Options (use with --session-option)

Try --session-option higgs_audio_tts.reference_prefill_mode=exact for excessive repetition or unexpectedly long, mostly silent output on repeated voice-cloning requests. In server configuration, set "session_options": {"higgs_audio_tts.reference_prefill_mode": "exact"} on the model entry. This preserves reference-KV reuse and leaves full-prompt prefill and AR decoding unchanged. It may increase prefill time and is not a general generation-quality fix. The mode is fixed for the session and applies to offline and streaming generation.

Option	Values	Default	Meaning
higgs_audio_tts.reference_prefill_mode	auto, exact	auto	Cached-reference prefill attention; auto preserves existing behavior, exact enables the workaround described above.
higgs_audio_tts.weight_type	native, f32, f16, bf16, q8_0	native	Shared AR and codec weight storage type.
higgs_audio_tts.ar_weight_type	native, f32, f16, bf16, q8_0	shared setting	Autoregressive decoder weight storage override.
higgs_audio_tts.codec_weight_type	native, f32, f16, bf16, q8_0	shared setting	Audio codec weight storage override.
higgs_audio_tts.ar_weight_context_mb	integer MiB >= 1	not set	AR weight context size.
higgs_audio_tts.codec_weight_context_mb	integer MiB >= 1	1536	Codec weight context size.
higgs_audio_tts.ar_decode_graph_arena_mb	integer MiB >= 1	512	AR decode graph arena size.
higgs_audio_tts.codec_decode_graph_arena_mb	integer MiB >= 1	128	Codec decode graph arena size.
higgs_audio_tts.codec_encode_graph_arena_mb	integer MiB >= 1	256	Reference-audio codec encode graph arena size.
higgs_audio_tts.reference_cache_slots	integer >= 0	1	Encoded reference-audio cache slots; 0 disables reuse.
higgs_audio_tts.attention	auto, flash, eager	auto	Attention kernel. auto uses flash except on Volta/Turing GPUs (e.g. V100), where it falls back to eager to avoid missing MMA kernels.