# IndexTTS

IndexTTS is wired as `--family index_tts2` for offline text to speech and voice
cloning. The same model family handles IndexTTS2 and IndexTTS2.5; the runtime
selects the variant from the model config.

## IndexTTS2

IndexTTS2 is a Chinese and English TTS model with voice cloning and expressive
emotion controls. It requires a speaker reference through the framework
`--voice-ref` path.

| Field | Value |
|---|---|
| Family | `index_tts2` |
| Model directory | `models/IndexTTS-2` |
| Task | `tts`, `clon` |
| Modes | `offline` |
| Languages | `zh`, `en` |
| Voice input | Required reference WAV through `--voice-ref` |
| Built-in voices | Not exposed |

Voice clone:

```bash
audiocpp_cli --task clon --family index_tts2 \
  --model /path/to/IndexTTS-2 \
  --backend cuda \
  --language en \
  --text "Hello from IndexTTS2." \
  --voice-ref /path/to/reference.wav \
  --out out.wav
```

Emotion text:

```bash
audiocpp_cli --task tts --family index_tts2 \
  --model /path/to/IndexTTS-2 \
  --backend cuda \
  --language zh \
  --text "今天的演示会更有情绪。" \
  --voice-ref /path/to/reference.wav \
  --emotion "你吓死我了！你是鬼吗？" \
  --request-option emotion_alpha=0.6 \
  --out out.wav
```

### IndexTTS2 Common Options (use directly)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--voice-ref` | WAV path | required | Reference speaker audio. |
| `--audio` | WAV path | not set | Emotion-reference audio. |
| `--language` | `zh`, `en` | empty | Text language label. |
| `--emotion` | text | not set | Emotion-text conditioning through the framework style field. |
| `--text-chunk-size` | characters | not set | Optional framework long-form text chunk size; omitted keeps the full input together. Must be positive. |
| `--text-chunk-mode` | `default`, `tag_aware`, `japanese`, `endline` | `default` | Framework chunking mode used only when `--text-chunk-size` is set. |
| `--max-tokens` | integer | model default | Maximum generated GPT mel tokens. |
| `--temperature` | float | model default | Sampling temperature. Must be positive. |
| `--top-p` | float | model default | Nucleus sampling probability. Must be in (0, 1]. |
| `--top-k` | integer | model default | GPT top-k sampling limit. |
| `--repetition-penalty` | float | model default | GPT repetition penalty. |
| `--do-sample` | `true`, `false` | model default | Enable stochastic GPT sampling. |
| `--num-beams` | integer | `3` | GPT beam count. |
| `--seed` | integer | random when omitted | Generation seed. |

### IndexTTS2 Request Options (use with `--request-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `text` | text | not set | Fallback target text when text_input is absent; prompt is also accepted. |
| `language` | text | not set | IndexTTS2.5 text language; empty or auto selects from text. |
| `emotion_alpha` | float in `[0, 1]` | `1` | Blend strength for explicit emotion conditioning. |
| `emotion_vector` | 8 floats | not set | Explicit emotion vector. |
| `use_emotion_text` | bool | `false` | Infer emotion from text. |
| `emotion_text` | text | not set | Text used when emotion-text conditioning is enabled. |
| `use_random_emotion` | bool | `false` | Use random emotion weights in the emotion mixer. |
| `interval_silence_ms` | milliseconds | `200` | Silence inserted between generated text chunks. |
| `do_sample` | bool | `true` | Enable sampling. |
| `top_p` | float | `0.8` | Nucleus sampling probability. Must be in (0, 1]. |
| `top_k` | integer >= 1 | `30` | Top-k sampling limit. |
| `temperature` | float | `0.8` | Sampling temperature. Must be positive. |
| `length_penalty` | float | `0` | GPT beam-search length penalty. |
| `num_beams` | integer >= 1 | `3` | Beam search width. |
| `repetition_penalty` | float | `10` | Sampling repetition penalty. |
| `max_tokens` | integer >= 1 | `1500` | Maximum generated mel tokens. |
| `duration_scale` | positive float | `1` | Positive waveform duration rescaling, supported by both IndexTTS2 and IndexTTS2.5. |
| `seed` | integer >= 0 | random | Sampling seed; omitted chooses a random seed. |
| `text_chunk_size` | integer | not set | Optional framework long-form text chunk size; omitted keeps the full input together. Must be positive. |
| `text_chunk_mode` | `default`, `tag_aware`, `japanese`, `endline` | `default` | Framework text chunking mode. |

### IndexTTS2 Session Options (use with `--session-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `index_tts2.weight_type` | `native`, `f32`, `f16`, `bf16`, `q8_0` | not set | Matmul weight storage type. |
| `index_tts2.conv_weight_type` | `native`, `f32`, `f16` | `native` | Convolution weight storage type. |
| `index_tts2.mem_saver` | bool | `false` | Release staged reference and conditioning graphs after request phases. |
| `index_tts2.speaker_cache_slots` | integer slots | `1` | Prepared speaker-reference cache slots; set `0` to disable reuse. |
| `index_tts2.emotion_cache_slots` | integer slots | `1` | Prepared emotion-reference cache slots; set `0` to disable reuse. |
| `index_tts2.emotion_text_cache_slots` | integer slots | `1` | Emotion-text weight cache slots; set `0` to disable reuse. |
| `index_tts2.gpt_graph_arena_mb` | MB | `32` | GPT graph arena size. |
| `index_tts2.s2mel_graph_arena_mb` | MB | `32` | S2Mel graph arena size. |
| `index_tts2.reference_graph_arena_mb` | MB | `512` | Reference encoder and codec graph arena size. |
| `index_tts2.emotion_text_prefill_graph_arena_mb` | MB | `2048` | Emotion-text prefill graph arena size. |
| `index_tts2.emotion_text_decode_graph_arena_mb` | MB | `512` | Emotion-text cached-step graph arena size. |
| `index_tts2.weight_context_mb` | MB | `32` | Shared ggml weight metadata context size. |
| `index_tts2.emotion_text_max_tokens` | tokens | `256` | Maximum generated tokens for emotion-text classification; old name `index_tts2.emotion_text_max_new_tokens` is still accepted. |

Text normalization note: audio.cpp reimplements the official IndexTTS text
front-end (wetext-style number/date/symbol rules, pinyin-tone and name
protection) as lightweight C++ rules instead of running the official OpenFst
grammars. Alignment is tracked by a golden-corpus parity test
(`index_tts_tn_corpus_test`, which replays 50+ tricky cases against outputs
generated by the official Python normalizer), but a rule-based port cannot be
exhaustive: for unusual inputs (letter-attached digits, rare date/unit formats,
emails, URLs) the normalized text can still differ from the official pipeline,
so the same prompt may be read differently. This is a text-frontend difference,
not a model-quality problem; the GPT/codec/S2Mel stack itself matches the
official pipeline token-for-token on the validated cases.

## IndexTTS2.5

IndexTTS2.5 is IndexTeam/bilibili's multilingual zero-shot TTS model (released
2026-07): a 0.8B GPT autoregressive model + DiT CFM + BigVGAN stack that keeps
IndexTTS2's timbre-emotion decoupling and adds Japanese, Spanish, and Arabic on
top of Chinese and English. It requires a speaker reference through the
framework `--voice-ref` path. Inline `<文字|发音>` pronunciation overrides
(pinyin, CMU phonemes, or kana) are supported. Upstream weights live at
[IndexTeam/IndexTTS-2.5](https://huggingface.co/IndexTeam/IndexTTS-2.5); the
reference implementation is [index-tts/index-tts](https://github.com/index-tts/index-tts)
branch `indextts-2.5`.

IndexTTS2.5 is implemented as a variant of the `index_tts2` family rather than
a separate family: both variants share the audio features, wav2vec2bert, Qwen
emotion, style encoder, BigVGAN vocoder, S2Mel, and GPT decode/cache code. The
tokenizer, GPT speaker conditioning, and semantic-codec decode path are selected
per variant from the model config `version` field (`"2.5"`). All
`index_tts2.*` session options apply to both variants.

| Field | Value |
|---|---|
| Family | `index_tts2` |
| Variant | `2.5`, selected from the model config `version` field |
| Model directory | `models/IndexTTS2.5-GGUF` |
| Default package | `index_tts2_5_q8_0` |
| Other packages | `index_tts2_5_f16`, `index_tts2_5_orig` |
| Task | `tts`, `clon` |
| Modes | `offline` |
| Languages | `zh`, `en`, `ja`, `es`, `ar` |
| Voice input | Required reference WAV through `--voice-ref` |
| Built-in voices | Not exposed |

Voice clone:

```bash
audiocpp_cli --task clon --family index_tts2 \
  --model /path/to/IndexTTS2.5-GGUF \
  --backend cuda \
  --text "Hello from IndexTTS2.5." \
  --voice-ref /path/to/reference.wav \
  --out out.wav
```

Emotion text:

```bash
audiocpp_cli --task tts --family index_tts2 \
  --model /path/to/IndexTTS2.5-GGUF \
  --backend cuda \
  --text "今天的演示会更有情绪。" \
  --voice-ref /path/to/reference.wav \
  --emotion "你吓死我了！你是鬼吗？" \
  --request-option emotion_alpha=0.6 \
  --out out.wav
```

The `language` request option selects the text language (`auto`, `zh`, `en`,
`ja`, `es`, `ar`, or any tokenizer language code). The default `auto` picks
`zh` when the text contains Han characters and `en` otherwise, so mixed
Japanese, Spanish, or Arabic text should set
`--request-option language=ja|es|ar` explicitly.

Emotion conditioning supports all three IndexTTS2 paths: an emotion reference
WAV through `--audio`, an explicit `emotion_vector`, and Qwen-based emotion-text
classification through `--emotion` / `use_emotion_text`. See the IndexTTS2
text-normalization note above for the zh/en caveat. Known limitations: the
Spanish NeMo text normalizer is not ported, so es text passes through without
upstream-style normalization. The official ja path has no NeMo grammar, so NeMo
TN is a no-op for Japanese upstream; the official ja wakachi/fugashi word
segmentation is also not ported, so ja text is tokenized unsegmented. It is
intelligible, but not token-identical to the official pipeline.

License: IndexTTS-2.5 weights are distributed under the bilibili Model Use
License, which is not OSI-approved. It requires separate commercial
authorization when monthly active users exceed 100 million or annual revenue
exceeds 1 billion RMB, and it forbids using model outputs to improve other AI
models. Check the upstream repository for the full terms before redistribution
or commercial use.

### IndexTTS2.5 Common Options (use directly)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--voice-ref` | WAV path | required | Reference speaker audio. |
| `--audio` | WAV path | not set | Emotion-reference audio. |
| `--emotion` | text | not set | Emotion-text conditioning through the framework style field. |
| `--text-chunk-size` | characters | not set | Optional framework long-form text chunk size; omitted keeps the full input together. Must be positive. |
| `--text-chunk-mode` | `default`, `tag_aware`, `japanese`, `endline` | `default` | Framework chunking mode used only when `--text-chunk-size` is set. |
| `--max-tokens` | integer | `1500` | Maximum generated GPT mel tokens. |
| `--temperature` | float | `0.8` | Sampling temperature. Must be positive. |
| `--top-p` | float | `0.8` | Nucleus sampling probability. Must be in (0, 1]. |
| `--top-k` | integer | `30` | GPT top-k sampling limit. |
| `--repetition-penalty` | float | `10.0` | GPT repetition penalty. |
| `--do-sample` | `true`, `false` | `true` | Enable stochastic GPT sampling. |
| `--seed` | integer | random when omitted | Generation seed. |

### IndexTTS2.5 Request Options (use with `--request-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `text` | text | not set | Fallback target text when text_input is absent; prompt is also accepted. |
| `language` | `auto`, `zh`, `en`, `ja`, `es`, `ar`, ... | not set | Text language hint; `auto` infers `zh` when the text contains Han characters, otherwise `en`. |
| `emotion_alpha` | float in `[0, 1]` | `1` | Blend strength for explicit emotion conditioning. |
| `emotion_vector` | 8 floats | not set | Explicit emotion vector. |
| `use_emotion_text` | bool | `false` | Infer emotion from text. |
| `emotion_text` | text | not set | Text used when emotion-text conditioning is enabled. |
| `use_random_emotion` | bool | `false` | Use random emotion weights in the emotion mixer. |
| `interval_silence_ms` | milliseconds | `200` | Silence inserted between generated text chunks. |
| `do_sample` | bool | `true` | Enable sampling. |
| `top_p` | float | `0.8` | Nucleus sampling probability. Must be in (0, 1]. |
| `top_k` | integer >= 1 | `30` | Top-k sampling limit. |
| `temperature` | float | `0.8` | Sampling temperature. Must be positive. |
| `length_penalty` | float | `0` | GPT beam-search length penalty. |
| `num_beams` | integer | `3` | GPT beam count. |
| `repetition_penalty` | float | `10` | Sampling repetition penalty. |
| `max_tokens` | integer >= 1 | `1500` | Maximum generated mel tokens. |
| `duration_scale` | positive float | `1` | Positive waveform duration rescaling, supported by both IndexTTS2 and IndexTTS2.5. |
| `seed` | integer >= 0 | random | Sampling seed; omitted chooses a random seed. |
| `text_chunk_size` | integer | not set | Optional framework long-form text chunk size; omitted keeps the full input together. Must be positive. |
| `text_chunk_mode` | `default`, `tag_aware`, `japanese`, `endline` | `default` | Framework text chunking mode. |

### IndexTTS2.5 Session Options (use with `--session-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `index_tts2.weight_type` | `native`, `f32`, `f16`, `bf16`, `q8_0` | not set | Matmul weight storage type. |
| `index_tts2.conv_weight_type` | `native`, `f32`, `f16` | `native` | Convolution weight storage type. |
| `index_tts2.mem_saver` | bool | `false` | Release staged reference and conditioning graphs after request phases. |
| `index_tts2.speaker_cache_slots` | integer slots | `1` | Prepared speaker-reference cache slots; set `0` to disable reuse. |
| `index_tts2.emotion_cache_slots` | integer slots | `1` | Prepared emotion-reference cache slots; set `0` to disable reuse. |
| `index_tts2.emotion_text_cache_slots` | integer slots | `1` | Emotion-text weight cache slots; set `0` to disable reuse. |
| `index_tts2.gpt_graph_arena_mb` | MB | `32` | GPT graph arena size. |
| `index_tts2.s2mel_graph_arena_mb` | MB | `32` | S2Mel graph arena size. |
| `index_tts2.reference_graph_arena_mb` | MB | `512` | Reference encoder and codec graph arena size. |
| `index_tts2.emotion_text_prefill_graph_arena_mb` | MB | `2048` | Emotion-text prefill graph arena size. |
| `index_tts2.emotion_text_decode_graph_arena_mb` | MB | `512` | Emotion-text cached-step graph arena size. |
| `index_tts2.weight_context_mb` | MB | `32` | Shared ggml weight metadata context size. |
| `index_tts2.emotion_text_max_tokens` | tokens | `256` | Maximum generated tokens for emotion-text classification; old name `index_tts2.emotion_text_max_new_tokens` is still accepted. |

## Converting IndexTTS2.5 From Upstream Weights

`tools/community_models/convert_index_tts2_5.py` turns an official `IndexTeam/IndexTTS-2.5`
snapshot (the `.pth` checkpoints) into the Safetensors staging layout the
engine expects, and prints or runs the matching `audiocpp_gguf` command. The
w2v-bert-2.0, CAMPPlus, and BigVGAN checkpoints are auto-detected under
`<model-dir>/hf_cache/` after running official inference once to populate it,
and each has an explicit override flag:

```bash
python tools/community_models/convert_index_tts2_5.py \
    --model-dir /path/to/IndexTTS-2.5 \
    --output-dir /path/to/staging \
    --run-converter /path/to/audiocpp_gguf --type q8_0
```

Pass `--native-dir /path/to/IndexTTS-2.5-native` to also emit a directly
loadable native Safetensors model directory hardlinked from the staging files,
without GGUF conversion.

The script repackages the checkpoints the loader needs, unwraps the
`s2mel.pth`/`codec.pth` container keys, prefixes CAMPPlus tensors with
`speaker_encoder.`, strips BigVGAN's `generator.` prefix, wraps the
`feat1/feat2.pt` matrices as a single `tensor`, and assembles the sidecar
`root/` config/tokenizer resources that get embedded into the GGUF. The staged
`config.yaml` has its `version` field normalized to `"2.5"` because the
official snapshot ships `version: 2.0`; the engine uses that field to select the
IndexTTS2 family variant.