# MiraTTS

MiraTTS is an experimental community port of
[ysharma3501/MiraTTS](https://github.com/ysharma3501/MiraTTS), a zero-shot
voice-cloning text-to-speech model. The native path includes the Qwen2 speech
token generator, ECAPA-TDNN plus Perceiver reference encoder, finite-scalar
speaker tokenizer, conditional acoustic processor, and DAC waveform decoder.

## Model and license

The upstream checkpoint is
[YatharthS/MiraTTS](https://huggingface.co/YatharthS/MiraTTS). Its model card
declares `CC-BY-NC-SA-4.0`; this is a non-commercial, attribution, share-alike
license. Review that license before downloading or redistributing converted
weights. audio.cpp does not redistribute the checkpoint.

No ready-to-run GGUF package is published yet, so MiraTTS intentionally has no
entry in the built-in download catalog. Convert a locally obtained upstream
checkpoint with:

```bash
python tools/community_models/convert_mira_tts.py /path/to/MiraTTS /path/to/mira-native
audiocpp_gguf \
  --input language_model=/path/to/mira-native/language_model.safetensors \
  --input speaker_encoder=/path/to/mira-native/speaker_encoder.safetensors \
  --input processor=/path/to/mira-native/processor.safetensors \
  --input decoder=/path/to/mira-native/decoder.safetensors \
  --input upsampler=/path/to/mira-native/upsampler.safetensors \
  --output /path/to/mira-native/mira-tts.gguf --type bf16 \
  --family mira_tts --root /path/to/mira-native
```

Use BF16 for the first parity-oriented conversion. Converting the natively
BF16 Qwen backbone to F16 can overflow and produce non-finite logits.

The converter also imports the official FastBiCodec and FlashSR component
checkpoints referenced by the upstream repository. Use `--help` to see its
component path overrides.

## Run

MiraTTS requires reference audio. The CLI voice-cloning request accepts the
converted model directory, target text, and a short clean reference WAV through
the normal audio.cpp TTS/clone arguments. Sampling defaults reproduce upstream:
temperature `0.8`, top-k `50`, top-p `0.95`, min-p `0.05`, and repetition
penalty `1.2`.

The upstream pipeline decodes at 16 kHz and applies its learned FlashSR
upsampler. The native runtime executes both stages and returns 48 kHz audio.

MiraTTS also exposes a streaming session. It splits long input at natural text
boundaries, reuses one encoded speaker identity for the whole request, and emits
each completed 48 kHz segment immediately. `text_chunk_size` controls the
maximum segment size (160 codepoints by default), while `text_chunk_mode`
selects the framework chunker. This is segment-level progressive synthesis;
the acoustic processor, DAC, and FlashSR still decode each segment as a unit.

The session caches one encoded reference voice by default, so repeated requests
with the same audio do not rerun the speaker encoder. Increase the bounded cache
with `--session-option reference_cache_slots=<n>`, or set it to `0` to disable
reference reuse.

## Session weight options

Use these with `--session-option`; storage is selected at session creation.
The legacy unprefixed names remain accepted.

| Option | Default | Meaning |
|---|---|---|
| `mira_tts.backbone_weight_type` | `native` | Language-model weight storage. |
| `mira_tts.linear_weight_type` | `native` | Codec linear weight storage. |
| `mira_tts.conv_weight_type` | `f32` | Codec convolution weight storage. |

## Validation status

- Official checkpoint conversion: validated.
- Native CUDA build: validated.
- Native CUDA smoke synthesis through all converted model components: validated.
- Segment-level streaming synthesis: validated through the native streaming
  session and `/v1/audio/speech/live` route.
- Deterministic upstream comparison with identical speech/context tokens:
  validated (48 kHz waveform correlation 0.99996, SNR 41.1 dB).
- End-to-end generation comparison: validated through matching tokenization,
  identical 32-token speaker codes, and the first six greedy LM tokens. Later
  autoregressive tokens can diverge between LMDeploy, Transformers, and the
  native backend because of backend floating-point differences.

Until end-to-end measurements are published, the family remains experimental
and is not advertised as an installable WebUI package.