# FireRedAudio

FireRedAudio is a multimodal speech and audio model. The native audio.cpp path
covers Chinese ASR, audio understanding, TTS cloning, voice design, semantic
editing, and acoustic editing from a single GGUF package.

## Quick Start

Chinese ASR:

```bash
audiocpp_cli \
  --task asr \
  --family firered_audio \
  --model models/FireRedAudio-GGUF/firered-audio-orig.gguf \
  --backend cuda \
  --audio input.wav \
  --text "Transcribe speech to text." \
  --request-option template_name=asr \
  --text-out transcript.txt
```

Voice cloning:

```bash
audiocpp_cli \
  --task clon \
  --family firered_audio \
  --model models/FireRedAudio-GGUF/firered-audio-orig.gguf \
  --backend cuda \
  --language zh \
  --text "安徽淮南秦师傅发现，停在小区的爱车右前驾驶窗玻璃被砸。" \
  --voice-ref reference/FireRedAudio/assets/examples/tts_zh_prompt.wav \
  --reference-text "同时，他强调微调要科学有序。" \
  --request-option template_name=tts_clone \
  --out firered_audio_clone.wav
```

Audio understanding:

```bash
audiocpp_cli \
  --task asr \
  --family firered_audio \
  --model models/FireRedAudio-GGUF/firered-audio-orig.gguf \
  --backend cuda \
  --audio input.wav \
  --text "What is happening in this audio?" \
  --request-option template_name=understand \
  --request-option enable_thinking=true \
  --request-option max_tokens=1024 \
  --text-out answer.txt
```

## Model

| Field | Value |
|---|---|
| Family | `firered_audio` |
| Tasks | `asr`, `tts`, `clon`, `vdes`, `gen` |
| Mode | `offline` |
| Default package | `models/FireRedAudio-GGUF/firered-audio-q8_0.gguf` |
| Generation paths | `tts_clone`, `voice_design`, `semantic_edit`, `acoustic_edit` |
| Text paths | `asr`, `understand` |

## Templates

| `template_name` | Task | Inputs |
|---|---|---|
| `asr` | `asr` | `--audio`, optional prompt through `--text` |
| `understand` | `asr` | `--audio`, question through `--text` |
| `tts_clone` | `clon` | `--text`, `--voice-ref`, `--reference-text` |
| `voice_design` | `vdes` | `--text`, `--request-option instruct=<text>` |
| `semantic_edit` | `gen` | `--audio`, `--request-option instruct=<text>` |
| `acoustic_edit` | `gen` | `--audio`, `--request-option instruct=<text>` |

## Common Options (use directly)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--language` | language tag | `zh` | Generation or text-path language tag. |
| `--voice-ref` | WAV path | required for clone | Prompt/reference voice. |
| `--reference-text` | text | empty | Transcript for the prompt audio. |

## Request Options (use with `--request-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `template_name` | `asr`, `understand`, `tts_clone`, `voice_design`, `semantic_edit`, `acoustic_edit` | see description | Defaults to voice_design for design, acoustic_edit for generation with input audio, tts_clone for other generation, and asr for ASR. |
| `language` | language tag | `zh` | Generation or text-path language tag. |
| `reference_text` | text | empty | Transcript for the prompt audio. |
| `instruct` | text | required for design/edit | Generation voice/delivery instruction. Legacy `instruction` remains accepted. ASR and understanding instead use the text input as their task prompt. |
| `num_inference_steps` | integer > 0 | `10` | FireRedAudio DiT flow steps per latent patch. |
| `guidance_scale` | float > 0 | `2.0` | FireRedAudio DiT CFG scale. |
| `max_audio_steps` | integer > 0 | `750` | Maximum generated RedAE latent patches. Legacy `max_new_audio_steps` remains accepted. |
| `min_audio_steps` | integer >= 0 | `6` | Minimum audio patches before stop may be accepted. Legacy `min_new_audio_steps` remains accepted. |
| `max_text_tokens` | integer > 0 | `512` | Text-mode token budget before audio mode. Legacy `max_new_text_tokens` remains accepted. |
| `max_tokens` | integer > 0 | see description | Output token budget; defaults to 300, or 1024 for understand with enable_thinking=true. |
| `enable_thinking` | bool | `false` | Enable open thinking block for `understand`; invalid for plain `asr`. |
| `top_k` | integer >= 0 | `20` | Understanding top-k sampling. |
| `top_p` | `0..1` | `0.8` | Understanding nucleus sampling. |
| `temperature` | float >= 0 | `0.7` | Text sampling temperature for understanding. Understanding-mode sampling requires a positive value; zero does not select greedy decoding. |
| `seed` | integer >= 0 | `1234` | Generation seed. |

## Session Options (use with `--session-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `firered_audio.reference_cache_slots` | integer >= 0 | `2` | Prepared reference-audio cache slots. |
| `firered_audio.mem_saver` | bool | `false` | Release runtime graphs after request phases. |
| `firered_audio.weight_type` | `native`, `f32`, `f16`, `bf16`, `q8_0` | `native` | Weight storage override for experiments. |

Legacy `firered_audio.helper_graph_arena_mb` remains accepted but has no effect
on allocation; it is not a memory control.