# LiveAvatar

LiveAvatar is an audio-to-video model that generates an animated avatar from a
reference image, an audio file, and a text description. The native runtime uses the Wan2.2 S2V
components with LiveAvatar blockwise generation.

## Run

The following configuration generates two clips at 720p and 16 FPS while using
the low-VRAM denoiser path:

```bash
build/debug/bin/audiocpp_cli \
  --task gen \
  --family liveavatar \
  --model /path/to/Wan2.2-S2V \
  --backend cuda \
  --threads 8 \
  --audio /path/to/reference.wav \
  --text "A detailed description of the speaker and scene." \
  --load-option "liveavatar.denoiser_gguf=models/unet/Qwen 2SV/Wan2.2-S2V-14B-NVFP4-LORA.gguf" \
  --session-option liveavatar.denoiser_weight_streaming=true \
  --request-option generation_mode=liveavatar \
  --request-option reference_image_file=/path/to/reference.jpg \
  --request-option height=720 \
  --request-option width=1280 \
  --request-option seed=420 \
  --request-option num_clips=2 \
  --request-option denoiser_layerwise=true \
  --request-option denoiser_layerwise_batch=16 \
  --request-option target_cache_blocks=1 \
  --request-option vae_encoder_chunk_size=4 \
  --request-option vae_decoder_tile_size=320 \
  --out-dir outputs/liveavatar \
  --log \
  --log-file outputs/liveavatar/run.log
```

Use `num_clips`, not `num_clip`. With the default 48-frame clip size, two clips
produce 93 output frames after block overlap is resolved.

## Memory Control

`liveavatar.denoiser_weight_streaming` is an opt-in session option. It is
selected when the model session is created and cannot be changed per request
without reloading the session.

- `false` keeps the denoiser weights resident on the accelerator and provides
  the fastest path when enough VRAM is available.
- `true` keeps the denoiser transformer-block weights in pinned host memory,
  stages one layer group at a time, and keeps the immutable condition KV cache
  in host memory after it is built. Common denoiser weights and active compute
  state remain on the accelerator.

Weight streaming requires both:

```bash
--request-option generation_mode=liveavatar
--request-option denoiser_layerwise=true
```

`denoiser_layerwise_batch` controls the number of transformer layers staged at
once. Larger values improve throughput but use more VRAM. Start with `16`; lower
it when the accelerator cannot fit that group.

The VAE has separate memory controls. `vae_encoder_chunk_size` limits encoder
work per chunk, while `vae_decoder_tile_size` enables tiled spatial decoding.
For 720p generation near a 16 GiB limit, use encoder chunks of `4`, decoder tiles
of `320`, and `target_cache_blocks=1` as shown above.

## Options

Request options (use with `--request-option`):

| Option | Default | Notes |
|---|---:|---|
| `generation_mode` | `offline` | Use `liveavatar` for blockwise audio-driven generation. |
| `reference_image_file` | required | Reference image used for identity and appearance. Legacy `reference_image_path` and `ref_image` remain accepted. |
| `height` / `width` | `480` / `832` | Output frame dimensions; each must be a positive multiple of 16. |
| `num_frames` | `48` | Positive frame count per segment; must be a multiple of 4 in blockwise mode (streaming or `generation_mode=liveavatar`). |
| `num_clips` | `1` | Maximum number of audio-driven clips to generate. |
| `num_inference_steps` | `4` | UniPC steps offline, Euler steps in blockwise mode; the LiveAvatar adapter is designed for four steps. |
| `shift` | `3` | Flow scheduler shift; must be > 0. |
| `guidance_scale` | `0` | LiveAvatar blockwise generation requires zero guidance. |
| `batched_cfg` | `true` | Run CFG branches in one denoiser graph. Legacy `fused_cfg` and `liveavatar.fused_cfg` remain accepted; the new name takes precedence. |
| `seed` | `420` | Generation seed. |
| `mem_saver` | `true` | Reuse immutable condition data and temporary denoiser graph storage between blocks. Legacy `memory_saver` and `liveavatar.memory_saver` remain accepted. |
| `denoiser_layerwise` | `false` | Required by denoiser weight streaming. Offline generation requires `batched_cfg=true` and `guidance_scale > 1`. |
| `denoiser_layerwise_batch` | `16` | Number of denoiser layers staged and evaluated together. |
| `target_cache_blocks` | `0` | Target KV cache window; `1` keeps one block. |
| `vae_encoder_chunk_size` | `16` | VAE encoder chunk size. |
| `vae_decoder_tile_size` | `0` | VAE decoder tile size; zero disables tiling. Enabled sizes must be multiples of 8 and greater than the 64-pixel overlap. |

Session options (use with `--session-option`):

| Option | Default | Notes |
|---|---:|---|
| `liveavatar.denoiser_weight_streaming` | `false` | Stream denoiser block weights and keep the completed condition KV cache in host memory. |

Load options (use with `--load-option`):

| Option | Default | Notes |
|---|---:|---|
| `support_gguf` | package default | Override the support component path relative to the model root. |
| `vae_gguf` | package default | Override the VAE component path relative to the model root. |
| `denoiser_gguf` | package default | Override the denoiser component path relative to the model root. |