# ControlFoley

ControlFoley is a multimodal Foley generator. It can generate sound from text
alone, from video alone, from text plus video, or from reference audio plus
video.

## Quick Start

Text-to-audio:

```bash
audiocpp_cli \
  --task gen \
  --family controlfoley \
  --model models/ControlFoley-GGUF/controlfoley-large-44k-f32.gguf \
  --backend cuda \
  --text "A bird sings melodically in a forest." \
  --request-option duration_sec=10 \
  --request-option num_inference_steps=25 \
  --request-option guidance_scale=4.5 \
  --request-option seed=123 \
  --out foley.wav
```

Video-conditioned generation:

```bash
audiocpp_cli \
  --task gen \
  --family controlfoley \
  --model models/ControlFoley-GGUF/controlfoley-large-44k-f32.gguf \
  --backend cuda \
  --text "skateboarding" \
  --request-option video_file=/path/to/video.mp4 \
  --request-option duration_sec=8 \
  --request-option num_inference_steps=25 \
  --request-option guidance_scale=4.5 \
  --out foley_video.wav
```

## Model

| Field | Value |
|---|---|
| Family | `controlfoley` |
| Task | `gen` |
| Mode | `offline` |
| Packages | `controlfoley-large-44k-f32.gguf`, `controlfoley-large-44k-q8_0.gguf` |
| Text input | Optional prompt through `--text` |
| Video input | Optional video path through `--request-option video_file=<path>` |
| Audio input | Optional reference WAV through `--audio` |

## Paths

| Path | Required inputs |
|---|---|
| T2A | `--text` |
| TV2A | `--text`, `--request-option video_file=<path>` |
| TC-V2A | `--text`, `--request-option video_file=<path>`, `--request-option mask_away_clip=true` |
| AC-V2A | `--audio`, `--request-option video_file=<path>` |
| V2A | `--request-option video_file=<path>` |

## Common Options (use directly)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `--text` | prompt text | optional | Text conditioning prompt. |
| `--audio` | WAV path | optional | Reference audio for audio-conditioned video generation. |

## Request Options (use with `--request-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `video_file` | video path | optional | Video conditioning source. Legacy `video` remains accepted. |
| `duration_sec` | float | `8` | Target temporal budget. |
| `num_inference_steps` | integer > 0 | `25` | Euler flow inference steps. |
| `guidance_scale` | float > 0 | `4.5` | Classifier-free guidance strength. Must be > 0. Values strictly between 0 and 1 bypass CFG and negative-prompt conditioning. |
| `negative_prompt` | text | empty | Negative text branch for CFG. |
| `mask_away_clip` | bool | `false` | Disable OpenCLIP video conditioning while keeping visual and sync conditioning. |
| `seed` | integer >= 0 | `42` | Sampler seed. |

## Session Options (use with `--session-option`)

| Option | Values | Default | Meaning |
|---|---|---:|---|
| `controlfoley.weight_type` | `native`, `f32`, `f16`, `bf16`, `q8_0` | `native` | Weight storage override for experiments. |