voice_audio_multi/docs/models/firered_vad.md

FireRed VAD

Native GGML inference for the FireRedTeam/FireRedVAD offline VAD and causal Stream-VAD checkpoints. Both models take 16 kHz mono audio and return speech segments. Stream-VAD also emits speech-start and speech-end events. The original code and weights are Apache-2.0 licensed.

Build and Convert

Run these commands from the repository root:

scripts/build_metal.sh --build-dir build/firered-vad --build-type Release \
  --model-set custom --models firered_vad --with-tests \
  --target audiocpp_cli --target audiocpp_gguf --target firered_vad_probe

Download VAD/model.pth.tar, VAD/cmvn.ark, Stream-VAD/model.pth.tar and Stream-VAD/cmvn.ark from FireRedTeam/FireRedVAD on Hugging Face. Preserve the VAD/ and Stream-VAD/ directories under <source-dir>. The converter needs torch, numpy, safetensors and kaldiio.

python tests/firered_vad/convert_gguf.py \
  --source <source-dir>/VAD --type f32 \
  --converter build/firered-vad/bin/audiocpp_gguf \
  --output models/FireRedVAD-GGUF/firered-vad-f32.gguf

python tests/firered_vad/convert_gguf.py \
  --source <source-dir>/Stream-VAD --type f32 \
  --converter build/firered-vad/bin/audiocpp_gguf \
  --output models/FireRedVAD-GGUF/firered-stream-vad-f32.gguf

Only F32 packages are provided: F16 and Q8 offer no meaningful performance or peak-VRAM benefit for these small models, and Q8 adds numerical drift. The converter still accepts f32, f16 and q8_0 for experimentation. Each GGUF contains its weights, FP32 CMVN coefficients, architecture configuration and v1 model spec. The family is marked experimental while converted packages are prepared for publication.

Run

Offline VAD:

build/firered-vad/bin/audiocpp_cli --task vad --family firered_vad \
  --model models/FireRedVAD-GGUF/firered-vad-f32.gguf \
  --backend metal --audio assets/resources/sample_16k.wav --metrics --log

Streaming with the causal checkpoint:

build/firered-vad/bin/audiocpp_cli --task vad --mode streaming --family firered_vad \
  --model models/FireRedVAD-GGUF/firered-stream-vad-f32.gguf \
  --backend metal --audio assets/resources/sample_16k.wav --log

Use --backend cpu --threads <count> for CPU inference. Thread count controls parallelism; the CPU graph and cache layout are the same across thread counts. CPU depthwise convolution uses channel-contiguous time blocks, and eligible matrix multiplications use GGML BLAS when available, following Moonshine's scheduler pattern. The requested thread count is passed to both backends; vendor BLAS libraries may manage their own worker threads. Input chunks use 160-sample hops with a retained 240-sample overlap for the 400-sample Kaldi window. The default transport chunk is 2560 samples.

The offline checkpoint includes lookahead; the causal checkpoint supports both full-input and streaming runs. Streaming resets caches and segmentation state at the start of each request.

Options

Pass options with --request-option name=value. Duration options are rounded to the nearest 10 ms model frame; halfway values round up.

Option	VAD default	Stream-VAD default
threshold	0.4	0.5
smooth_window_ms	50	50
min_speech_duration_ms	200	80
max_speech_duration_sec	20	20
min_silence_duration_ms	200	200
merge_silence_duration_ms	0	Offline VAD only
speech_pad_ms	0	Offline VAD only
speech_start_pad_ms	Stream-VAD only	50
audio_chunk_duration_sec	300	300 for full-input runs

Full-input runs split network input at audio_chunk_duration_sec, resetting network history between chunks as upstream does. Live streaming retains network history.

Validation

The reference tools use the official Python implementation, including Kaldi features, CMVN, frame probabilities and segment rules. Install soundfile and kaldi-native-fbank as well as the conversion dependencies. --gguf additionally requires the Python gguf package and compares against dequantized package weights to separate quantization drift from backend numerical error.

python tests/firered_vad/python_reference.py \
  --reference <FireRedVAD-source-repo> --source <source-dir>/VAD \
  --audio assets/resources/sample_16k.wav --threads 1 \
  --output-prefix build/firered-vad/reference --device cpu --log

build/firered-vad/bin/firered_vad_probe \
  models/FireRedVAD-GGUF/firered-vad-f32.gguf \
  assets/resources/sample_16k.wav metal 8 \
  build/firered-vad/native build/firered-vad/reference --log

python tests/firered_vad/check_parity.py \
  --reference build/firered-vad/reference --native build/firered-vad/native

The probe checks streaming/full-input agreement for causal checkpoints and exports features, probabilities and timestamps. It measures frontend, network and complete detection separately, with five warmups and the median of 21 measured runs. It accepts cpu, cuda, vulkan, and metal; the Python reference accepts --device cpu or --device cuda and synchronizes CUDA timing. For causal regression coverage, also run the official TEST_MEETING_T0000000001_S00000.wav example with the Stream-VAD checkpoint: its near-threshold speech boundary exposes small-matrix precision loss that the shorter examples do not. The default parity checks bound normalized frontend RMSE by 1e-5, frontend maximum error by 1e-3, frame-probability error by 0.002, and require exact timestamps.

Unit tests cover upstream postprocessing fixtures and sparse/dense Kaldi frontend equivalence. Sparse projection is opt-in and disabled by default for existing callers.

cmake --build build/firered-vad --target \
  firered_vad_rules_test kaldi_fbank_sparse_test
ctest --test-dir build/firered-vad \
  -R 'firered_vad_rules_test|kaldi_fbank_sparse_test' \
  --output-on-failure

Measured results are recorded in validation.md.