Fabio
c36510c21e
Some checks failed
Linux Build / Linux cpu (push) Has been cancelled
Linux Build / Linux cpu-clang (push) Has been cancelled
Linux Build / Linux vulkan (push) Has been cancelled
mac-build / macOS CPU (push) Has been cancelled
Nix Build / Nix (cpu) (push) Has been cancelled
Nix Build / Nix (vulkan) (push) Has been cancelled
windows-build / Windows CPU (push) Has been cancelled
104 lines
5.1 KiB
Markdown
104 lines
5.1 KiB
Markdown
# Audio8-ASR-0.1B in audio.cpp
|
||
|
||
[Audio8-ASR-0.1B](https://huggingface.co/Audio8/Audio8-ASR-0.1B) is a compact
|
||
multilingual autoregressive ASR model (en / zh / yue / ja / ko / fr / de): a
|
||
Qwen3-ASR audio encoder adapted by an MLP tower into an 8-layer Qwen2-style
|
||
decoder with only ~103M language-model parameters (324M end-to-end). The
|
||
checkpoint ships under **CC-BY-NC-4.0**, so audio.cpp loads it from locally
|
||
converted weights only; the converted GGUF must not be redistributed.
|
||
|
||
## Architecture
|
||
|
||
- **Audio frontend**: 16 kHz Whisper log-mel, 128 bins, hop 160, n_fft 400
|
||
(shared with the `qwen3_asr` family). The reference processor emits
|
||
bfloat16 mel values; the audio8_asr frontend rounds to bfloat16 before
|
||
encoding to match.
|
||
- **Audio encoder**: Qwen3-ASR audio tower (d_model 896, 18 layers, 14 heads,
|
||
FFN 3584, 128 mel bins, output dim 1024) — bit-for-bit the same
|
||
architecture as `Qwen/Qwen3-ASR-0.6B`, loaded through the shared
|
||
`qwen3_asr` encoder implementation.
|
||
- **Adapter**: 4 pre-norm residual MLP blocks (1024 → 4096, erf GELU), a
|
||
final LayerNorm, then an adaptive average pool (merge factor 4 against the
|
||
mel-frame count) followed by LayerNorm + Linear(1024 → 512).
|
||
- **Decoder**: Qwen2-style causal LM, 8 layers, hidden 512, 8 heads,
|
||
head_dim 64, SwiGLU FFN 1408, tied embeddings over a 151,936 Qwen BPE
|
||
vocabulary, RoPE theta 1e6, RMSNorm eps 1e-6.
|
||
- **Prompt**: `<|user|><|begin_of_audio|><|audio|>×N<|end_of_audio|>Please
|
||
transcribe this audio.<|assistant|>` where
|
||
`N = max(floor(floor((floor(samples / hop) + 1) / 2) / merge_factor), 1)`
|
||
(all divisions integer).
|
||
|
||
## Usage
|
||
|
||
```bash
|
||
# Convert locally (requires the audiocpp_gguf tool and a self-downloaded
|
||
# checkpoint from the Audio8/Audio8-ASR-0.1B HF repository):
|
||
python tools/community_models/audio8_asr/convert_audio8_asr.py \
|
||
--checkpoint models/Audio8-ASR-0.1B-hf \
|
||
--converter build/debug/bin/audiocpp_gguf \
|
||
--type q8_0 \
|
||
--output models/Audio8-ASR-0.1B-GGUF/audio8-asr-0.1b-q8_0.gguf
|
||
|
||
# Transcribe
|
||
audiocpp_cli --task asr --family audio8_asr \
|
||
--model models/Audio8-ASR-0.1B-GGUF/audio8-asr-0.1b-q8_0.gguf \
|
||
--audio sample.wav
|
||
|
||
# The safetensors package loads directly, no conversion required:
|
||
audiocpp_cli --task asr --family audio8_asr \
|
||
--model models/Audio8-ASR-0.1B-hf --audio sample.wav
|
||
```
|
||
|
||
## Parity
|
||
|
||
`tools/community_models/audio8_asr/audio8_asr_reference.py` runs the Hugging Face
|
||
`trust_remote_code` reference (torch CPU, fp32) and
|
||
`tools/community_models/audio8_asr/audio8_asr_stages.py` captures staged tensors (mel,
|
||
encoder output, projected audio embeddings) for comparison.
|
||
|
||
Greedy transcription on the repo test clips matched the fp32 reference
|
||
exactly with the Q8_0 GGUF on both the Metal and CPU backends:
|
||
|
||
| Audio | Reference (fp32) | audio.cpp (Q8_0 GGUF, Metal + CPU) |
|
||
|---|---|---|
|
||
| `assets/resources/a.wav` (5.95 s) | "This little work was finished in the year eighteen o three, and intended for immediate publication." | identical |
|
||
| `assets/resources/sample_16k.wav` (14.07 s) | "Some call me nature. Others call me Mother Nature. I've been here for over four point five billion years, twenty-two thousand five hundred times longer than you." | identical |
|
||
|
||
A 61-second clip transcribed through rate-correct 30-second windows matched
|
||
the per-window reference transcripts (also verified by an independent review
|
||
pass). `test_audio8_asr_golden_transcription` asserts the first row
|
||
end-to-end — run it manually once weights exist (it is not part of ctest,
|
||
matching the granite5asr golden-test convention); `test_audio8_asr_units`
|
||
covers the token-count formula and bfloat16 rounding and runs under ctest.
|
||
|
||
## Measured performance
|
||
|
||
Release build (`-DCMAKE_BUILD_TYPE=Release`), Apple M4, Metal backend,
|
||
Q8_0 GGUF (345 MB weights):
|
||
|
||
| Audio | Session wall (`session.wall_ms`) | Effective RTF | CLI wall (incl. ~2.2 s process + load) |
|
||
|---|---|---|---|
|
||
| 5.95 s | ~0.75 s | ~8x realtime | 2.2 s |
|
||
| 14.07 s | 757 ms | 18.6x realtime | 3.8 s |
|
||
| 61 s (3 windows) | 3.46 s | 17.6x realtime | 5.8 s |
|
||
|
||
The audio encoder dominates (~64% of session time; its Metal shaders are
|
||
insensitive to build type, so Debug and Release measure within 3%).
|
||
Peak RSS is ~1.0 GB for a single clip and ~1.3 GB for a three-window
|
||
transcription (Metal buffers + per-window-shape graph pools on top of the
|
||
345 MB of weights; CPU backend measures the same ~1.0 GB).
|
||
|
||
## Known limitations
|
||
|
||
- **Offline only**: no streaming mode, no word timestamps, no language-id
|
||
output.
|
||
- **30-second windows**: audio longer than 30 s is transcribed in fixed
|
||
windows sized at the input sample rate (0.5 s minimum tail folds into the
|
||
previous window) and space-joined, without VAD segmentation. Unlike the
|
||
single-pass reference, each window is peak-normalized independently, so
|
||
relative loudness across a window boundary can shift.
|
||
- **CC-BY-NC-4.0**: non-commercial use only; convert locally, do not
|
||
redistribute the converted GGUF. There is no release GGUF package and no
|
||
WebUI catalog entry (the package manager cannot download
|
||
unsupported-license packages).
|
||
- Hotword logit boosting from the reference implementation is not ported.
|